An autonomous intelligent robot system for home-based elderly care and construction method

Abstract

The present invention provides an autonomous intelligent robot system for home-based elderly care and a construction method thereof, comprising: collecting multi-channel signals, capturing user data, obtaining multi-channel data of user emotions and behaviors, and performing data vector splicing to form a multi-channel data splicing complete vector; performing short-term emotion and behavior understanding and long-term emotion and behavior understanding according to the multi-channel data splicing complete vector, analyzing hierarchical emotion and behavior understanding, and obtaining hierarchical emotion and behavior understanding results; triggering a corresponding time-sensitive multimodal response of the robot system according to the hierarchical emotion and behavior understanding results; outputting hierarchical emotion and behavior decisions through a multimodal output channel, and executing dual-space parallel tasks in physical space and virtual space; performing autonomous driving level evaluation and task completion evaluation to evaluate the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimizing the autonomous driving level and task completion of the decision-making autonomous intelligent robot system for home-based elderly care.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis intelligent elderly care information testing, and more specifically, to an autonomous intelligent robot system and method for home-based elderly care. Background Art

[0002] At present, most intelligent service robots or companion robots have weak autonomous decision-making capabilities and can only provide passive voice or video communication when interacting with users. Although such products can provide rich digital content and interactive modes, due to the lack of autonomy, it is difficult for them to become true partners with users. Therefore, it is difficult for them to play a greater role in providing emotional value. Currently, some researchers are working on improving the appearance of intelligent robots, among which digital virtual humans are the representative, aiming to improve the interactive experience by improving the realism of the appearance of virtual humans. This type of research has made great progress and has achieved good results in digital human reconstruction, multimodal interaction, etc. However, this type of virtual human is still at a primary level in terms of intelligence, even with a large language model (Large Language Model). The access of virtual human models has not yet completely overcome the problem of autonomous driving; improving the intelligence level of virtual people, especially the level of autonomous driving, is a key technical issue in improving the interactive experience of intelligent robots; at the same time, for robots with physical entities, this improvement in autonomy is also particularly important; the degree of robot autonomy largely determines the role it plays in society; therefore, how to solve the problem of robot autonomous driving has become an important problem in the field of robotics; for home-based elderly care, there are several new problems that need to be addressed; the elderly usually go out less, and their demand for social communication may be more significant, so robots need to be able to construct ways to enhance the social experience for the elderly; the mobility of the elderly group is usually lower than that of young people, so many daily life affairs (such as tidying the room, cooking, etc.) need robots to help deal with them to reduce the burden of life for the elderly; how to accurately infer the real needs of users, make decisions and reasonable behaviors that meet the expectations of users, and assist elderly users in solving emotional and behavioral difficulties and other problems remain to be solved; therefore, it is necessary to propose an autonomous intelligent robot system and method for home-based elderly care to at least partially solve the problems existing in the existing technology. Summary of the invention

[0003] A series of simplified concepts are introduced in the content of the invention, which will be further described in detail in the specific implementation method section; the content of the invention of the present invention does not mean to attempt to limit the key features and essential technical features of the technical solution claimed for protection, nor does it mean to attempt to determine the scope of protection of the technical solution claimed for protection.

[0004] In order to at least partially solve the above problems, the present invention provides an autonomous intelligent robot system for home-based elderly care, comprising:

[0005] The multi-channel acquisition vector splicing subsystem collects multi-channel signals, captures user data, obtains multi-channel data of user emotions and behaviors, and performs data vector splicing to form a multi-channel data splicing complete vector;

[0006] The short-term and long-term hierarchical reasoning subsystem splices complete vectors based on multi-channel data, performs short-term emotion and behavior understanding and long-term emotion and behavior understanding, analyzes hierarchical emotion and behavior understanding, and obtains hierarchical emotion and behavior understanding results;

[0007] The time-sensitive response parallel task subsystem triggers the robot system's corresponding time-sensitive multimodal response based on the hierarchical emotion and behavior understanding results; outputs hierarchical emotion and behavior decisions through the multimodal output channel, and executes dual-space parallel tasks in physical space and virtual space;

[0008] The decision-making task evaluation test subsystem conducts autonomous driving level assessment and task completion assessment, assesses the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimizes the decision-making autonomous driving level and task completion of the autonomous intelligent robot system for home-based elderly care.

[0009] Preferably, the multi-channel acquisition vector stitching subsystem includes:

[0010] The multimodal sensing and capturing subsystem collects multi-channel signals, captures user data, and obtains multi-channel data on user emotions and behaviors through the multimodal sensing module;

[0011] The data vector splicing subsystem splices the multi-channel data vectors of user emotions and behaviors to form a complete multi-channel data splicing vector.

[0012] Preferably, the short-term and long-term hierarchical reasoning subsystem includes:

[0013] The short-term inference and understanding subsystem splices complete vectors based on multi-channel data and extracts short-term observation data; infers the user's current emotional state and behavioral state, performs short-term emotional and behavioral understanding, and obtains short-term emotional and behavioral understanding results;

[0014] The long-term inference and understanding subsystem collects short-term observation data, forms long-term observation data, conducts long-term emotion and behavior understanding of the long-term observation data, and obtains long-term emotion and behavior understanding results;

[0015] The hierarchical reasoning and judgment subsystem performs hierarchical emotion and behavior understanding and obtains hierarchical emotion and behavior understanding results.

[0016] Preferably, the time-sensitive response parallel task subsystem includes:

[0017] The time-sensitive multimodal response subsystem triggers the robot system's corresponding time-sensitive multimodal response according to the robot's behavioral decision; the corresponding time-sensitive multimodal response includes: short-term emotional response or long-term emotional response;

[0018] The dual-space parallel task subsystem generates images, voices, and various physical behavior tasks in the physical space, as well as virtual interaction scene tasks in the virtual space, based on the corresponding time-sensitive multimodal responses, and executes dual-space parallel tasks in the physical space and virtual space.

[0019] Preferably, the decision-making task evaluation test subsystem includes:

[0020] The test embodied scenario building subsystem constructs diversified task test scenarios and virtual embodied interaction scenarios in a virtual test environment;

[0021] The robot decision-making evaluation subsystem evaluates the hierarchical emotion and behavior decision-making of the robot system based on autonomous emotion value according to diversified task test scenarios, evaluates the autonomous intelligent driving decision-making ability of the robot system, and obtains the decision-making evaluation results of the robot system;

[0022] The task completion evaluation subsystem performs task completion evaluation in the virtual embodied interaction scenario according to the task completion measurement indicators, evaluates the robot system's task completion capability, and obtains the task completion evaluation results;

[0023] The evaluation feedback continuously optimizes the subsystem. Based on the robot system decision-making evaluation results and task completion evaluation results, the decision-making autonomous driving level and task completion of the autonomous intelligent robot system for home-based elderly care are continuously optimized.

[0024] The present invention provides a method for constructing an autonomous intelligent robot system for home-based elderly care, comprising:

[0025] S100, collecting multi-channel signals, capturing user data, obtaining multi-channel data of user emotions and behaviors, and performing data vector splicing to form a multi-channel data splicing complete vector;

[0026] S200, splicing complete vectors based on multi-channel data, performing short-term emotion and behavior understanding and long-term emotion and behavior understanding, analyzing hierarchical emotion and behavior understanding, and obtaining hierarchical emotion and behavior understanding results;

[0027] S300, based on the hierarchical emotion and behavior understanding results, triggers the robot system to respond in a timely multimodal manner; outputs hierarchical emotion and behavior decisions through a multimodal output channel, and executes dual-space parallel tasks in physical space and virtual space;

[0028] S400 conducts autonomous driving level assessment and task completion assessment to assess the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimizes the autonomous driving level and task completion of the home-based elderly care autonomous intelligent robot system.

[0029] Preferably, S100 includes:

[0030] S101, through the multimodal perception module, collect multi-channel signals, capture user data, and obtain multi-channel data of user emotions and behaviors;

[0031] S102, splicing the multi-channel data vectors of user emotions and behaviors to form a multi-channel data splicing complete vector.

[0032] Preferably, S200 includes:

[0033] S201, splicing a complete vector based on multi-channel data, extracting short-term observation data; inferring the user's current emotional state and behavioral state, performing short-term emotional and behavioral understanding, and obtaining short-term emotional and behavioral understanding results;

[0034] S202, collecting short-term observation data to form long-term observation data, performing long-term emotion and behavior understanding on the long-term observation data, and obtaining long-term emotion and behavior understanding results;

[0035] S203, performing hierarchical emotion and behavior understanding to obtain hierarchical emotion and behavior understanding results.

[0036] Preferably, S300 includes:

[0037] S301, triggering a corresponding time-effect multimodal response of the robot system according to the robot behavior decision; the corresponding time-effect multimodal response includes: a short-term emotional response or a long-term emotional response;

[0038] S302, based on the corresponding time-sensitive multimodal response, generate images, voices and various physical behavior tasks in the physical space, as well as virtual interaction scene tasks in the virtual space, and execute dual-space parallel tasks in the physical space and the virtual space.

[0039] Preferably, S400 includes:

[0040] S401, constructing diversified task test scenarios and virtual embodiment interaction scenarios in a virtual test environment;

[0041] S402, based on the diversified task test scenarios, evaluate the hierarchical emotion and behavior decision-making of the robot system based on the autonomous emotion value, evaluate the autonomous intelligent driving decision-making ability of the robot system, and obtain the decision-making evaluation results of the robot system;

[0042] S403, in the virtual embodied interaction scene, performing task completion evaluation according to the task completion measurement index, evaluating the task completion capability of the robot system, and obtaining the task completion evaluation result;

[0043] S404, based on the robot system decision-making evaluation results and task completion evaluation results, continuously optimize the decision-making autonomous driving level and task completion of the autonomous intelligent robot system for home-based elderly care.

[0044] Compared with the prior art, the present invention has at least the following beneficial effects:

[0045] The present invention discloses an autonomous intelligent robot system for home-based elderly care and a construction method thereof. The system adopts a multi-channel acquisition vector splicing subsystem to collect multi-channel signals, capture user data, obtain multi-channel data of user emotions and behaviors, and perform data vector splicing to form a multi-channel data splicing complete vector; the short-term and long-term hierarchical reasoning subsystem performs short-term emotion and behavior understanding and long-term emotion and behavior understanding according to the multi-channel data splicing complete vector, analyzes hierarchical emotion and behavior understanding, and obtains hierarchical emotion and behavior understanding results; The time-sensitive response parallel task subsystem triggers the robot system's corresponding time-sensitive multimodal response according to the hierarchical emotion and behavior understanding results; outputs hierarchical emotion and behavior decisions through multimodal output channels, and executes dual-space parallel tasks in physical space and virtual space; the decision-making task evaluation test subsystem conducts autonomous driving level evaluation and task completion evaluation, evaluates the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimizes the autonomous driving level and task completion of the home-based elderly care autonomous intelligent robot system; based on long-term and short-term observation data, a hierarchical decision-making framework is constructed to integrate autonomous emotional value and behavioral value to deal with decision-making problems in complex situations; taking into account multimodal feedback from physical space and virtual space, the emotional companionship effect is enhanced through the operation of virtual space, and the physical assistance effect is guaranteed through the operation of physical space; constructs embodied interaction test scenarios, automatically matches according to expert-annotated data, and outputs autonomous levels and task completion; a hierarchical emotion understanding and behavior understanding framework based on short-term and long-term emotion understanding; with a variety of virtual Embodied test scenarios are used to test the autonomy and task completion of intelligent robots; the autonomy level of autonomous intelligent robots is quantified; intelligent assistance that integrates virtual feedback and physical feedback is used; an autonomous intelligent robot system is built for home-based elderly care groups, which can accurately analyze the behavioral characteristics of elderly users by combining short-term and long-term observation data, infer the real emotional needs of elderly users, and make reasonable feedback that takes into account physical assistance and virtual experience, thereby solving the problems of lack of emotional companionship and insufficient physical assistance faced by the elderly group in the process of home-based elderly care, and improving the autonomous service capabilities of existing technical solutions, thereby providing better emotional companionship and physical assistance; effect examples: when elderly users have not communicated for a long time and are in a low mood, the robot will take the initiative to communicate with them, share news or interesting events through sound and images, etc., to increase their willingness to communicate; when elderly users sit on the sofa for a long time without activity, the robot will take the initiative to prepare drinks, fruits, etc. for them, suggest getting up and moving for a while or going outdoors, and provide physical assistance during the human activity, etc.

[0046] The present invention describes an autonomous intelligent robot system for home-based elderly care and a construction method. Other advantages, objectives and features of the present invention will be partially reflected through the following description, and will also be partially understood by technicians in this field through research and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0048] Figure 1 This is a diagram of an embodiment of an autonomous intelligent robot system for home-based elderly care described in the present invention.

[0049] Figure 2 This is a diagram of another embodiment of the autonomous intelligent robot system for home-based elderly care described in the present invention.

[0050] Figure 3 This is a diagram of an embodiment of a method for constructing an autonomous intelligent robot system for home-based elderly care described in the present invention. DETAILED DESCRIPTION

[0051] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, so that those skilled in the art can implement it according to the instructions; as shown in the figure, the present invention provides an autonomous intelligent robot system for home-based elderly care, including:

[0052] The multi-channel acquisition vector splicing subsystem collects multi-channel signals, captures user data, obtains multi-channel data of user emotions and behaviors, and performs data vector splicing to form a multi-channel data splicing complete vector;

[0053] The short-term and long-term hierarchical reasoning subsystem splices complete vectors based on multi-channel data, performs short-term emotion and behavior understanding and long-term emotion and behavior understanding, analyzes hierarchical emotion and behavior understanding, and obtains hierarchical emotion and behavior understanding results;

[0054] The time-sensitive response parallel task subsystem triggers the robot system's corresponding time-sensitive multimodal response based on the hierarchical emotion and behavior understanding results; outputs hierarchical emotion and behavior decisions through the multimodal output channel, and executes dual-space parallel tasks in physical space and virtual space;

[0055] The decision-making task evaluation test subsystem conducts autonomous driving level assessment and task completion assessment, assesses the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimizes the decision-making autonomous driving level and task completion of the autonomous intelligent robot system for home-based elderly care.

[0056] The principle and effect of the above technical solution are as follows: the present invention provides an autonomous intelligent robot system for home-based elderly care, including: a multi-channel acquisition vector splicing subsystem, which collects multi-channel signals, captures user data, obtains multi-channel data of user emotions and behaviors, and performs data vector splicing to form a multi-channel data splicing complete vector; a short-term and long-term hierarchical reasoning subsystem, which performs short-term emotion and behavior understanding and long-term emotion and behavior understanding according to the multi-channel data splicing complete vector, analyzes hierarchical emotion and behavior understanding, and obtains hierarchical emotion and behavior understanding results; The time-sensitive response parallel task subsystem triggers the robot system's corresponding time-sensitive multimodal response according to the hierarchical emotion and behavior understanding results; outputs hierarchical emotion and behavior decisions through multimodal output channels, and executes dual-space parallel tasks in physical space and virtual space; the decision-making task evaluation test subsystem conducts autonomous driving level evaluation and task completion evaluation, evaluates the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimizes the autonomous driving level and task completion of the home-based elderly care autonomous intelligent robot system; based on long-term and short-term observation data, a hierarchical decision-making framework is constructed to integrate autonomous emotional value and behavioral value to deal with decision-making problems in complex situations; taking into account multimodal feedback from physical space and virtual space, the emotional companionship effect is enhanced through the operation of virtual space, and the physical assistance effect is guaranteed through the operation of physical space; constructs embodied interaction test scenarios, automatically matches according to expert-annotated data, and outputs autonomous levels and task completion; a hierarchical emotion understanding and behavior understanding framework based on short-term and long-term emotion understanding; with a variety of virtual Embodied test scenarios are used to test the autonomy and task completion of intelligent robots; the autonomy level of autonomous intelligent robots is quantified; intelligent assistance that integrates virtual feedback and physical feedback is used; an autonomous intelligent robot system is built for home-based elderly care groups, which can accurately analyze the behavioral characteristics of elderly users by combining short-term and long-term observation data, infer the real emotional needs of elderly users, and make reasonable feedback that takes into account physical assistance and virtual experience, thereby solving the problems of lack of emotional companionship and insufficient physical assistance faced by the elderly group in the process of home-based elderly care, and improving the autonomous service capabilities of existing technical solutions, thereby providing better emotional companionship and physical assistance; effect examples: when elderly users have not communicated for a long time and are in a low mood, the robot will take the initiative to communicate with them, share news or interesting events through sound and images, etc., to increase their willingness to communicate; when elderly users sit on the sofa for a long time without activity, the robot will take the initiative to prepare drinks, fruits, etc. for them, suggest getting up and moving for a while or going outdoors, and provide physical assistance during the human activity, etc.

[0057] In one embodiment, a multi-channel acquisition vector stitching subsystem includes:

[0058] The multimodal sensing and capturing subsystem collects multi-channel signals, captures user data, and obtains multi-channel data on user emotions and behaviors through the multimodal sensing module;

[0059] The data vector splicing subsystem splices the multi-channel data vectors of user emotions and behaviors to form a complete multi-channel data splicing vector.

[0060] The principle and effect of the above technical solution are as follows: a multi-channel acquisition vector stitching subsystem, including: a multi-modal perception capture subsystem, which collects multi-channel signals, captures user data, and obtains multi-channel data of user emotions and behaviors through a multi-modal perception module; a data vector stitching subsystem, which stitches multi-channel data vectors of user emotions and behaviors to form a multi-channel data stitching complete vector; the multi-modal perception module includes: a video perception unit, a speech recognition unit, a tactile sensing unit and a multi-modal signal processing unit; the video perception unit, the speech recognition unit and the tactile sensing unit respectively collect video data to intelligently perceive the emotional state and action behavior of the user in the video, and collect voice information for intelligent recognition and semantic perception. The user expresses his emotional state and semantic behavior, collects touch pressure tactile information, intelligently identifies touch pressure, perceives the user's tactile state and touch behavior; the multimodal signal processing unit generates independent input information for each channel from the multi-channel signal, converts it into a vector through the encoder, and obtains a multi-channel data vector; the multi-channel data vectors of user emotions and behaviors are spliced ​​to form a complete multi-channel data splicing vector, including: data processing is performed according to the signal acquisition frequency of different channels, and the low-frequency signal is interpolated and completed based on the signal acquisition frequency of the highest frame rate; the complete vector after splicing represents the input data of each frame; the user's behavior in each frame can be collected by the multimodal perception module and transmitted to the subsequent process for processing.

[0061] In one embodiment, the short-term and long-term hierarchical reasoning subsystem includes:

[0062] The short-term inference and understanding subsystem splices complete vectors based on multi-channel data and extracts short-term observation data; infers the user's current emotional state and behavioral state, performs short-term emotional and behavioral understanding, and obtains short-term emotional and behavioral understanding results;

[0063] The long-term inference and understanding subsystem collects short-term observation data, forms long-term observation data, conducts long-term emotion and behavior understanding of the long-term observation data, and obtains long-term emotion and behavior understanding results;

[0064] The hierarchical reasoning and judgment subsystem performs hierarchical emotion and behavior understanding and obtains hierarchical emotion and behavior understanding results.

[0065] The principle and effect of the above technical solution are as follows: short-term and long-term hierarchical reasoning subsystem, including: short-term inference and understanding subsystem, splicing complete vectors according to multi-channel data, extracting short-term observation data; inferring the user's current emotional state and behavioral state, performing short-term emotional and behavioral understanding, and obtaining short-term emotional and behavioral understanding results; long-term inference and understanding subsystem, collecting short-term observation data, forming long-term observation data, performing long-term emotional and behavioral understanding of long-term observation data, and obtaining long-term emotional and behavioral understanding results; hierarchical reasoning and judgment subsystem, performing hierarchical emotional and behavioral understanding, and obtaining hierarchical emotional and behavioral understanding results; splicing complete vectors according to multi-channel data, extracting short-term observation data; inferring the user's current emotional state and behavioral state, performing short-term emotional and behavioral understanding, and obtaining short-term emotional and behavioral understanding results include: splicing complete vectors according to multi-channel data, extracting current frame and adjacent frames of data, forming short-term observation data; extracting emotional information and behavioral information according to real-time short-term observation data information, and through neural network The processing units such as the network perform calculations, infer the emotional categories and behavioral states that can be estimated from the short-term data, infer the user's current emotional state and behavioral state, perform short-term emotional and behavioral understanding, and obtain short-term emotional and behavioral understanding results; real-time observation data information includes: facial expressions and body movements; collect short-term observation data, form long-term observation data, perform long-term emotional and behavioral understanding of the long-term observation data, and obtain long-term emotional and behavioral understanding results including: collect short-term observation data, and summarize them on the stored past observation data to form long-term observation data, and infer the user's long-term emotional state and behavioral state; then add the current frame data to the existing historical data, perform uniform key frame sampling on a larger time scale, form long-term observation data, perform calculations through the neural network processing unit, infer the emotional categories and behavioral states that can be estimated from the long-term data, perform long-term emotional and behavioral understanding of the long-term observation data, and obtain long-term emotional and behavioral understanding results; the neural network processing unit includes ResNet;

[0066] Conduct hierarchical emotion and behavior understanding, and obtain hierarchical emotion and behavior understanding results, including: when the short-term emotion and behavior understanding results are consistent with the long-term emotion and behavior understanding results, enter the next link for processing; when the short-term emotion and behavior understanding results conflict with the long-term emotion and behavior understanding results, the emotion and behavior understanding module integrates the short-term understanding results and the long-term understanding results to make a final judgment; the understanding result conflicts include: the long-term emotion is judged as happy, and the short-term emotion is bad mood; the long-term behavior is judged as preferring quiet and solitude, and the short-term behavior is judged as active and active; the input information of the comprehensive emotion and behavior understanding module, in addition to the hierarchical emotion and behavior understanding results, that is, the short-term emotion and behavior understanding results and the long-term emotion and behavior understanding results, The understanding results also include preset emotion value and behavior value parameters; the emotion value parameters can be adjusted twice based on the emotion understanding results to form different output results; including: when the long-term emotion is judged to be happy and the short-term emotion is judged to be angry, if the emotion value parameter is set to happiness is better than anger, then the operation result is that the long-term emotion is normal and the short-term emotion needs to be adjusted, and the robot will make a behavior to adjust the short-term emotion; conversely, if the emotion value parameter is set to anger is better than happiness, then the short-term emotion is normal and the long-term emotion needs to be adjusted, and the robot will make a behavior to adjust the long-term emotion; the usage method of the behavior value parameter is the same as the usage method of the emotion value parameter, which is used to analyze and process the short-term behavior understanding results and the long-term behavior understanding results.

[0067] In one embodiment, the time-sensitive response parallel task subsystem includes:

[0068] The time-sensitive multimodal response subsystem triggers the robot system's corresponding time-sensitive multimodal response according to the robot's behavioral decision; the corresponding time-sensitive multimodal response includes: short-term emotional response or long-term emotional response;

[0069] The dual-space parallel task subsystem generates images, voices, and various physical behavior tasks in the physical space, as well as virtual interaction scene tasks in the virtual space, based on the corresponding time-sensitive multimodal responses, and executes dual-space parallel tasks in the physical space and virtual space.

[0070] The principle and effect of the above technical solution are as follows: a time-sensitive response parallel task subsystem, including: a time-sensitive multimodal response subsystem, which triggers the corresponding time-sensitive multimodal response of the robot system according to the robot behavior decision; the corresponding time-sensitive multimodal response includes: short-term emotional response or long-term emotional response; a dual-space parallel task subsystem, which generates images, voices and various physical behavior tasks in the physical space, and virtual interactive scene tasks in the virtual space according to the corresponding time-sensitive multimodal response, and executes dual-space parallel tasks in the physical space and the virtual space; executing dual-space parallel tasks in the physical space and the virtual space includes: in the physical space, responding to and adjusting the user's short-term emotions or long-term emotions, assisting the user's behavior, and in the home-based elderly care mode, performing elderly user behavior risk alarms or elderly user dangerous rescue; and solving daily life tasks for users by controlling mechanical devices; using digital technology in the virtual space to create virtual interactive task scene tasks for users , providing the elderly user group with emotional value of life companionship; virtual interaction scene tasks include: virtual emotional communication scene tasks, leisure and entertainment scene tasks and life social scene tasks; virtual emotional communication scene tasks determine the emotional needs and behavioral status of elderly users by identifying the elderly users' voices and analyzing the elderly users' behaviors, and intelligently respond to the elderly users' emotional exchanges. When the elderly users are emotionally depressed, timely obtain the elderly users' emotional response information through the emotional big data analysis of the elderly group, and assist in improving the elderly users' emotional positivity; life social scene tasks respond to command information or determine the elderly users' behavioral status according to command information or elderly users' behavior analysis information, and determine whether the elderly users have difficulty in taking care of themselves or need emergency assistance. When it is determined that they have difficulty in taking care of themselves or have physically dangerous behaviors and need emergency assistance, timely send out difficulty assistance information to assist in solving the elderly users' social difficulties in taking care of themselves and emergency rescue of physically dangerous behaviors.

[0071] In one embodiment, the decision-making task evaluation test subsystem includes:

[0072] The test embodied scenario building subsystem constructs diversified task test scenarios and virtual embodied interaction scenarios in a virtual test environment;

[0073] The robot decision-making evaluation subsystem evaluates the hierarchical emotion and behavior decision-making of the robot system based on autonomous emotion value according to diversified task test scenarios, evaluates the autonomous intelligent driving decision-making ability of the robot system, and obtains the decision-making evaluation results of the robot system;

[0074] The task completion evaluation subsystem performs task completion evaluation in the virtual embodied interaction scenario according to the task completion measurement indicators, evaluates the robot system's task completion capability, and obtains the task completion evaluation results;

[0075] The evaluation feedback continuously optimizes the subsystem. Based on the robot system decision-making evaluation results and task completion evaluation results, the decision-making autonomous driving level and task completion of the autonomous intelligent robot system for home-based elderly care are continuously optimized.

[0076] The principle and effect of the above technical solution are as follows: the decision-making task evaluation test subsystem includes: the test embodied scene construction subsystem, which constructs diversified task test scenes and virtual embodied interaction scenes in a virtual test environment; the robot decision evaluation subsystem, which evaluates the hierarchical emotion and behavior decision-making of the robot system based on autonomous emotional value according to diversified task test scenes, evaluates the autonomous intelligent driving decision-making ability of the robot system, and obtains the decision evaluation results of the robot system; the task completion evaluation subsystem, in the virtual embodied interaction scene, performs task completion evaluation according to the task completion measurement index, evaluates the task completion ability of the robot system, and obtains the task completion evaluation results; the evaluation feedback continuous optimization subsystem, which continuously optimizes the decision-making autonomous driving level and task completion of the autonomous intelligent robot system for home-based elderly care according to the robot system decision evaluation results and task completion evaluation results; according to diversified task test scenes, the robot system is evaluated The robot system makes decisions on various behaviors based on autonomous emotional values, and the evaluation of the robot system's autonomous intelligent driving decision-making capabilities includes: using the graphics engine to generate a variety of task test scenarios, setting evaluation test questions for autonomous driving capabilities, and setting an API application programming interface that supports the access of the autonomous intelligent robot for home care to be tested; starting the test task, according to the evaluation test questions, providing visual signals and voice signals to the robot system to be tested, waiting for the robot system to make a decision, and obtaining the robot system decision; comparing the robot system decision with the expert decision, and the labeled data in the same task type to obtain the decision labeling difference; according to the decision labeling difference, classify the robot decision into the corresponding autonomous driving level according to the principle of maximum similarity; according to the autonomous driving level, evaluate the robot system's autonomous intelligent driving decision-making ability; feedback the decision labeling difference and autonomous driving level, and continuously optimize the robot system's autonomous intelligent driving decision-making ability;

[0077] In the virtual embodied interaction scenario, task completion evaluation is performed based on the task completion measurement indicators. The evaluation of the robot system's task completion capability includes: setting the task completion measurement indicators based on the expert experience model; constructing a virtual embodied interaction scenario based on the expert's pre-labeled reference data to test the robot system's task completion; completion measurement indicators include: task completion time, task completion rate and user goal satisfaction; the task completion rate is set as the percentage of task completion to the total number of tasks; the user goal satisfaction is set as a target satisfaction percentage of 0-100%, and the larger the target satisfaction percentage, the higher the satisfaction; the task completion is calculated by multiplying the task completion time by the task completion The rate is multiplied by the user's goal satisfaction. The larger the calculated value, the higher the task completion. A test scenario for elderly users is constructed. The test scenarios for elderly users include: falling, going up and down stairs slowly, and difficulty in opening the door for visiting guests in time. The test scenario data for elderly users is provided to the robot to be tested. The robot to be tested performs intelligent observation and response based on the test scenario data for elderly users, and performs corresponding operations in the virtual space of the virtual embodied interaction scenario. The task completion of the robot system is calculated based on the task completion measurement index. For test tasks in the physical space, a real physical test environment is built in the real physical space, and the physical space task completion is evaluated with reference to the virtual embodied interaction scenario.

[0078] The present invention provides a method for constructing an autonomous intelligent robot system for home-based elderly care, comprising:

[0079] S100, collecting multi-channel signals, capturing user data, obtaining multi-channel data of user emotions and behaviors, and performing data vector splicing to form a multi-channel data splicing complete vector;

[0080] S200, splicing complete vectors based on multi-channel data, performing short-term emotion and behavior understanding and long-term emotion and behavior understanding, analyzing hierarchical emotion and behavior understanding, and obtaining hierarchical emotion and behavior understanding results;

[0081] S300, based on the hierarchical emotion and behavior understanding results, triggers the robot system to respond in a timely multimodal manner; outputs hierarchical emotion and behavior decisions through a multimodal output channel, and executes dual-space parallel tasks in physical space and virtual space;

[0082] S400 conducts autonomous driving level assessment and task completion assessment to assess the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimizes the autonomous driving level and task completion of the home-based elderly care autonomous intelligent robot system.

[0083] The principle and effect of the above technical solution are as follows: the present invention provides a method for constructing an autonomous intelligent robot system for home-based elderly care, including: collecting multi-channel signals, capturing user data, obtaining multi-channel data of user emotions and behaviors, and performing data vector splicing to form a multi-channel data splicing complete vector; performing short-term emotion and behavior understanding and long-term emotion and behavior understanding according to the multi-channel data splicing complete vector, analyzing hierarchical emotion and behavior understanding, and obtaining hierarchical emotion and behavior understanding results; According to the results of hierarchical emotion and behavior understanding, the robot system is triggered to respond in a timely multimodal manner; through the multimodal output channel, hierarchical emotion and behavior decisions are output, and dual-space parallel tasks in physical space and virtual space are executed; autonomous driving level evaluation and task completion evaluation are conducted to evaluate the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimize the autonomous driving level and task completion of the decision-making of the autonomous intelligent robot system for home-based elderly care; based on long-term and short-term observation data, a hierarchical decision-making framework is constructed to integrate autonomous emotional value and behavioral value to deal with decision-making problems in complex situations; taking into account multimodal feedback from physical space and virtual space, the emotional companionship effect is enhanced through the operation of virtual space, and the physical assistance effect is ensured through the operation of physical space; an embodied interaction test scenario is constructed, and automatic matching is performed based on expert-annotated data to output the autonomy level and task completion; a hierarchical emotion understanding and behavior understanding framework based on short-term and long-term emotion understanding is constructed; and intelligent machines are tested with a variety of virtual embodied test scenarios. The autonomy and task completion of robots; quantification of the autonomy level of autonomous intelligent robots; intelligent assistance that integrates virtual feedback and physical feedback; building an autonomous intelligent robot system for home-based elderly care groups, which can accurately analyze the behavioral characteristics of elderly users by combining short-term and long-term observation data, infer the real emotional needs of elderly users, and make reasonable feedback that takes into account physical assistance and virtual experience, solve the problems of lack of emotional companionship and insufficient physical assistance faced by the elderly group in the process of home-based elderly care, and improve the autonomous service capabilities of existing technical solutions, so as to provide better emotional companionship and physical assistance; effect examples: when elderly users have not communicated for a long time and are in a low mood, the robot will take the initiative to communicate with them, share news or interesting events through sound and images, etc., to increase their willingness to communicate; when elderly users sit on the sofa for a long time without activity, the robot will take the initiative to prepare drinks, fruits, etc. for them, suggest getting up and moving for a while or going outdoors, and provide physical assistance during the human activity, etc.

[0084] In one embodiment, S100 includes:

[0085] S101, through the multimodal perception module, collect multi-channel signals, capture user data, and obtain multi-channel data of user emotions and behaviors;

[0086] S102, splicing the multi-channel data vectors of user emotions and behaviors to form a multi-channel data splicing complete vector.

[0087] The principle and effect of the above technical solution are as follows: through the multimodal perception module, multi-channel signals are collected, user data is captured, and multi-channel data of user emotions and behaviors are obtained; the multi-channel data vectors of user emotions and behaviors are spliced ​​to form a complete vector of multi-channel data splicing; the multimodal perception module includes: a video perception unit, a speech recognition unit, a tactile sensing unit and a multimodal signal processing unit; the video perception unit, the speech recognition unit and the tactile sensing unit respectively collect video data to intelligently perceive the user's emotional state and user action behavior in the video, collect voice information to intelligently recognize semantics to perceive the user's emotional state and user semantic behavior, and collect Touch pressure tactile information intelligently identifies touch pressure to perceive the user's tactile state and user touch behavior; the multimodal signal processing unit generates independent input information for each channel from the multi-channel signal, converts it into a vector through an encoder, and obtains a multi-channel data vector; the multi-channel data vectors of user emotions and behaviors are spliced ​​to form a complete multi-channel data splicing vector, including: data processing based on the signal acquisition frequency of different channels, and interpolating and completing the low-frequency signal based on the signal acquisition frequency of the highest frame rate; the complete vector after splicing represents the input data of each frame; the user's behavior in each frame can be collected by the multimodal perception module and transmitted to the subsequent process processing.

[0088] In one embodiment, S200 includes:

[0089] S201, splicing a complete vector based on multi-channel data, extracting short-term observation data; inferring the user's current emotional state and behavioral state, performing short-term emotional and behavioral understanding, and obtaining short-term emotional and behavioral understanding results;

[0090] S202, collecting short-term observation data to form long-term observation data, performing long-term emotion and behavior understanding on the long-term observation data, and obtaining long-term emotion and behavior understanding results;

[0091] S203, performing hierarchical emotion and behavior understanding to obtain hierarchical emotion and behavior understanding results.

[0092] The principle and effect of the above technical solution are: splicing complete vectors according to multi-channel data, extracting short-term observation data; inferring the user's current emotional state and behavioral state, conducting short-term emotional and behavioral understanding, and obtaining short-term emotional and behavioral understanding results; collecting short-term observation data to form long-term observation data, conducting long-term emotional and behavioral understanding of long-term observation data, and obtaining long-term emotional and behavioral understanding results; performing hierarchical emotional and behavioral understanding to obtain hierarchical emotional and behavioral understanding results; splicing complete vectors according to multi-channel data, extracting short-term observation data; inferring the user's current emotional state and behavioral state, conducting short-term emotional and behavioral understanding, and obtaining short-term emotional and behavioral understanding results include: splicing complete vectors according to multi-channel data, extracting the current frame and several adjacent frames of data to form short-term observation data; extracting emotional information and behavioral information based on real-time short-term observation data information, performing calculations through processing units such as neural networks, and inferring what can be estimated by short-term data The emotional category and behavior state of the user are inferred, the current emotional state and behavior state of the user are understood in the short term, and the short-term emotional and behavioral understanding results are obtained; the real-time observation data information includes: facial expressions and body movements; short-term observation data are collected to form long-term observation data, and long-term emotional and behavioral understanding of the long-term observation data is performed, and the long-term emotional and behavioral understanding results are obtained, including: collecting short-term observation data, and aggregating them to the stored past observation data to form long-term observation data, and inferring the user's long-term emotional state and behavioral state; then adding the current frame data to the existing historical data, uniformly sampling key frames from a larger time scale, forming long-term observation data, and performing calculations through the neural network processing unit to infer the emotional category and behavioral state that can be estimated by the long-term data, and long-term emotional and behavioral understanding of the long-term observation data is performed to obtain long-term emotional and behavioral understanding results; the neural network processing unit includes ResNet;

[0093] Conduct hierarchical emotion and behavior understanding, and obtain hierarchical emotion and behavior understanding results, including: when the short-term emotion and behavior understanding results are consistent with the long-term emotion and behavior understanding results, enter the next link for processing; when the short-term emotion and behavior understanding results conflict with the long-term emotion and behavior understanding results, the emotion and behavior understanding module integrates the short-term understanding results and the long-term understanding results to make a final judgment; the understanding result conflicts include: the long-term emotion is judged as happy, and the short-term emotion is bad mood; the long-term behavior is judged as preferring quiet and solitude, and the short-term behavior is judged as active and active; the input information of the comprehensive emotion and behavior understanding module, in addition to the hierarchical emotion and behavior understanding results, that is, the short-term emotion and behavior understanding results and the long-term emotion and behavior understanding results, The understanding results also include preset emotion value and behavior value parameters; the emotion value parameters can be adjusted twice based on the emotion understanding results to form different output results; including: when the long-term emotion is judged to be happy and the short-term emotion is judged to be angry, if the emotion value parameter is set to happiness is better than anger, then the operation result is that the long-term emotion is normal and the short-term emotion needs to be adjusted, and the robot will make a behavior to adjust the short-term emotion; conversely, if the emotion value parameter is set to anger is better than happiness, then the short-term emotion is normal and the long-term emotion needs to be adjusted, and the robot will make a behavior to adjust the long-term emotion; the usage method of the behavior value parameter is the same as the usage method of the emotion value parameter, which is used to analyze and process the short-term behavior understanding results and the long-term behavior understanding results.

[0094] In one embodiment, S300 includes:

[0095] S301, triggering a corresponding time-effect multimodal response of the robot system according to the robot behavior decision; the corresponding time-effect multimodal response includes: a short-term emotional response or a long-term emotional response;

[0096] S302, based on the corresponding time-sensitive multimodal response, generate images, voices and various physical behavior tasks in the physical space, as well as virtual interaction scene tasks in the virtual space, and execute dual-space parallel tasks in the physical space and the virtual space.

[0097] The principle and effect of the above technical solution are: according to the robot's behavioral decision, the corresponding time-sensitive multimodal response of the robot system is triggered; the corresponding time-sensitive multimodal response includes: short-term emotional response or long-term emotional response; according to the corresponding time-sensitive multimodal response, images, voices and various physical behavior tasks in the physical space, and virtual interactive scene tasks in the virtual space are generated, and dual-space parallel tasks in the physical space and the virtual space are executed; the execution of dual-space parallel tasks in the physical space and the virtual space includes: in the physical space, responding to and adjusting the user's short-term emotions or long-term emotions, assisting the user's behavior, and in the home-based elderly care mode, performing elderly user behavior risk alarms or elderly user dangerous rescue; and solving daily life tasks for users by controlling mechanical devices; using digital technology in the virtual space to create virtual interactive task scene tasks for users, and providing life companionship emotional value for the elderly user group. ; Virtual interaction scene tasks include: virtual emotional communication scene tasks, leisure and entertainment scene tasks, and life social scene tasks; virtual emotional communication scene tasks identify the elderly users' voices and analyze the elderly users' behaviors, determine the elderly users' emotional needs and behavioral states, and respond to the elderly users' emotional exchanges intelligently. When the elderly users are emotionally depressed, timely obtain the elderly users' emotional response information through the emotional big data analysis of the elderly group, to assist in improving the elderly users' emotional positivity; life social scene tasks respond to command information or determine the elderly users' behavioral states based on command information or elderly users' behavior analysis information, determine whether the elderly users have difficulty taking care of themselves or need emergency assistance, and when it is determined that they have difficulty taking care of themselves or have physically dangerous behaviors and need emergency assistance, timely send out difficulty assistance information to assist in solving the elderly users' difficulties in taking care of themselves and social dilemmas and emergency rescue of physically dangerous behaviors.

[0098] In one embodiment, S400 includes:

[0099] S401, constructing diversified task test scenarios and virtual embodiment interaction scenarios in a virtual test environment;

[0100] S402, based on the diversified task test scenarios, evaluate the hierarchical emotion and behavior decision-making of the robot system based on the autonomous emotion value, evaluate the autonomous intelligent driving decision-making ability of the robot system, and obtain the decision-making evaluation results of the robot system;

[0101] S403, in the virtual embodied interaction scene, performing task completion evaluation according to the task completion measurement index, evaluating the task completion capability of the robot system, and obtaining the task completion evaluation result;

[0102] S404, based on the robot system decision-making evaluation results and task completion evaluation results, continuously optimize the decision-making autonomous driving level and task completion of the autonomous intelligent robot system for home-based elderly care.

[0103] The principle and effect of the above technical solution are as follows: in a virtual test environment, construct diversified task test scenarios and virtual embodied interaction scenarios; according to the diversified task test scenarios, evaluate the hierarchical emotions and behavioral decisions of the robot system based on autonomous emotional values, evaluate the autonomous intelligent driving decision-making ability of the robot system, and obtain the decision evaluation results of the robot system; in the virtual embodied interaction scenario, according to the task completion measurement indicators, conduct task completion evaluation, evaluate the task completion ability of the robot system, and obtain the task completion evaluation results; according to the robot system decision evaluation results and task completion evaluation results, continuously optimize the decision-making autonomous driving level and task completion of the autonomous intelligent robot system for home-based elderly care; according to the diversified task test scenarios, evaluate the various behavioral robot system decisions based on autonomous emotional values ​​of the robot system, and evaluate the autonomous driving ability of the robot system Intelligent driving decision-making capabilities include: using the graphics engine to generate a variety of task test scenarios, setting evaluation test questions for autonomous driving capabilities, and setting an API application programming interface that supports the access of the autonomous intelligent robot for home-based elderly care to be tested; starting the test task, according to the evaluation test questions, providing visual signals and voice signals to the robot system to be tested, waiting for the robot system to make a decision, and obtaining the robot system decision; comparing the robot system decision with the expert decision, and the labeled data in the same task type to obtain the decision labeling difference; according to the decision labeling difference, classify the robot decision into the corresponding autonomous driving level according to the principle of maximum similarity; according to the autonomous driving level, evaluate the autonomous intelligent driving decision-making ability of the robot system; feedback the decision labeling difference and autonomous driving level, and continuously optimize the autonomous intelligent driving decision-making ability of the robot system;

[0104] In the virtual embodied interaction scenario, task completion evaluation is performed based on the task completion measurement indicators. The evaluation of the robot system's task completion capability includes: setting the task completion measurement indicators based on the expert experience model; constructing a virtual embodied interaction scenario based on the expert's pre-labeled reference data to test the robot system's task completion; completion measurement indicators include: task completion time, task completion rate and user goal satisfaction; the task completion rate is set as the percentage of task completion to the total number of tasks; the user goal satisfaction is set as a target satisfaction percentage of 0-100%, and the larger the target satisfaction percentage, the higher the satisfaction; the task completion is calculated by multiplying the task completion time by the task completion The rate is multiplied by the user's goal satisfaction. The larger the calculated value, the higher the task completion. A test scenario for elderly users is constructed. The test scenarios for elderly users include: falling, going up and down stairs slowly, and difficulty in opening the door for visiting guests in time. The test scenario data for elderly users is provided to the robot to be tested. The robot to be tested performs intelligent observation and response based on the test scenario data for elderly users, and performs corresponding operations in the virtual space of the virtual embodied interaction scenario. The task completion of the robot system is calculated based on the task completion measurement index. For test tasks in the physical space, a real physical test environment is built in the real physical space, and the physical space task completion is evaluated with reference to the virtual embodied interaction scenario.

[0105] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. An autonomous intelligent robot system for home-based elderly care, characterized in that: include: The multi-channel acquisition vector splicing subsystem collects multi-channel signals, captures user data, obtains multi-channel data of user emotions and behaviors, and performs data vector splicing to form a multi-channel data splicing complete vector; The short-term and long-term hierarchical reasoning subsystem splices complete vectors based on multi-channel data, performs short-term emotion and behavior understanding and long-term emotion and behavior understanding, analyzes hierarchical emotion and behavior understanding, and obtains hierarchical emotion and behavior understanding results; The time-sensitive response parallel task subsystem triggers the robot system's corresponding time-sensitive multimodal response based on the hierarchical emotion and behavior understanding results; outputs hierarchical emotion and behavior decisions through the multimodal output channel, and executes dual-space parallel tasks in physical space and virtual space; The decision-making task evaluation test subsystem conducts autonomous driving level evaluation and task completion evaluation, evaluates the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimizes the decision-making autonomous driving level and task completion of the home-based elderly care autonomous intelligent robot system; Time-sensitive response parallel task subsystem, including: The time-sensitive multimodal response subsystem triggers the robot system's corresponding time-sensitive multimodal response according to the robot's behavioral decision; the corresponding time-sensitive multimodal response includes: short-term emotional response or long-term emotional response; The dual-space parallel task subsystem generates images, voices, and various physical behavior tasks in the physical space, as well as virtual interactive scene tasks in the virtual space, based on the corresponding time-sensitive multimodal responses, and executes dual-space parallel tasks in the physical space and virtual space; Decision-making task evaluation test subsystem, including: The test embodied scenario building subsystem constructs diversified task test scenarios and virtual embodied interaction scenarios in a virtual test environment; The robot decision-making evaluation subsystem evaluates the hierarchical emotion and behavior decision-making of the robot system based on autonomous emotion value according to diversified task test scenarios, evaluates the autonomous intelligent driving decision-making ability of the robot system, and obtains the decision-making evaluation results of the robot system; The task completion evaluation subsystem performs task completion evaluation in the virtual embodied interaction scenario according to the task completion measurement indicators, evaluates the robot system's task completion capability, and obtains the task completion evaluation results; The evaluation feedback continuously optimizes the subsystem. Based on the robot system decision-making evaluation results and task completion evaluation results, the decision-making autonomous driving level and task completion of the autonomous intelligent robot system for home-based elderly care are continuously optimized.

2. The home-based elderly care autonomous intelligent robot system according to claim 1, characterized in that: Multi-channel acquisition vector stitching subsystem, including: The multimodal sensing and capturing subsystem collects multi-channel signals, captures user data, and obtains multi-channel data on user emotions and behaviors through the multimodal sensing module; The data vector splicing subsystem splices the multi-channel data vectors of user emotions and behaviors to form a complete multi-channel data splicing vector.

3. The home-based elderly care autonomous intelligent robot system according to claim 1, characterized in that: Short-term and long-term hierarchical reasoning subsystem, including: The short-term inference and understanding subsystem splices complete vectors based on multi-channel data and extracts short-term observation data; infers the user's current emotional state and behavioral state, performs short-term emotional and behavioral understanding, and obtains short-term emotional and behavioral understanding results; The long-term inference and understanding subsystem collects short-term observation data, forms long-term observation data, conducts long-term emotion and behavior understanding of the long-term observation data, and obtains long-term emotion and behavior understanding results; The hierarchical reasoning and judgment subsystem performs hierarchical emotion and behavior understanding and obtains hierarchical emotion and behavior understanding results.

4. A method for constructing an autonomous intelligent robot system for home-based elderly care, characterized in that: include: S100, collecting multi-channel signals, capturing user data, obtaining multi-channel data of user emotions and behaviors, and performing data vector splicing to form a multi-channel data splicing complete vector; S200, splicing complete vectors based on multi-channel data, performing short-term emotion and behavior understanding and long-term emotion and behavior understanding, analyzing hierarchical emotion and behavior understanding, and obtaining hierarchical emotion and behavior understanding results; S300, based on the hierarchical emotion and behavior understanding results, triggers the robot system to respond in a timely multimodal manner; outputs hierarchical emotion and behavior decisions through a multimodal output channel, and executes dual-space parallel tasks in physical space and virtual space; S400, conducts autonomous driving level assessment and task completion assessment, assesses the robot's autonomous intelligent driving decision-making ability and task completion ability, and continuously optimizes the decision-making autonomous driving level and task completion of the home-based elderly care autonomous intelligent robot system; S300 includes: S301, triggering a corresponding time-effect multimodal response of the robot system according to the robot behavior decision; the corresponding time-effect multimodal response includes: a short-term emotional response or a long-term emotional response; S302, generating images, voices, and various physical behavior tasks of the physical space, and virtual interaction scene tasks of the virtual space according to the corresponding time-sensitive multimodal response, and executing dual-space parallel tasks of the physical space and the virtual space; S400 includes: S401, constructing diversified task test scenarios and virtual embodiment interaction scenarios in a virtual test environment; S402, based on the diversified task test scenarios, evaluate the hierarchical emotion and behavior decision-making of the robot system based on the autonomous emotion value, evaluate the autonomous intelligent driving decision-making ability of the robot system, and obtain the decision-making evaluation results of the robot system; S403, in the virtual embodied interaction scene, performing task completion evaluation according to the task completion measurement index, evaluating the task completion capability of the robot system, and obtaining the task completion evaluation result; S404, based on the robot system decision evaluation results and task completion evaluation results, continuously optimize the decision-making autonomous driving level and task completion of the home-based elderly care autonomous intelligent robot system.

5. The method for constructing an autonomous intelligent robot system for home-based elderly care according to claim 4, characterized in that: S100 includes: S101, through the multimodal perception module, collect multi-channel signals, capture user data, and obtain multi-channel data of user emotions and behaviors; S102, splicing the multi-channel data vectors of user emotions and behaviors to form a multi-channel data splicing complete vector.

6. The method for constructing an autonomous intelligent robot system for home-based elderly care according to claim 4, characterized in that: S200 includes: S201, splicing a complete vector based on multi-channel data, extracting short-term observation data; inferring the user's current emotional state and behavioral state, performing short-term emotional and behavioral understanding, and obtaining short-term emotional and behavioral understanding results; S202, collecting short-term observation data to form long-term observation data, performing long-term emotion and behavior understanding on the long-term observation data, and obtaining long-term emotion and behavior understanding results; S203, performing hierarchical emotion and behavior understanding to obtain hierarchical emotion and behavior understanding results.

Images