Emotion service robot system based on multi-mode mental theory

Abstract

The invention relates to the technical field of artificial intelligence, intelligent and robot control, and discloses an emotion service robot system based on a multi-mode mental theory, which comprises a hierarchical mental reasoning module, a connection module and an action semantic execution module, the hierarchical mental reasoning module is used for generating a decision text containing a high-level strategy according to the multi-modal environment information collected by the robot; the connection module is used for constructing a semantic instruction according to the decision text generated by the hierarchical mental reasoning module; and the action semantic execution module is used for generating a control action of the robot according to the semantic instruction constructed by the connection module and the real-time image acquired by the robot. According to the method, a control architecture fusing a vision-language-action model and a robot center view angle mental reasoning hierarchy is constructed, so that the robot can carry out multi-order belief reasoning and implicit target inference from the view angle of the robot. According to the system, the robot is no longer a pure instruction follower and has an active service capability.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, embodied intelligence and robot control technology, and specifically to an emotional service robot system based on multimodal mind theory. Background Technology

[0002] With the development of embodied intelligence, robots have gradually acquired basic command-following capabilities. However, existing technologies still have significant limitations when facing complex human-computer interaction scenarios. In terms of reasoning techniques based on theory of mind, current methods mainly rely on text or a combination of video images to infer beliefs, desires, and intentions. However, their interaction formats are mostly limited to multiple-choice or short-answer questions, making it difficult to support open-ended, real-world, and continuous reasoning. Furthermore, they lack the ability to reason from the agent's own perspective, and cannot autonomously formulate decisions and actions based on the inference results. Simultaneously, existing technologies cannot effectively handle erroneous beliefs arising from human memory errors or perceptual limitations in dynamic interactions, and they lack the ability to infer implicit goals based on behavioral cues and proactively generate assistance strategies, resulting in insufficient service efficiency in non-command scenarios.

[0003] In the realm of embodied agents based on vision-language models, while existing systems can decompose complex goals into sub-tasks and support multi-agent collaboration, they still heavily rely on predefined goals or mimicry signals. They lack self-perspective psychological reasoning and social intelligence, and generally lack the ability to reason about first- or second-order beliefs, making it difficult to achieve truly autonomous decision-making based on social perception. Furthermore, end-to-end vision-language-action models are essentially "instruction followers," with training data primarily consisting of pairings of explicit instructions with action trajectories. They lack systematic thought chain reasoning capabilities and cannot effectively handle scenarios involving implicit goal inference or correction of erroneous beliefs. This can easily lead to hallucinatory behavior or blindly mimicking users' erroneous actions, limiting their reliable application in complex home service environments. Summary of the Invention

[0004] This invention aims to solve the technical problems of existing embodied intelligent robots in human-computer interaction, such as their inability to passively follow instructions and handle implicit human goals and cognitive conflicts due to the lack of social reasoning ability.

[0005] To address this, the present invention constructs a control architecture that integrates a VLA (Visual-Language-Motion) model with a robot-centric perspective (Robot-Centric) mental reasoning hierarchy. This architecture enables the robot to infer not only the psychological state of humans from its own perspective but also to construct its own Belief, Desire, and Intention levels. This allows the robot to autonomously identify and assist in correcting abnormal behaviors caused by erroneous beliefs, or to infer implicit goals without explicit instructions. The high-level mental reasoning results are then transformed into precise semantic instructions to guide the underlying VLA model in generating actions. Ultimately, this results in the control of a home service robot with proactive service capabilities, high robustness, and logically interpretable logic.

[0006] To achieve the above objectives, the present invention provides an emotional service robot system based on multimodal mental theory, comprising: a hierarchical mental reasoning module, a connection module, and an action semantic execution module;

[0007] The hierarchical mental reasoning module is used to generate decision text containing high-level strategies based on the multimodal environmental information collected by the robot.

[0008] The connection module is used to construct semantic instructions based on the decision text generated by the hierarchical mental reasoning module;

[0009] The action semantic execution module is used to generate robot control actions based on the semantic instructions constructed by the connection module and the real-time images collected by the robot.

[0010] Preferably, the hierarchical mental reasoning module includes a perception unit, a mental reasoning unit, and a decision-making unit connected in sequence;

[0011] The sensing unit is used to generate objective descriptive text about the environment based on the image sequence collected by the robot.

[0012] The mental reasoning unit is used to infer and generate the robot's beliefs, desires, and intentional states based on the objective descriptive text generated by the perception unit.

[0013] The decision-making unit is used to generate decision text containing high-level strategies based on the objective descriptive text generated by the perception unit and the belief, desire and intention state generated by the mental reasoning unit.

[0014] Preferably, the workflow of the sensing unit includes:

[0015] Visual features are extracted from the image sequences collected by the robot.

[0016] The visual features are mapped to the word embedding space of the language model to obtain the visual embedding vector;

[0017] Based on the visual embedding vector and system prompts, an objective descriptive text of the environment is generated.

[0018] Preferably, the workflow of the mental reasoning unit includes:

[0019] Based on the objective description text, infer the factual state from the robot's perspective and the state of human beliefs inferred by the robot;

[0020] Based on the difference between the stated factual state and the stated human belief state, the robot's desire state is generated;

[0021] Based on the stated desire state, the robot's intention state is generated.

[0022] Preferably, the workflow of the decision-making unit includes:

[0023] The objective descriptive text is combined with the beliefs, desires, and intentions to form a complete context;

[0024] Based on the complete context, a natural language decision text containing high-level strategies is generated through an autoregressive generation method.

[0025] Preferably, the workflow of the connection module includes:

[0026] Extract task descriptions from the decision text generated by the hierarchical mental reasoning module;

[0027] The task description is filled into a preset prompt word template to generate semantic instructions for the action semantic execution module.

[0028] Preferably, the workflow of the action semantic execution module includes:

[0029] The real-time images acquired by the robot are subjected to hybrid visual encoding to obtain a visual tag sequence;

[0030] The semantic instructions generated by the connection module are text-encoded to obtain a text tag sequence;

[0031] Based on the visual tag sequence and the text tag sequence, perform cross-modal reasoning to predict the probability distribution of the next tag;

[0032] Based on the final hidden layer state output by the model, the robot's multidimensional motion vector is generated.

[0033] Preferably, the steps for performing hybrid visual encoding include:

[0034] The first pre-trained visual encoder is used to extract the geometric and spatial features of the image;

[0035] The semantic and texture features of the image are extracted using a second pre-trained visual encoder;

[0036] The geometric and spatial features are fused and projected with the semantic and texture features to generate the visual marker sequence.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] This invention constructs a control architecture that integrates a visual-language-action model with a robot-centric perspective and a hierarchical mental reasoning system, enabling the robot to perform multi-level belief reasoning and implicit goal inference from its own perspective. This system transforms the robot from a mere command follower into a proactive service provider, capable of autonomously identifying cognitive conflicts and generating assistive corrective strategies when humans develop erroneous beliefs due to memory errors or perceptual limitations.

[0039] Meanwhile, the system effectively avoids hallucinatory behavior or blind imitation caused by the lack of a reasoning chain in end-to-end models by translating natural language decisions generated by high-level mental reasoning into precise semantic instructions to drive the underlying model. The entire decision-making process is based on an explicit logical chain of "perception-belief-desire-intention-decision," which not only improves robustness in dealing with dynamic interactions in unstructured home environments but also makes the robot's behavior interpretable. In addition, a dedicated multimodal training dataset and a phased training strategy ensure that the mental reasoning model can stably generate logical thought chains, thereby achieving a leap from passive response to proactive cognitive assistance and significantly improving service efficiency and reliability in complex human-machine collaboration scenarios. Attached Figure Description

[0040] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the overall architecture of a home service robot control system with mental reasoning ability based on a VLA (Vision-Language-Motion) model, according to an embodiment of the present invention.

[0042] Figure 2 This is a flowchart illustrating the construction process of the MindPower multimodal reasoning dataset according to an embodiment of the present invention.

[0043] Figure 3This is a schematic diagram illustrating the training process and reward mechanism of the mental reasoning model in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the OpenVLA model architecture according to an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Example 1:

[0048] This embodiment provides an emotion service robot system based on multimodal mental theory, including: a hierarchical mental reasoning module, a connection module, and an action semantic execution module.

[0049] The following will describe in detail, with reference to this embodiment, how the present invention solves the technical problems in practical work.

[0050] (1) Hierarchical mental reasoning module based on robot-centric perspective.

[0051] This is the "high-level brain" of the system in this embodiment, designed to understand the environment and generate strategies from the robot's own perspective. This module does not directly interact with the underlying joint control, but instead outputs a structured thought chain based on the MindPower reasoning model.

[0052] This embodiment will use "assisting in finding items" as an example to demonstrate the reasoning process of the system of the present invention:

[0053] Level-1 Perception: Utilizes perceptual units to perform environmental semantic parsing based on a visual encoder.

[0054] At this level, the system generates objective descriptions of the environment and human behavior through an alignment mechanism between the visual encoder and the large language model.

[0055] ①Visual Encoding:

[0056] The system first acquires real-time video streams using a robot camera device, and then samples the data in time windows (32 frames per second) to obtain an image sequence. Using a pre-trained visual encoder, each frame of image I... t The image is divided into N fixed-size patches and mapped to a high-dimensional visual feature vector sequence H. v The mathematical expression for this process is:

[0057] ,

[0058] Where d is the feature dimension.

[0059] ② Cross-modality alignment:

[0060] In order for visual features to be understood by the language model, the system uses an adapter module to transfer the visual features H... v Mapping to the text embedding space of the language model yields the visual embedding vector E. v :

[0061] ,

[0062] Among them, W p and b p These are learnable projection parameters.

[0063] ③Perceptual text generation:

[0064] Visual embedding vector E v Embedded with system-preset prompt words E sys The concatenated text is then passed as input to the decoder (Decoder-only LLM). The model generates objective descriptive text Y based on the principle of maximizing autoregressive probability. per The joint probability distribution formula for its generation process is:

[0065] ,

[0066] Among them, y i For the currently generated Token, A sequence of tokens generated historically.

[0067] Output example: <perception>The owner moved to the kitchen counter andtried to reach the cup...

[0068] Level-2 Mental Reasoning (BDI Reasoning): Utilizes mental reasoning units to deduce causal chains based on the BDI framework.

[0069] At this level, the system leverages the contextual reasoning capabilities of the large language model to construct a logical closed loop of "belief-desire-intention" within the latent space. This process captures implicit cues in the perceived text through a multi-head self-attention mechanism.

[0070] ① Second-order Belief Modeling:

[0071] The model will output the Y value from Level-1. per As context, infer the belief state B from the robot's perspective. robot Unlike existing technologies that only infer human states, this embodiment introduces second-order belief computation, that is, it simultaneously computes the "fact state" S. true With "inferred human beliefs" S human The model focuses on Y through an attention mechanism. per Calculate the conditional probability of belief generation using specific tokens that describe human behavior:

[0072] ,

[0073] Where Y represents the candidate's belief description text; The parameters represent the mental reasoning model.

[0074] If detected (For example, if the fact is that the cup is far away and a human cannot reach it, but the human tries to reach for it), the system determines that there is a belief conflict.

[0075] Output example:<Robot Belief> I believe the owner wants to take the cup...I believe he cannot reach it because it is placed too far.

[0076] ②Desire Generation:

[0077] Based on the aforementioned belief states, when a belief conflict or implicit need is detected, the model activates the "facilitative strategy" generation path. This process can be formalized as a text generation task given perceptual and belief conditions:

[0078] ,

[0079] This step maps negative states in beliefs (such as "cannot reach") to the robot's positive desires.

[0080] Output example:<Robot Desire> I want to help the owner get the cup theycannot reach.

[0081] ③ Intention Generation:

[0082] Based on clear and specific wishes and action commitments It serves as a bridge connecting reasoning and decision-making.

[0083] Output example:<Robot Intention> Retrieve the cup from the kitchen counterand hand it to the owner.

[0084] Level-3 Decision Making and Action: At this level, the decision-making unit utilizes the autoregressive generative capabilities of the large language model to map implicit intention states into explicit natural language policies and structured control instructions. This process is mathematically described as... per ) and BDI inference chain The process of maximizing the joint probability under the given conditions, where This represents the robot's desired description generated by the model.

[0085] ① <decision>(High-level strategy generation):

[0086] The system utilizes the attention mechanism of the Transformer decoder to semantically expand the intent of the previous level output, generating embodied task planning in natural language form.

[0087] Algorithm process: Concatenate the outputs of Level-1 and Level-2 into a complete context vector. The model calculates the conditional probability distribution of the next token and generates decision text using a greedy search strategy. The mathematical formula is shown below:

[0088] ,

[0089] in, These are the model parameters.

[0090] This step transforms abstract intentions (such as "help Alice") into concrete task descriptions (such as "open the refrigerator and get an apple"), which serve as semantic constraints for the generation of subsequent actions.

[0091] (2) Connection module: Prompt (prompt word) construction and injection.

[0092] This module serves as a semantic interface connecting "high-level cognition" and "low-level execution." Its core function is to directly map the natural language decisions generated by the hierarchical mental reasoning module Level-3 into instruction templates for the OpenVLA model.

[0093] ① Decision Extraction:

[0094] The system directly extracts the Level-3 output. <decision>text Since the text is already a high-level structure that conforms to human language habits (e.g., "open the refrigerator and get an apple"), it naturally meets the semantic understanding needs of the large language model within the OpenVLA model, without requiring additional format conversion.

[0095] ② Prompt Template Construction:

[0096] The system uses the OpenVLA standard prompt word template. ,like Figure 4 The figure illustrates the overall architecture of OpenVLA: the system processes the input image and language commands through the DINOv2 / SigLIP visual encoder and the LLaMA Tokenizer (segmenter) respectively, and after alignment by the MLP Projector (projection layer), they are sent to LLaMA-2-7B for joint inference. Finally, the Action De-Tokenizer (action parsing tokenizer) decodes the model output into 7D action commands (position, pose, and grasp) that the robot can execute, realizing end-to-end vision-language-action mapping.

[0097] This embodiment will extract Fill it into the task slot {task}. For example, if For the task "Help the owner get the cup," the generated prompt would be "What should the robot do to help the owner get the cup? A:". This allows the VLA model to automatically plan low-level actions based on a high-level understanding of the task's objective.

[0098] (3) Action semantic execution module based on OpenVLA.

[0099] This module adopts an end-to-end VLA (Vision-Language-Action) large model architecture (specifically based on OpenVLA implementation), such as... Figure 4 As shown, its internal processing flow includes four key steps:

[0100] Step 1: Hybrid Visual Encoding:

[0101] The system receives real-time RGB images of the robot. To capture both semantic features and geometric details simultaneously, the system processes images in parallel using two pre-trained visual encoders:

[0102] a) DINOv2: Used to extract fine-grained geometric and spatial features.

[0103] b) SigLIP: Used to extract high-level semantic and texture features.

[0104] Subsequently, an MLP projection layer is used to fuse the features from both and map them onto the embedding space of the language model to generate a visual token sequence. :

[0105] ,

[0106] Where MLP represents the projection layer of a multilayer perceptron; Concat represents the feature stitching operation; This indicates a DINOv2 vision encoder; This indicates the SigLIP visual encoder.

[0107] Step Two: Instruction Tokenization:

[0108] LLaMA Tokenizer (tokenizer) for processing prompt words generated by the previous level Perform word segmentation and encoding to generate a text token sequence. .

[0109] Step 3: Cross-modal reasoning (Llama2 Backbone):

[0110] Visual Token With text token After concatenation, the data is input into the Llama-2-7B large language model. The model uses a multi-layer self-attention mechanism to understand the association between the image and the instruction, and predicts the probability distribution of the next token.

[0111] Step 4: Action De-Tokenization:

[0112] This is a crucial step in translating semantics into physical actions. The final hidden layer state output by the model does not pass through a standard text decoder, but is instead mapped through a set of Action De-Tokenizers (usually linear layers).

[0113] This module directly outputs 7-dimensional robot motion vectors. Includes the pose change of the end effector. and the opening and closing state of the grippers :

[0114] ,

[0115] in, This represents an action dediscreter, used to decode the high-dimensional hidden layer state into physical values.

[0116] This motion vector directly drives the joint controller of the physical robot to perform physical operations at high frequency.

[0117] The system framework of this embodiment is as follows: Figure 1 As shown in the diagram, this figure illustrates the core components of the system and their data flow connections. The first stage in the diagram is hierarchical mental reasoning based on the robot's central perspective. It demonstrates that the information flow sequentially passes through Level-1 Perception, Level-2 Mental Reasoning (including BDI loop reasoning encompassing Beliefs, Desires, and Intentions), and Level-3 (Generation). <decision>High-level strategies and <action>The processing of atomic action sequences.

[0118] The second stage in the diagram is the execution of action semantics based on OpenVLA. It demonstrates how the action sequence generated by Level-3 is transformed into semantic instructions through the Prompt construction and injection module. These instructions, along with real-time images captured by the camera, are input into the VLA large model (OpenVLA), and finally, the underlying control signals that drive the robot joints or end effectors are calculated and output.

[0119] Example 2:

[0120] To train the mental reasoning module in Example 1 to possess reasoning capabilities from a Robot-Centric perspective, this example constructs a dedicated dataset. For example... Figure 2 The first stage, as shown in the diagram, involves scene simulation and script generation. An initial story script was generated using a home simulation platform (VirtualHome / ThreeDWorld) and the GPT-4o large model. After manual selection and refinement, 590 interactive scenes covering different layouts and characters were ultimately constructed.

[0121] The second stage involves multimodal data acquisition, which involves executing script playback in the simulator and recording through a virtual camera in the robot's center-view to obtain video stream data containing the complete event process.

[0122] The third stage is six-layer hierarchical annotation, which is the core of dataset construction. A pyramid structure is used to perform six-layer hierarchical annotation on the collected data.

[0123] Final output: After the above three stages of processing and fine annotation, the dataset (green area on the right) was finally generated and used to train the robot's mental reasoning ability.

[0124] The build process is as follows:

[0125] (1) Scene simulation and script generation:

[0126] Step 1: Constructing the Parametric Prompt Template:

[0127] The system does not directly input open commands, but instead pre-configures a structured input vector containing four functional modules. :

[0128] ,

[0129] The specific definitions are as follows:

[0130] a) Role context Define the model role as "VirtualHome Simulation Script Generation Assistant".

[0131] b) State variables ( ): Defines dynamic interactive elements in the scene, including target objects. (e.g., perfume), initial position L1, intermediate position L2, and final position L3.

[0132] c) Event topology logic This is the core algorithmic logic for generating "false beliefs." The system forces the model to generate stories according to the following four-step spatiotemporal transition chain:

[0133] Character A places the object in L1;

[0134] Character B moves the object from L1 to L2;

[0135] Character C moves the object from L2 to L3;

[0136] (Key conflict point): Character A returns to L1 to look for the object (at this time the object is no longer in L1, thus constructing A's false belief).

[0137] d) Action space constraints ): The output action must belong to the set of atomic actions supported by the simulator.

[0138] ,

[0139] Among them, walk represents the movement action; pick_up represents the picking action; put_down represents the placement action; open represents the opening action; and close represents the closing action.

[0140] It also prohibits the inclusion of facial expressions or psychological descriptions, and only outputs physical action text.

[0141] Step 2: Few-Shot Enhancement:

[0142] To improve the stability of the generated format, the system injects N (e.g., ...) elements into the input. The standard "input-output" pairs serve as examples (Demonstrations) to guide the model in learning the mapping relationship from variables to story text. .

[0143] Step 3: Script generation and regular expression parsing:

[0144] The above-constructed The GPT-4o model is input and autoregressive generation is performed. The system receives the output natural language text and uses a regular expression parser (RegexParser) to verify whether it strictly follows the rules. The defined four-step logic. If the validation passes, the final script is output; if there is a logical break or illegal action, a regeneration mechanism is triggered. Ultimately, 590 interactive scripts that conform to the physical simulation logic were manually selected.

[0145] (2) Multimodal data acquisition:

[0146] The simulation was played back strictly according to the script, and video streams were captured through a virtual camera from the robot's perspective. Finally, 590 samples were generated, covering 8 apartment layouts and 16 different human roles, with each sample lasting approximately 1 to 2 minutes, ensuring the complete development of the event was included.

[0147] (3) Six-level hierarchical annotation (core step):

[0148] This embodiment constructs a hierarchical data protocol based on the robot-centric perspective, mapping unstructured video data into a six-dimensional semantic space. This process is not arbitrary but follows a strict state-space mapping function, specifically defined as follows:

[0149] Level I: Perception Layer:

[0150] Define the perception function This is used to parse unstructured visual information in video segment V into a structured set of objective states L1. This set consists of several semantic triples of "subject-action-object", and its formal expression is as follows:

[0151] ,

[0152] in, This represents the i-th interactive entity identified in the scene (e.g., Human_Alice, Robot). This indicates the j-th type of physical action that the subject is currently performing (constraints: limited to positional movement or object manipulation, such as Walk, Grasp; words expressing mental states such as "think" or "believe" are strictly prohibited). This represents the k target objects to which the action is applied (e.g., Apple, Fridge). A set of indices representing interactive entities; A set of indices representing categories of physical actions; A set of indices representing the target object.

[0153] This definition ensures that the perception layer only describes observable objective physical facts, providing unbiased basic data for subsequent second-order belief derivation.

[0154] Level II: Second-order belief difference modeling (Belief Layer):

[0155] This is the core feature of this dataset. Belief states are defined as binary tuples:

[0156] .

[0157] in, (Robot / God's-eye view) is the current real position of the object (e.g., "the apple is in the refrigerator"); (Inferred human perspective) refers to the location of an object inferred from the trajectory of human historical observations (e.g., "the last time humans saw an apple was on a table").

[0158] Level III: Desire & Intention Layer

[0159] Define wish D as the difference function of belief state:

[0160] ,

[0161] That is, the content of the desire must be "to eliminate the discrepancy between human cognition and objective facts" (e.g., "to help humans correct the mistake of looking for an apple on a table"). Intention I is the high-level commitment to achieving this desire.

[0162] Level IV: Atomized Action Serialization (Action Layer)

[0163] To accommodate the training requirements of VLA models, this step does not use natural language; instead, it transforms decisions into standardized sequences of atomic actions. Define the action space:

[0164] ,

[0165] in, Indicates moving to the target object; This indicates that the target object has been opened; This indicates that the target object is being grabbed.

[0166] The labeled output must strictly conform to the tuple sequence format:

[0167] ,

[0168] The structured protocol described above ensures that the dataset contains an explicit "false belief correction" logical chain, which can be used as a supervisory signal to train the mental reasoning model.

[0169] Example 3:

[0170] This embodiment details the training process of the mental reasoning module. For example... Figure 3 As shown, in order to enable the model to generate logically rigorous inference chains that conform to the BDI architecture, a two-stage training strategy of "supervised fine-tuning + reinforcement learning" is adopted:

[0171] Step 1: Supervised Fine-tuning (SFT):

[0172] This stage aims to enable the pre-trained model to initially grasp the six-layer output format and basic reasoning logic from perception to action through high-quality labeled data.

[0173] (1) Data construction:

[0174] The multimodal dataset constructed in Example 2 is divided into a training set. For each video sample, 32 frames of image sequences are uniformly sampled. and corresponding six-level hierarchical text annotations. They are then spliced ​​together to form input-output pairs.

[0175] (2) Model architecture:

[0176] Qwen2.5-VL-7B-Instruct was selected as the base model. This model includes a visual encoder and a language decoder (LLM Decoder).

[0177] Optimization objective: Employing standard autoregressive language modeling objectives, the model parameters are updated by minimizing the negative log-likelihood loss function. The calculation formula is as follows:

[0178] ,

[0179] in, The token predicted at the current moment. V represents the historically generated token sequence, and V represents the input video features.

[0180] (3) Training parameters:

[0181] The training epochs were set to 5, and the learning rate adopted a cosine annealing strategy to ensure that the model converged quickly and aligned with the underlying distribution.

[0182] Step 2: GRPO-based reinforcement learning optimization:

[0183] To address the issues that SFT models may encounter in complex scenarios, such as "accurate perception but broken reasoning logic" or "lack of proactive assistance intent," this embodiment introduces the GRPO (Group Relative Policy Optimization) algorithm for two-stage training.

[0184] Algorithm flow:

[0185] The GRPO algorithm does not rely on an additional value network (Critic), thus saving memory overhead. The specific iterative process is as follows:

[0186] (1) Sampling: For a given input problem q, the old policy model is used. Sample to generate a set of output sequences (In this embodiment, the group size G is set to 8).

[0187] (2) Evaluation: Calculate each output o using the Mind-Reward function described below. i The original reward value R i .

[0188] (3) Advantage Estimation: Calculate the relative advantage A within the group. i To reduce the variance of the gradient:

[0189] .

[0190] in, To minimize the value, prevent the denominator from being 0.

[0191] (4) Policy Update: By maximizing the following objective function To update the current policy model :

[0192] ,

[0193] in, For probability ratios, To limit the update step size for the trimming operation, This is a KL divergence penalty term used to prevent the model from deviating from the reference model. Too far. The number of iterations in this stage is set to 400.

[0194] Step 3: The specific calculation logic of the Mind-Reward function:

[0195] To accurately guide the optimization direction of GRPO, this invention designs a composite reward function R, which consists of mental reward (R0). mind ) and format rewards (R) format It consists of two parts, namely:

[0196] .

[0197] (1) Extraction and structuring of atomic actions:

[0198] The system introduces an auxiliary large language model (Qwen3-Max) as the parser and defines an extraction function. This function will process the natural language inference text generated by the model. Transformed into a standardized sequence of atomic actions Each element in the sequence is parsed into a structured tuple. (e.g., Robot, Walk, Fridge), thereby eliminating evaluation errors caused by the diversity of natural language expressions.

[0199] (2) Mental reward (R) mind Calculation formula:

[0200] The weighted summation of the multidimensional ROUGE index is calculated using the following formula:

[0201] .

[0202] In this embodiment, the weight is set as follows: The definitions of each indicator are as follows:

[0203] ① Atomic accuracy reward (R atomic ):

[0204] Calculated generated atomic action sequence With the sequence of truth values ROUGE-1 score:

[0205] ,

[0206] Right now:

[0207] ,

[0208] Where u represents the smallest semantic unit (unigram) in the atomic action sequence, i.e., a triple. Individual elements in the text (e.g., "Robot", "Walk", "Fridge"); This represents the total number of times the unit appears in the truth sequence (as the denominator, reflecting a recall-oriented approach). This indicates the total number of times the unit appears in the predicted sequence; The operation is used to calculate the number of overlaps (Matches) between the predicted sequence and the true sequence for that unit.

[0209] Technical effect: Ensures that the model correctly distinguishes between "robot perspective" and "human perspective" and accurately identifies the agent and object of the action.

[0210] ② Local consistency reward (R) local ):

[0211] Calculate adjacent inference levels (e.g.) arrive ROUGE-2 scores between )

[0212] ,

[0213] Right now:

[0214] ,

[0215] Among them, L k With L k+1 These represent two adjacent levels of text in the reasoning chain (e.g., L). k For the Belief layer, L k+1 (For the Desire layer); b represents a bigram in the text, that is, two adjacent semantic units (e.g., "Help->Owner"); L k+1 This indicates the frequency of the bigram phrase in text L, and N represents the total number of reasoning levels.

[0216] This formula essentially calculates the overlap rate (recall) of logical connectors between adjacent levels, which is used to quantify whether the reasoning process has the coherence of a "thought chain" and prevent the model from generating unrelated jumps in content.

[0217] Technical effect: Captures the causal coherence of reasoning chains in the short term, ensuring that desires are based on belief conflicts rather than being generated out of thin air.

[0218] ③ Global Consistency Reward (R) global ):

[0219] Calculation from initial perception To the final decision ROUGE-L score (longest common subsequence) of the entire long text sequence.

[0220] ,

[0221] Right now:

[0222] ,

[0223] in, For recall based on the longest common subsequence, For the accuracy based on the longest common subsequence, Function to calculate predicted text With truth text The longest common subsequence length between the two, and allows the subsequences to be discontinuous in the original text; β is an adjustment parameter (usually set to 1.2) used to balance recall and precision.

[0224] Technical effect: Ensures that the final decision is highly consistent with the initial objective perceptual input, and prevents semantic drift in the reasoning process.

[0225] (3) Formatted Rewards (R) format )calculate:

[0226] Define a Boolean verification function Using regular expression matching, check if the output text strictly contains and is in the following six tags in order:

[0227] .

[0228] The formula for calculating the format reward is as follows:

[0229] .

[0230] Technical effect: It forces the model to follow a specific thought chain format, which facilitates the parsing and execution of subsequent modules.

[0231] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.< / action> < / decision> < / decision> < / decision> < / perception>

Claims

1.A multi-modal mind theory based emotional service robot system, characterized by, The application relates to a hierarchical mind reasoning module, a connection module and an action semantic execution module. The hierarchical mind reasoning module is used for generating decision text containing high-level strategies according to multi-modal environment information collected by a robot. The connection module is used for constructing semantic instructions according to the decision text generated by the hierarchical mind reasoning module. The action semantic execution module is used for generating control actions of the robot according to the semantic instructions constructed by the connection module and real-time images collected by the robot. The hierarchical mind reasoning module comprises a perception unit, a mind reasoning unit and a decision unit connected in sequence. 2.The multi-modal mind theory based emotional service robot system according to claim 1, wherein, The perception unit is used for generating objective description text of an environment according to image sequences collected by the robot. The mind reasoning unit is used for inferring and generating belief, desire and intention states of the robot according to the objective description text generated by the perception unit. The decision unit is used for generating decision text containing high-level strategies according to the objective description text generated by the perception unit and the belief, desire and intention states generated by the mind reasoning unit. The working process of the perception unit comprises the following steps: 3.The multi-modal mental theory based emotional service robot system according to claim 2, wherein, visual features are extracted according to image sequences collected by the robot; the visual features are mapped to a word embedding space of a language model to obtain visual embedding vectors; objective description text of an environment is generated according to the visual embedding vectors and system prompt words. The working process of the mind reasoning unit comprises the following steps: 4.The multi-modal mental theory based emotional service robot system according to claim 2, wherein, fact states in the robot's perspective and human belief states inferred by the robot are inferred according to the objective description text; desire states of the robot are generated according to the difference between the fact states and the human belief states; intention states of the robot are generated according to the desire states. The working process of the decision unit comprises the following steps: 5.The multi-modal mental theory based emotional service robot system according to claim 2, wherein, the objective description text and the belief, desire and intention states are spliced to form a complete context; natural language decision text containing high-level strategies is generated through a self-recurrence generation mode according to the complete context. The working process of the connection module comprises the following steps: 6.The multi-modal mental theory based emotional service robot system according to claim 1, wherein, task descriptions are extracted from the decision text generated by the hierarchical mind reasoning module; the task descriptions are filled into a preset prompt word template to generate semantic instructions for the action semantic execution module. The working process of the action semantic execution module comprises the following steps: 7.The multi-modal mental theory based emotional service robot system according to claim 1, wherein, real-time images collected by the robot are hybridly coded to obtain a visual mark sequence; the semantic instructions generated by the connection module are textually coded to obtain a text mark sequence; cross-modal reasoning is performed on the visual mark sequence and the text mark sequence to predict a probability distribution of a next mark; a multi-dimensional action vector of the robot is mapped and generated according to a last hidden layer state output by a model. The step of hybridly coding comprises the following steps: 8.The multi-modal mental theory based emotional service robot system according to claim 7, wherein, geometric and spatial features of the image are extracted by using a first pre-trained visual encoder; semantic and texture features of the image are extracted by using a second pre-trained visual encoder; the geometric and spatial features and the semantic and texture features are fused and projected to generate the visual mark sequence. ​

Images