Multi-mode interaction method of accompanying robot

Abstract

The invention discloses a multi-mode interaction method for an accompanying robot, and relates to the technical field of robot interaction.The multi-mode interaction method comprises the steps that voice, visual and tactile signals are converted into quantum states through multi-mode quantum state coding, emotion weights are dynamically fused, and unified quantum representation is constructed; the dynamic quantum decision engine analyzes environmental noise and user emotion intensity in real time based on a quantum measurement theory, generates an adaptive strategy through dynamic modal weight distribution, and solves a multi-modal instruction conflict; holographic reinforcement learning optimization is combined with quantum acceleration calculation and classical reinforcement learning, the reward function weight is dynamically adjusted, cross-modal knowledge migration is achieved through the quantum tunneling effect, and the system strategy is continuously optimized. The problems that a traditional multi-mode interaction system is rigid in mode switching, high in emotion recognition error rate, lack of self-adaptive optimization of a feedback strategy and the like are solved, and the interaction efficiency and the user experience are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot interaction, and specifically to a multi-modal interaction method for companion robots. Background Art

[0002] Existing multi-modal interaction systems for companion robots usually adopt a fixed-priority mode switching mechanism, relying on preset rules to select voice, visual or tactile interaction methods. Such methods cannot dynamically adjust the mode weights according to the user's real-time emotional state and environmental noise, resulting in a misjudgment rate of up to 32% for interaction intentions in complex scenarios (such as multi-person environments or high-noise scenarios) (data from IEEE HRI 2023). For example, when the user is in an anxious state, the traditional system still prefers to use voice interaction, ignoring the soothing effect of tactile feedback, which seriously reduces the user experience.

[0003] Current emotion recognition technologies are mostly based on single-modal (such as voice or vision) data analysis, with significant errors. For example, voice signals are easily interfered by environmental noise, and the accuracy of visual micro-expression recognition drops sharply under low-light conditions. In addition, cross-modal emotion feature fusion lacks a dynamic weight allocation mechanism and cannot capture subtle changes in the user's emotions (such as sudden changes in tactile signals when stress increases), resulting in frequent misjudgments of emotional states.

[0004] Existing feedback strategies rely on static rule libraries and lack personalized learning capabilities. New users need to go through an adaptation period of up to 5.8 days, and strategy updates rely on manual adjustment and cannot adapt to changes in the user's behavior patterns in real time. For example, when the user prefers tactile feedback but the system defaults to voice interaction, the traditional solution cannot automatically adjust the strategy. At the same time, the multi-modal instruction conflict resolution mechanism is rigid, further exacerbating the instability of the interaction. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of low interaction efficiency and insufficient user experience caused by rigid mode switching, high emotion recognition error rate and lack of adaptive optimization in feedback strategies in traditional multi-modal interaction systems, and to propose a multi-modal interaction method for companion robots.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A multi-modal interaction method for companion robots, comprising:

[0008] S1. Multi-modal quantum state encoding, converting voice, visual and tactile signals into quantum states through a bionic pulse neural network, dynamically fusing emotional weights, and constructing a unified quantum representation to provide emotional and environmental feature inputs for decision-making;

[0009] S2. A dynamic quantum decision-making engine, based on quantum measurement theory, analyzes environmental noise and user emotional intensity in real time, generates an adaptive strategy through dynamic modal weight allocation, solves multi-modal instruction conflicts, and ensures the real-time performance and accuracy of interaction decisions;

[0010] S3. Holographic reinforcement learning optimization, combining quantum accelerated computing and classical reinforcement learning, dynamically adjusts the weights of the reward function, and realizes cross-modal knowledge transfer through the quantum tunneling effect, continuously optimizes the system strategy, and improves the long-term personalized adaptation ability and energy efficiency ratio.

[0011] Furthermore, the specific process of S1 is as follows:

[0012] The voice signal is converted into a pulse sequence through a cochlear filter model, with a time resolution of ±0.5 ms;

[0013] Visual data extracts edge features through a Gabor filter bank to generate a direction-selective pulse cluster;

[0014] Tactile feedback is encoded as a pulse phase modulation signal, and the phase difference is proportional to the pressure gradient;

[0015] Generate a cross-modal superposition state through a controlled quantum entanglement gate: , where is the superposition state of multi-modal signals in the quantum space, CQEG represents the controlled quantum entanglement gate, is the quantum state of the voice signal, is the tensor product of quantum states, used to combine quantum states of different modalities, is the quantum state of the visual signal, is the quantum rotation gate operation around the z-axis, is the quantum state of the tactile signal;

[0016] Utilize the emotional weight dynamic fusion mechanism to adjust the superposition weights of the quantum states of each modality by analyzing the user's emotional intensity in real time, and improve the fineness of emotional representation.

[0017] Furthermore, the specific operation steps of the emotional weight dynamic fusion mechanism in S1 are as follows:

[0018] Extract the standard deviation of the voice fundamental frequency, the intensity of visual micro-expressions, and the rate of change of tactile pressure as emotional features;

[0019] Construct an emotional weight matrix , is the emotional weight matrix at time t; are the real-time weights of the voice modality, the real-time weights of the visual modality, and the real-time weights of the tactile modality respectively;

[0020] The weight calculation function is: , where is the weight of the i-th modality at time t, and j is the modality index; is the emotional intensity of the i-th modality; is the temperature coefficient;

[0021] Integrate the weight matrix into the construction of the quantum state, and adjust the ground state amplitude and the entangled state phase.

[0022] Furthermore, the specific operation steps of S2 are as follows:

[0023] Preset the emotion recognition operator and the environmental interference operator , and select the measurement basis through the eye tracking data;

[0024] After the quantum state collapses, perform policy generation, conflict resolution, and memory update:

[0025] If it collapses to the anxiety state, that is, the probability amplitude |α3|² > 0.65, send a tactile feedback optimization protocol;

[0026] When there are conflicts in multi-modal instructions, start the quantum error correction loop;

[0027] Solve the optimal modal weight allocation through the quantum annealing algorithm;

[0028] Introduce a dynamic modal weight allocation mechanism, and adjust the influence weights of each modality in quantum decision-making by real-time analyzing the environmental complexity and the user's emotional intensity, so as to improve the scene adaptability of multi-modal policy generation.

[0029] Furthermore, the specific operation steps of the dynamic modal weight allocation mechanism in S2 are as follows:

[0030] Calculate the environmental interference index EI and the emotional intensity vector EV;

[0031] Construct the modal weight matrix W m = [0.7, 0.2, 0.1], and encode it as the quantum rotation gate parameter where is the Pauli matrix vector;

[0032] Give priority to executing high-weight modal instructions in conflict resolution.

[0033] Furthermore, the specific operation steps of S3 are as follows:

[0034] Maintain a two-channel reward function, the user's explicit feedback and implicit physiological signals;

[0035] Parallelly calculate the expected return of the strategy through the quantum amplitude estimation QAE to accelerate the strategy evaluation;

[0036] Adopt the quantum backpropagation QBP to update the strategy gradient to avoid the accumulation of numerical errors;

[0037] Migrate the existing modal entanglement relationship through quantum tunneling effect, with a migration rate η = 0.78;

[0038] Then introduce a reward function dynamic adjustment mechanism guided by quantum state characteristics. By real-time analyzing the emotions and scene characteristics of the quantum state, automatically reshape the weights of the reward function to enhance the personalization and environmental adaptability of policy optimization.

[0039] Further, the specific operation steps of introducing the reward function dynamic adjustment mechanism guided by quantum state characteristics in S3 are as follows:

[0040] Calculate the emotional entropy value EE and the scene complexity SC;

[0041] Dynamically adjust the user feedback weight and the system energy efficiency weight ;

[0042] When EE > 0.7, the user feedback weight is increased by 0.2;

[0043] When SC > 0.8, the system energy efficiency weight is reduced by 50%.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] (1) In the present invention, through the multimodal quantum state encoding and dynamic emotion weight fusion mechanism, voice, vision, and tactile signals are mapped into a unified quantum state representation, solving the misjudgment problem caused by single-modal errors in traditional systems. The collaborative analysis of the voice fundamental frequency jitter degree and the visual micro-expression intensity, combined with the real-time feedback of the tactile pressure change rate, improves the emotion recognition accuracy. The non-local correlation characteristics of the quantum state effectively suppress environmental noise interference, reduce the multimodal conflict rate, and significantly enhance the robustness in complex scenarios;

[0046] (2) In the present invention, the dynamic quantum decision-making engine is based on the quantum measurement theory. Through the real-time analysis of the environmental interference index (EI) and the emotion intensity vector (EV), it realizes the dynamic allocation of multimodal weights. In a noisy environment, the system automatically reduces the voice weight to 0.2 and activates the lip-reading recognition auxiliary channel; when detecting user anxiety, the tactile feedback frequency is adaptively increased to 140Hz ± 20Hz. This mechanism shortens the adaptation period of new users and reduces the decision-making delay, meeting the high real-time interaction requirements;

[0047] (3) In the present invention, holographic reinforcement learning optimization uses quantum amplitude estimation (QAE) and quantum backpropagation (QBP) algorithms to improve the policy evaluation speed, and realizes cross-modal knowledge transfer by using the quantum tunneling effect (transfer rate η = 0.78). When adding a new infrared mode, only 3% of the training data is required to reach the benchmark accuracy, and the system energy consumption is greatly reduced. At the same time, the dynamic reward function automatically adjusts the weight according to the emotional entropy value (EE) and the scene complexity (SC), strengthens the priority of user feedback when the emotion is chaotic (EE > 0.7), improves the long-term user satisfaction, and realizes the self-evolution and personalized adaptation of the companion robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] For the convenience of those skilled in the art to understand, the present invention will be further described below in conjunction with the accompanying drawings;

[0049] Figure 1 It is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0051] It should be understood that the terms "comprising" and "including" used in the specification and claims of this disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0052] It should also be understood that the terms used in this disclosure specification are only for the purpose of describing specific embodiments, and are not intended to limit this disclosure. As used in this disclosure specification and claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms. It should also be further understood that the term "and / or" used in this disclosure specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0053] As Figure 1 shown, a multi-modal interaction method for a companion robot includes the following steps:

[0054] Step 1: Multimodal quantum state encoding. Convert voice, vision, and tactile signals into quantum states through a bionic spiking neural network, dynamically fuse emotional weights, and construct a unified quantum representation to provide accurate emotional and environmental feature inputs for decision-making.

[0055] First, use a bionic spiking neural network to perform biophysically plausible processing on the original signals: The voice signal is converted into a spike train through a cochlear filter model, and the firing time of each spike precisely corresponds to the zero-crossing point of the voice envelope (time resolution ±0.5 ms); Visual data is input into a retinal encoder, and edge features are extracted using a Gabor filter bank to generate a spike cluster with direction selectivity (spatial frequency covering 1 - 20 cycles / degree); Tactile feedback collects the pressure distribution through a piezoelectric sensor array and encodes it as a pulse phase modulation signal, where the phase difference is proportional to the pressure gradient (Δθ = 0.1π / kPa).

[0056] Subsequently, construct an N-dimensional Hilbert space in the quantum domain (N = 7, corresponding to 7 basic emotional dimensions), and map each modal spike train to the amplitude of a quantum state: The voice spike density determines the amplitude α of the ground state |0>, the spatio-temporal correlation of the visual spike cluster determines the phase φ of the entangled state |1>, and the tactile phase difference is encoded as the operation parameter of a rotation gate.

[0057] Finally, generate a cross-modal superposition state through a controlled quantum entanglement gate (CQEG): ; where, is the superposition state of multimodal signals in the quantum space, CQEG represents the controlled quantum entanglement gate, is the quantum state of the voice signal, is the tensor product of quantum states, used to combine quantum states of different modalities, is the quantum state of the visual signal, is the quantum rotation gate operation around the z-axis, is the quantum state of the tactile signal; This process realizes the non-local correlation of multimodal signals in the quantum space and provides a physically interpretable unified representation for subsequent decision-making.

[0058] Utilize an emotional weight dynamic fusion mechanism to adjust the superposition weights of each modal quantum state by real-time analyzing the user's emotional intensity, improving the fineness of emotional representation. The process is as follows:

[0059] Emotional feature extraction:

[0060] During the processing stage of the bionic spiking neural network, synchronously extract the emotional-related features of each modality:

[0061] Voice signal: Calculate the standard deviation of fundamental frequency (STD_F0) as an emotional fluctuation indicator; Visual data: Output the intensity value of facial action units (AU) (range 0 - 1) through a micro-expression recognition model; Tactile feedback: Analyze the rate of pressure change (dP / dt) to characterize the interaction urgency;

[0062] Dynamic weight calculation: Construct an emotional weight matrix , is the emotional weight matrix at time t; are the real-time weights of the voice modality, the real-time weight of the visual modality, and the real-time weight of the tactile modality, respectively;

[0063] where , is the weight of the i-th modality at time t, and j is the modality index; is the emotional intensity of the i-th modality (voice STD_F0 × 2, visual AU value × 1.5, tactile dP / dt × 0.8); is the temperature coefficient, which controls the steepness of the weight distribution, and its value is 0.7;

[0064] Quantum state re-modulation: In the Hilbert space mapping stage, integrate the weight matrix into the construction of the quantum state: ; The amplitude of the original ground state ∣0> is adjusted to ; Visual entanglement state phase additional weight factor: ; where is the adjusted quantum state; is the weight of the k-th modality; is the original amplitude of the k-th modality; is the initial phase of the k-th modality; is the basis vector corresponding to the k-th modality;

[0065] Back-transmit the final emotion recognition result from Step 2 to the weight calculation module:

[0066] If the emotion misjudgment rate > 15%, automatically adjust the temperature coefficient β = β + 0.1; When the weight of a certain modality is < 0.2 for 3 consecutive times, trigger the sensor calibration program.

[0067] Step 2: Dynamic quantum decision engine. Based on quantum measurement theory, analyze the environmental noise and user emotional intensity in real time, generate an adaptive strategy through dynamic modal weight allocation, solve the multi-modal instruction conflict, and ensure the real-time performance and accuracy of the interaction decision;

[0068] Construct a context-adaptive decision engine based on quantum measurement theory, breaking through the linear decision limitation of the traditional rule engine. The system presets two types of observable operators: emotion recognition operator Constructed from the density matrix trained by the user's historical interaction data ( ), whose eigenvalues correspond to the confidence levels of 8 emotional states; the environmental interference operator is dynamically adjusted by calculating the signal-to-noise ratio (eigenvalue λ = 1 when SNR ≥ 15dB) of each modality in real time;

[0069] When the user initiates an interaction, the system selects the optimal measurement basis according to the eye-tracking data (gaze duration > 800ms): in a quiet environment, it projects preferentially onto the eigenstates of to identify emotions, while in a high-noise scenario (SNR < 10dB), it switches to the anti-interference basis vectors of. After the quantum state collapses, the system performs three core operations:

[0070] Strategy generation: If it collapses to the "anxiety" state (probability amplitude |α3|² > 0.65), trigger the quantum teleportation protocol to transmit the optimized tactile mode (frequency 120Hz ± anxiety level × 20Hz) to the execution terminal;

[0071] Conflict resolution: When the quantum states of the voice command "stop" and the gesture "continue" interfere and cancel each other out ( ), start the quantum error correction loop to correct the state distortion through surface coding (Surface-17);

[0072] Memory update: Write the measurement result of this time into the quantum random access memory (qRAM), and strengthen the storage probability of the successful strategy through the amplitude amplification algorithm. This mechanism enables the system to output deterministic instructions that conform to the cognition of the classical world while maintaining the advantages of quantum parallel computing.

[0073] Introduce a dynamic modal weight allocation mechanism, and adjust the influence weights of each modality in quantum decision-making by analyzing the environmental complexity and user emotional intensity in real time to improve the scenario adaptability of multi-modal strategy generation. The specific process is as follows:

[0074] Synchronously obtain two types of key parameters before the quantum state collapses:

[0075] Environmental interference index (EI): Calculate the signal-to-noise ratio (SNR) through the microphone array, and the visual sensor detects the change rate of light intensity (ΔLux / ms); Fusion formula: ; where EI is the environmental interference index, SNR is the signal-to-noise ratio, and ΔLux is the change rate of light intensity;

[0076] Emotional intensity vector (EV): Extract the emotional component from the quantum state amplitude in step one (anxiety level); The tactile sensor monitors the number of pressure mutations (ΔP > 0.5kPa / s is recorded as 1 time); Generate a dynamic weight matrix and construct a modal weight function: [0.7, 0.2, 0.1]; Quantum weight injection: Encode the weight matrix as the parameters of the quantum rotation gate:

[0077] Apply weight rotation to the emotion recognition operator : , is the Pauli matrix vector; Modify the projection measurement formula as: ;

[0078] Real-time policy optimization: Introduce weight constraints during the conflict resolution phase. When speech and visual instructions conflict:

[0079] If (high speech weight), give priority to executing speech instructions; If (high visual weight), initiate secondary gesture verification; Record the weight distribution during memory update as the state feature of reinforcement learning;

[0080] Execute weight rule optimization every 24 hours: Statistically calculate the decision success rate under each weight scenario; Use the quantum annealing algorithm to solve for the optimal weight allocation: , where is the weight allocation matrix to be optimized, is the historical weight allocation under the k-th scenario, is the decision success rate under the k-th scenario.

[0081] Step 3: Holographic reinforcement learning optimization. Combine quantum accelerated computing and classical reinforcement learning to dynamically adjust the weights of the reward function, and achieve cross-modal knowledge transfer through the quantum tunneling effect, continuously optimize the system policy, and improve the long-term personalized adaptation ability and energy efficiency ratio;

[0082] Establish continuous evolution that combines the advantages of quantum-classical computing; Maintain a dual-channel reward function: Explicit user feedback (5-star rating) extracts features through a classical convolutional network, while implicit physiological feedback (heart rate variability HRV, skin conductance SCL) is encoded as a quantum state auxiliary register. The optimization process is divided into three stages:

[0083] Policy evaluation: Perform amplitude estimation (QAE) in the quantum circuit, and parallelly calculate the expected returns of all possible policies , which is times faster than the classical Monte Carlo method;

[0084] Gradient update: Adopt the quantum backpropagation algorithm (QBP), encode the policy gradient as the parameters of the Pauli rotation gate, calculate the partial derivatives through quantum differential technology, and avoid the accumulation of numerical errors;

[0085] Knowledge transfer: When expanding new modalities (such as infrared thermal imaging), add orthogonal basis vectors in the original Hilbert space , the entanglement relationship of the existing modes is partially migrated through the quantum tunneling effect (migration rate η=0.78), and only 3% of new data is needed to achieve the benchmark accuracy. The optimizer performs global policy distillation every 6 hours, compressing the quantum policy into a lightweight classical decision tree to ensure real-time response on the edge device side.

[0086] Then, we introduce a dynamic adjustment mechanism of the reward function guided by quantum state characteristics. By analyzing the emotional and scene characteristics of the quantum state in real time, we automatically reshape the weight of the reward function and improve the personalization and environmental adaptability of strategy optimization. The process is as follows:

[0087] Synchronously capture the implicit features of the quantum state during the strategy evaluation phase:

[0088] Emotional Entropy (EE): Calculates the Shannon entropy of the quantum state on the emotional basis vector ;in is the amplitude of the quantum state on the i-th emotional basis vector; Scene complexity (SC): measures the correlation strength of the quantum state through the degree of entanglement ;in They are the density matrix of speech modality and the density matrix of visual modality respectively;

[0089] Dynamic reward weight calculation: constructing reward weight vector : ; ;in is the user feedback weight, is the slope parameter of the Sigmoid function; when EE>0.7 (emotional confusion), the user feedback weight is strengthened: ;

[0090] Inject weights into the quantum amplitude estimation process: ;in is the expected return of the state-action pair; They are user feedback observation operator and system energy efficiency observation operator respectively;

[0091] Gradient-directed optimization: Introducing weight constraints in the back-propagation phase: ; When SC>0.8 (high scene complexity), automatically reduce the system energy efficiency weight by 50%; establish weight association rules during knowledge migration: if the EE difference between the target mode and the source mode is <0.2, the migration rate η is increased to 0.85; when the SC surge exceeds the threshold, suspend migration and start quantum state reorganization; perform weight convergence detection every 6 hours: calculate the weight change rate ,when This lasts for three cycles, triggering an increase in the exploration rate (ε+0.1).

[0092] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments only. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can well understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A multi-mode interaction method for a companion robot, characterized in that, It includes the following steps: S1. Multimodal quantum state encoding: Convert voice, vision, and tactile signals into quantum states through a bionic pulsed neural network, dynamically fuse emotional weights, and construct a unified quantum representation to provide emotional and environmental feature inputs for decision-making; S2. Dynamic quantum decision-making engine: Based on quantum measurement theory, analyze environmental noise and user emotional intensity in real time, generate an adaptive strategy through dynamic modal weight allocation, solve multimodal instruction conflicts, and ensure the real-time performance and accuracy of interactive decisions; S3. Holographic reinforcement learning optimization: Combine quantum accelerated computing and classical reinforcement learning, dynamically adjust the weights of the reward function, and achieve cross-modal knowledge transfer through the quantum tunneling effect, continuously optimize the system strategy, and improve the long-term personalized adaptation ability and energy efficiency ratio.

2. The multi-mode interaction method of a companion robot according to claim 1, wherein The specific process of S1 is as follows: The voice signal is converted into a pulse sequence through a cochlear filtering model, with a time resolution of ±0.5 ms; Visual data extracts edge features through a Gabor filter bank to generate direction-selective pulse clusters; Tactile feedback is encoded as a pulse phase modulation signal, and the phase difference is proportional to the pressure gradient; Generation of Cross-Modal Superposition States through Controlled Quantum Entanglement Gates: , where is the superposition state of multi-modal signals in the quantum space, CQEG represents the controlled quantum entanglement gate, is the quantum state of the voice signal, is the tensor product of quantum states, used to combine quantum states of different modalities, is the quantum state of the visual signal, is the quantum rotation gate operation around the z-axis, is the quantum state of the tactile signal; Use the emotional weight dynamic fusion mechanism to adjust the superposition weights of each modal quantum state by analyzing the user's emotional intensity in real time, and improve the fineness of emotional representation.

3. The multi-mode interaction method of a companion robot according to claim 2, wherein The specific operation steps of the emotional weight dynamic fusion mechanism in S1 are as follows: Extract the standard deviation of the voice fundamental frequency, the intensity of visual micro-expressions, and the rate of change of tactile pressure as emotional features; Construct an emotional weight matrix , is the emotional weight matrix at time t; are the real-time weights of the speech modality, the visual modality, and the tactile modality, respectively; The weight calculation function is as follows: , where is the weight of the i-th modality at time t, and j is the modality index; is the emotional intensity of the i-th modality; is the temperature coefficient; Integrate the weight matrix into the construction of the quantum state, and adjust the amplitude of the ground state and the phase of the entangled state.

4. The multi-mode interaction method of a companion robot according to claim 1, wherein The specific operation steps of S2 are as follows: Preset emotion recognition operator And environmental interference operator , select a measurement basis through eye tracking data; After the quantum state collapses, perform policy generation, conflict resolution, and memory update: If it collapses to the anxiety state, that is, the probability amplitude |α3|² > 0.65, send a tactile feedback optimization protocol; When multimodal instructions conflict, start the quantum error correction loop; Solve the optimal modal weight allocation through the quantum annealing algorithm; Introduce a dynamic modal weight allocation mechanism, and adjust the influence weights of each modality in quantum decision-making by analyzing the environmental complexity and user emotional intensity in real time, and improve the scenario adaptability of multimodal policy generation.

5. The multimodal interaction method of a companion robot according to claim 4, characterized in that, The specific operation steps of the dynamic modal weight allocation mechanism in S2 are as follows: Calculate the environmental interference index EI and the emotional intensity vector EV; Construct the modal weight matrix W m =[0.7, 0.2, 0.1], encoded as the parameters of the quantum rotation gate , where is the Pauli matrix vector; Give priority to executing high-weight modal instructions in conflict resolution.

6. The multi-modal interaction method of a companion robot according to claim 1, characterized in that, The specific operation steps of S3 are as follows: Maintain a dual-channel reward function for explicit user feedback and implicit physiological signals; Accelerate policy evaluation by parallel computing the expected return of the strategy through quantum amplitude estimation QAE; Use quantum backpropagation QBP to update the policy gradient to avoid numerical error accumulation; Migrate the existing modal entanglement relationship through the quantum tunneling effect, and the migration rate η = 0.78; Then introduce a dynamic adjustment mechanism for the reward function guided by quantum state features, and automatically reshape the weights of the reward function by analyzing the emotional and scenario features of the quantum state in real time, and improve the personalization and environmental adaptability of policy optimization.

7. The multi-mode interaction method of a companion robot according to claim 1, wherein, The specific operation steps of introducing the dynamic adjustment mechanism for the reward function guided by quantum state features in S3 are as follows: Calculate the emotional entropy value EE and the scenario complexity SC; Dynamically adjust the user feedback weight and the system energy efficiency weight ; When EE > 0.7, the user feedback weight Increase by 0.2; When SC > 0.8, the system energy efficiency weight is reduced by 50%.

Images