A tabletop companion robot interaction control method and chip

Abstract

The application is suitable for the technical field of robot control, and provides a tabletop companion robot interaction control method and a chip. The method comprises the following steps: being applied to a tabletop companion robot multi-agent system integrated with one dialogue cognitive agent and multiple feedback execution agents; according to a dialogue emotion state value, performing consistent cooperative correction on a local emotion state value of the feedback execution agent to obtain a target emotion state value; generating an event trigger judgment result according to the target emotion state value and a dynamic event trigger threshold; when the event trigger judgment result is a trigger condition, performing hierarchical driving through a companion feedback execution layer; and when the event trigger judgment result is not a trigger condition, maintaining a companion feedback execution action output by the companion feedback execution layer at present. The application guarantees the real-time performance and accuracy of emotional feedback, and can intelligently adapt the driving strength according to the power state, so as to adapt to the use scene of the tabletop robot long-time standby companion.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to an interactive control method and chip for a desktop companion robot. Background Technology

[0002] Most existing desktop companion robots adopt an integrated processing architecture, with highly coupled internal speech perception, emotion computing, and feedback execution modules. The robot independently collects user speech signals through a built-in microphone, centrally performs speech recognition, semantic analysis, and emotion classification, and drives the execution structures such as lights, screen expressions, and limb servos to output companion feedback actions based on the single calculation result. In the traditional architecture, the various feedback execution units inside the robot respond independently without differentiation, and there is no state coordination or synchronization mechanism between the execution units.

[0003] When desktop companion robots are equipped with multiple feedback execution units to achieve multi-dimensional and three-dimensional companion interaction scenarios, the existing integrated architecture still adopts a working mode of synchronous operation of all modules and independent response of each execution unit. All feedback execution units need to continuously participate in state calculation and action judgment, which is prone to the problem of inconsistent companion feedback actions and emotional expressions of various execution units within the robot, reducing the immersion and coordination of human-robot companionship. Moreover, the traditional architecture requires the entire machine to continuously run high-frequency computing modules such as voice perception and emotion calculation. All execution units maintain real-time state detection and calculation throughout the process, resulting in a high frequency of invalid calculations and redundant judgments, leading to high overall power consumption of the robot. Summary of the Invention

[0004] This invention provides an interactive control method and chip for a desktop companion robot. This invention not only ensures the real-time and accuracy of emotional feedback, but also intelligently adapts the drive intensity according to the battery status to avoid high power consumption and full load operation. Under the premise of achieving highly consistent feedback from multiple intelligent agents of the robot and optimizing the interactive experience, it maximizes the reduction of the overall power consumption of the whole machine, and is suitable for the use scenario of desktop robots providing long-term standby companionship.

[0005] In a first aspect, embodiments of this application provide an interactive control method for a desktop companion robot, applied to a multi-agent system of a desktop companion robot integrating one dialogue cognitive agent and multiple feedback execution agents, wherein the dialogue cognitive agent has only a dialogue semantic perception layer, and each of the feedback execution agents has only a companion feedback execution layer, comprising the following steps: The user voice signal collected by the dialogue cognitive agent is input into the dialogue semantic perception layer for voice recognition and emotion classification to obtain the dialogue emotion state value. Based on the dialogue emotion state value, the local emotion state value of the feedback execution agent is consistent and collaboratively corrected to obtain the target emotion state value. The dynamic event triggering threshold is calculated based on the dialogue silence duration and remaining battery value of the intelligent agent according to the feedback, and an event triggering judgment result is generated based on the target emotional state value and the dynamic event triggering threshold. When the event trigger judgment result is that the trigger condition is met, the target emotional state value and the remaining battery value are input into the companion feedback execution layer for hierarchical driving, and the companion feedback execution action is output, and the target emotional state value is used as the local emotional state value at the most recent synchronization time; when the event trigger judgment result is that the trigger condition is not met, the companion feedback execution action currently output by the companion feedback execution layer is maintained.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the step of inputting the user's voice signal collected by the dialogue cognitive agent into the dialogue semantic perception layer for voice recognition and emotion classification to obtain a dialogue emotion state value includes: The dialogue cognitive agent collects user voice signals and performs voice endpoint detection and voice recognition on the user voice signals to obtain dialogue text. Calculate the fundamental frequency, speech rate, and energy parameters of the user's speech signal, and use the fundamental frequency, speech rate, and energy parameters as speech prosody parameters; Based on the prosodic parameters, the emotional tendency of the dialogue text is analyzed to obtain the emotional state value of the dialogue.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of performing sentiment analysis on the dialogue text based on the speech prosody parameters to obtain a dialogue emotional state value includes: The dialogue text is classified into sentiment tendencies to obtain candidate sentiment categories and category confidence scores. The speech prosody parameters are weighted and summed to calculate the acoustic sentiment intensity value. The confidence level of the candidate emotion category is adjusted based on the acoustic emotion intensity value to determine the dialogue emotion state value.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of performing consistent collaborative correction on the local emotional state value of the feedback execution agent based on the dialogue emotional state value to obtain the target emotional state value includes: Based on the dialogue emotional state value and the local emotional state value of the feedback execution agent, the cognitive tracking correction amount of the feedback execution agent is calculated. The cognitive tracking correction amount is used to enable the feedback execution agent to track the global emotional state of the dialogue cognitive agent. The neighbor collaboration correction amount is calculated based on the local emotional state value of the feedback execution agent and the local emotional state value of each neighboring feedback execution agent at the most recent synchronization time. The neighbor collaboration correction amount is used to keep the emotional state of multiple feedback execution agents consistent. The cognitive tracking correction amount and the neighbor collaborative correction amount are superimposed, and the local emotional state value of the feedback execution agent is subjected to consistent collaborative correction to obtain the target emotional state value.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of calculating the neighbor cooperative correction amount based on the local emotional state value of the feedback execution agent and the local emotional state values ​​of each neighboring feedback execution agent at the most recent synchronization time includes: Based on the local emotional state value of the feedback execution agent and the local emotional state value of each neighboring feedback execution agent at the most recent synchronization time, multiple neighbor deviation values ​​are calculated. Based on the preset adjacency weights of the intelligent agent communication topology, the multiple neighbor deviation values ​​are weighted and summed to obtain the neighbor collaborative correction amount.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of calculating a dynamic event triggering threshold based on the dialogue silence duration and remaining battery value of the feedback execution agent, and generating an event triggering judgment result based on the target emotional state value and the dynamic event triggering threshold, includes: Based on the feedback, the dialogue silence duration and remaining battery value of the intelligent agent are used to calculate the threshold amplification factor, and the preset benchmark threshold is adjusted based on the threshold amplification factor to obtain the dynamic event triggering threshold. An event triggering judgment result is generated based on the target emotional state value, the local emotional state value of the feedback execution agent, and the dynamic event triggering threshold.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of generating an event triggering judgment result based on the target emotional state value, the local emotional state value of the feedback execution agent, and the dynamic event triggering threshold includes: The emotional deviation value is calculated based on the target emotional state value and the local emotional state value of the feedback execution agent, and the emotional deviation value is normalized to obtain the degree of difference. The degree of difference is compared with the dynamic event triggering threshold to obtain the event triggering judgment result.

[0012] Optionally, in the seventh implementation of the first aspect of the present invention, when the event triggering judgment result is that the triggering condition is met, the target emotional state value and the remaining battery value are input into the companion feedback execution layer for hierarchical driving, and the companion feedback execution action is output, and the target emotional state value is used as the local emotional state value at the most recent synchronization time; when the event triggering judgment result is that the triggering condition is not met, the companion feedback execution action currently output by the companion feedback execution layer is maintained, including: When the event triggering judgment result is that the triggering condition is met, the corresponding execution power level is matched according to the remaining power value of the intelligent agent based on the feedback; The companionship feedback execution layer is driven according to the target emotional state value and the execution power level, and the companionship feedback execution action is output. The target emotional state value is used as the local emotional state value at the most recent synchronization time. When the event triggering judgment result is that the triggering condition is not met, the companion feedback execution action currently output by the companion feedback execution layer is maintained.

[0013] Optionally, in an eighth implementation of the first aspect of the present invention, the step of driving the companionship feedback execution layer according to the target emotional state value and the execution power level, outputting a companionship feedback execution action, and using the target emotional state value as the local emotional state value at the most recent synchronization moment includes: The target feedback action parameter set is obtained by matching the target emotional state value with a basic feedback action parameter set and adjusting the amplitude of the basic feedback action parameter set based on the execution power level. The target feedback action parameter group is input into the companion feedback execution layer, the companion feedback execution action is output, and the target emotional state value is used as the local emotional state value at the most recent synchronization moment.

[0014] Secondly, embodiments of this application provide a desktop companion robot interactive control chip for executing the steps of the above-described desktop companion robot interactive control method.

[0015] In one solution provided by the aforementioned desktop companion robot interactive control method and chip, this invention achieves complete decoupling of the robot's cognitive perception and feedback execution functions. Only an independent dialogue cognitive agent carries a dialogue semantic perception layer, uniformly completing voice acquisition, voice recognition, and emotion classification to generate a unique dialogue emotion state value. Each feedback execution agent is only configured with a companion feedback execution layer, eliminating the need for continuous high-load computations such as voice processing and emotion calculation. This completely avoids the repetitive computation problems caused by independent computation of traditional multi-execution units, reducing the continuous computational burden on the entire robot and effectively reducing idle power consumption. This invention constructs a two-layer emotion consistency collaborative correction mechanism for multiple feedback execution agents. It combines the deviation of the dialogue emotion state value output by the dialogue cognitive agent with the historical synchronous emotion state deviation of each adjacent feedback execution agent to perform weighted collaborative correction of each local emotion state value. Compared to the traditional method of independent state updates by each execution unit, this invention ensures a high degree of coordination and uniformity in the emotion state and feedback actions of all feedback execution agents within the robot, solving the problem of disordered and inconsistent multi-dimensional companion feedback performance and improving the overall integrity and immersive experience of human-computer companion interaction. This invention employs an asynchronous event-triggered state synchronization mechanism. Each feedback execution agent only transmits its local emotional state value from the most recently synchronized moment to its neighboring agents at the trigger moment, eliminating the need for continuous, all-day state interaction and data synchronization. This reduces the frequency of data interaction and collaborative computation among agents within the robot, further optimizing overall energy efficiency. This invention designs a dynamic event triggering threshold mechanism that adapts to interaction and battery status. Based on the dialogue silence duration and remaining battery value, it dynamically amplifies a preset baseline threshold, replacing the traditional fixed threshold and periodic forced triggering method. It can adaptively adjust the trigger sensitivity according to the robot's actual interaction scenario and remaining battery status, effectively avoiding invalid state comparisons and redundant action driving problems in non-interactive scenarios, further reducing unnecessary power consumption. This invention implements hierarchical drive control of feedback actions. When trigger conditions are met, the corresponding execution power level is matched according to the remaining battery of the feedback execution agent, and the amplitude of the feedback action parameters is adaptively adjusted based on the corrected emotional state value; when trigger conditions are not met, the original output state is maintained. It ensures the real-time and accuracy of emotional feedback, and can intelligently adapt the drive intensity according to the battery status to avoid high power consumption and full load operation. While achieving high consistency of feedback from multiple intelligent agents of the robot and optimizing the interactive experience, it minimizes the overall power consumption of the whole machine and is suitable for desktop robots to accompany users for a long time. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a multi-agent system for a desktop companion robot according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating an interactive control method for a desktop companion robot according to an embodiment of the present invention; Figure 3 yes Figure 2 A schematic diagram of the implementation process of step S101; Figure 4 yes Figure 2 A schematic diagram of the implementation process of step S102; Figure 5 yes Figure 2 A schematic diagram of the implementation process of step S103; Figure 6 yes Figure 2 A schematic diagram of the implementation process of step S104. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. It should also be understood that, as used in this specification and the appended claims, the term "and / or" refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0020] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0021] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0022] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0023] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0024] To address the problems mentioned above in the background art, this application proposes an interactive control method and chip for a desktop companion robot. The interactive control method for a desktop companion robot provided by this invention can be applied to applications such as... Figure 1 The desktop companion robot multi-agent system shown includes a dialogue cognitive agent and multiple feedback execution agents. The dialogue cognitive agent only has a dialogue semantic perception layer, and each of the feedback execution agents only has a companion feedback execution layer.

[0025] In one embodiment, such as Figure 2 As shown, a desktop companion robot interactive control method is provided, which is applied to... Figure 1 Taking the desktop companion robot multi-agent system as an example, the following steps are included: S101: Input the user voice signal collected by the dialogue cognitive agent into the dialogue semantic perception layer for voice recognition and emotion classification to obtain the dialogue emotion state value; S102: Based on the dialogue emotion state value, perform consistent collaborative correction on the local emotion state value of the feedback execution agent to obtain the target emotion state value; S103: Calculate the dynamic event trigger threshold based on the dialogue silence duration and remaining battery value of the intelligent agent, and generate the event trigger judgment result based on the target emotional state value and the dynamic event trigger threshold; S104: When the event trigger judgment result is that the trigger condition is met, the target emotional state value and the remaining battery value are input into the companion feedback execution layer for hierarchical driving, the companion feedback execution action is output, and the target emotional state value is used as the local emotional state value at the most recent synchronization moment; when the event trigger judgment result is that the trigger condition is not met, the companion feedback execution action currently output by the companion feedback execution layer is maintained.

[0026] This embodiment completely decouples the robot's cognitive perception and feedback execution functions. Only an independent dialogue cognitive agent carries a dialogue semantic perception layer, uniformly completing voice acquisition, voice recognition, and emotion classification to generate a unique dialogue emotion state value. Each feedback execution agent is only configured with a companion feedback execution layer, eliminating the need for continuous high-load computations such as voice processing and emotion calculation. This completely avoids the repetitive computation problems caused by independent computation of traditional multi-execution units, reducing the continuous computational burden on the robot and effectively lowering idle power consumption. This embodiment constructs a two-layer emotion consistency collaborative correction mechanism for multiple feedback execution agents. It combines the deviation of the dialogue emotion state value output by the dialogue cognitive agent with the historical synchronous emotion state deviation of each adjacent feedback execution agent to perform weighted collaborative correction on each local emotion state value. Compared to the traditional method of each execution unit independently updating its state, this embodiment ensures a high degree of coordination and uniformity in the emotion states and feedback actions of all feedback execution agents within the robot, solving the problem of disordered and inconsistent multi-dimensional companion feedback performance and improving the overall integrity and immersive experience of human-machine companion interaction. This embodiment employs an asynchronous event-triggered state synchronization mechanism. Each feedback execution agent only transmits its local emotional state value from the most recently synchronized moment to its neighboring agents at the trigger moment, eliminating the need for continuous, all-day state interaction and data synchronization. This reduces the frequency of data interaction and collaborative computation among agents within the robot, further optimizing overall energy efficiency. This embodiment also features a dynamic event triggering threshold mechanism that adapts to interaction and battery status. Based on the dialogue silence duration and remaining battery value, a preset baseline threshold is dynamically amplified, replacing the traditional fixed threshold and periodic forced triggering method. This mechanism can adaptively adjust trigger sensitivity according to the robot's actual interaction scenario and remaining battery status, effectively avoiding invalid state comparisons and redundant action driving issues in non-interactive scenarios, further reducing unnecessary power consumption. This embodiment implements hierarchical drive control of feedback actions. When trigger conditions are met, the corresponding execution power level is matched based on the remaining battery of the feedback execution agent, and the amplitude of the feedback action parameters is adaptively adjusted in conjunction with the corrected emotional state value. When trigger conditions are not met, the original output state is maintained. It ensures the real-time and accuracy of emotional feedback, and can intelligently adapt the drive intensity according to the battery status to avoid high power consumption and full load operation. While achieving high consistency of feedback from multiple intelligent agents of the robot and optimizing the interactive experience, it minimizes the overall power consumption of the whole machine and is suitable for desktop robots to accompany users for a long time.

[0027] In one embodiment, such as Figure 3 As shown, step S101 specifically includes the following steps: S201: Collect user voice signals through a dialogue cognitive agent, and perform voice endpoint detection and speech recognition on the user voice signals to obtain dialogue text; S202: Calculate the fundamental frequency, speech rate, and energy parameters of the user's speech signal, and use the fundamental frequency, speech rate, and energy parameters as speech prosody parameters; S203: Perform sentiment analysis on the dialogue text based on the phonetic prosody parameters to obtain the dialogue emotional state value.

[0028] In this embodiment, the dialogue cognitive agent continuously collects user voice signals through the built-in microphone array of the desktop companion robot. The collected multi-channel voice data is processed synchronously according to a unified sampling clock. The voice sampling rate can be preset to 16 kHz and the quantization bit width to 16 bits to meet the voice recognition needs of daily near-field human-computer dialogue, while controlling the computational load on the edge. During continuous voice acquisition, the gain of each channel signal is calibrated, and the enhanced voice signal corresponding to the direction of the user's sound source is selected based on the beamforming results of the microphone array. Then, the effective voice interval is identified using voice endpoint detection. The voice signal can be divided into frames with a single frame length of 20 ms and an adjacent frame shift of 10 ms. The short-time energy and zero-crossing features of each frame are extracted. When the voice activity determination results of 5 consecutive frames meet the effective voice condition, the user is determined to have started speaking. When no effective voice activity is detected for 30 consecutive frames, the current sentence is determined to have ended, thus extracting an effective voice segment corresponding to a complete user speech. In the case of echoes from fans, motors, or speakers in the environment, echo suppression and background noise suppression can be performed before endpoint detection to prevent the voice played by the robot's own speakers from being mistaken for new user input. Valid speech segments are input into the speech recognition module. Through acoustic feature extraction, speech encoding, and text decoding, the corresponding dialogue text is obtained, and the start time, end time, and duration of the valid speech segment are saved simultaneously, establishing a one-to-one correspondence between the obtained dialogue text and the original speech segment. Prosodic features are calculated for the valid user speech segments. The fundamental frequency (FFM) is used to characterize the periodicity of vocal cord vibration and pitch changes during the user's phonation. The FFM is extracted frame by frame within the valid speech interval where endpoint detection has been completed, and unvoiced frames without a stable FFM are filtered out. The FFM results of the remaining audible frames are then statistically analyzed to obtain a fundamental frequency parameter that represents the overall pitch level of the speech. The speech rate is determined based on the dialogue text and speech duration corresponding to the same valid speech segment. The number of valid syllables or characters recognized per unit time can be counted to characterize the speed of the user's current speaking rhythm. The energy parameter is calculated based on the energy envelope of the valid speech segment. The amplitude energy of each speech frame is statistically analyzed, and a time window smoothing process is used to reduce the impact of individual instantaneous peaks on the overall result. To improve the comparability of parameters obtained from different users, speaking distances, and microphone gains, the robot can record the user's normal fundamental frequency, normal speech rate, and normal energy level when it first establishes a user interaction baseline. During the interaction, the current parameters can be relatively calibrated. For example, the speech confirmed to be in a normal interaction state within the last 30 seconds can be used as a short-term reference window to reduce the interference of the user's natural pitch and volume differences on emotion judgment.After processing, the fundamental frequency, speech rate, and energy parameters corresponding to the current speech segment are used together as speech prosody parameters, and are bound and saved with the corresponding dialogue text to ensure that the semantic content and acoustic performance come from the same user speech.

[0029] In this embodiment, the dialogue text is input into the sentiment analysis module to identify semantic information indicating positive, negative, or neutral sentiments. Simultaneously, fundamental frequency, speech rate, and energy parameters corresponding to the dialogue text are introduced, allowing the semantic judgment to be corrected based on the user's actual speaking state. For example, the same sentence with a neutral literal meaning can reflect stronger excitement when the user's fundamental frequency is significantly increased, speech rate is faster, and speech energy is greater; conversely, it can reflect lower emotional activity when the fundamental frequency is lower, speech rate is slower, and speech energy is weaker. Therefore, candidate emotion categories and their corresponding confidence levels can be obtained first, and then acoustic emotion intensity information formed by speech prosody parameters can be used to correct the confidence level of each category, thus determining the dialogue emotion state value corresponding to the user's speech. After performing sentiment analysis on the dialogue text based on speech prosody parameters to obtain the dialogue emotional state value, the process further includes: constructing a four-component dialogue semantic state vector by combining the dialogue emotional state value, the semantic intent number extracted from the dialogue text, the current dialogue turn count, and the energy envelope value of the user's voice signal, which serves as the global baseline state output by the dialogue cognitive agent; the dialogue cognitive agent pushes the four-component dialogue semantic state vector to each feedback execution agent through an internal state distribution channel; after receiving the four-component dialogue semantic state vector, each feedback execution agent maps the dialogue emotional state value to the display screen expression number, jointly maps the semantic intent number and the dialogue turn count to the head servo target angle and the eye servo target angle, and maps the energy envelope value to the ambient light brightness parameter and the speaker volume gain parameter; each feedback execution agent combines the mapped display screen expression number, head servo target angle, eye servo target angle, ambient light brightness parameter, and speaker volume gain parameter into a local execution parameter set, which is used to subsequently drive the companion feedback execution layer to output the corresponding multimodal companion feedback execution action.

[0030] In one embodiment, step S203 specifically includes the following steps: The dialogue text is classified into sentiment tendencies to obtain candidate sentiment categories and category confidence scores. The speech prosody parameters are weighted and summed to calculate the acoustic sentiment intensity value. The confidence level of candidate emotion categories is adjusted based on the acoustic emotion intensity value to determine the emotion state value of the dialogue.

[0031] In this embodiment, the dialogue text obtained from speech recognition is input into the sentiment classification model. The model performs word segmentation, semantic encoding, and contextual feature extraction on the text, focusing on identifying words with emotional orientation, negation relations, degree adverbs, and semantic relationships between sentences, thereby outputting prediction results for multiple preset sentiment categories. Candidate sentiment categories can be preset to three types: positive, neutral, and negative. The classification model outputs the corresponding category confidence scores for each, and selects the category with the highest confidence score as the current candidate sentiment category. For example, if the confidence score for the positive category is 0.72, the confidence score for the neutral category is 0.19, and the confidence score for the negative category is 0.09, then positive is determined as the candidate sentiment category, while 0.72 is retained as a category confidence score to be corrected. Meanwhile, to avoid the natural differences in pitch, speaking speed, and volume among different users directly affecting emotion judgment, the current fundamental frequency, speaking speed, and energy parameters are normalized using the aforementioned user's normal fundamental frequency, normal speaking speed, and normal energy parameters as a reference, so that the three speech prosody parameters are on a uniform numerical scale. Then, they are weighted and summed according to preset weights, where the weight of the fundamental frequency parameter can be preset to 0.4, the weight of the speaking speed parameter to 0.3, and the weight of the energy parameter to 0.3, thus obtaining the acoustic emotion intensity value. When the fundamental frequency increases significantly, the speaking speed increases, and the speech energy increases, the acoustic emotion intensity value increases accordingly. When all three parameters are close to the user's normal level, the acoustic emotion intensity value remains in the middle range, thus reflecting the degree of emotional activation of the current speech expression relative to the user's normal speaking style. This embodiment uses acoustic emotion intensity values ​​to correct the confidence of candidate emotion categories generated from text classification. It determines whether the acoustic performance and candidate emotion categories show a consistent trend. For example, if the candidate emotion category is positive and the current acoustic emotion intensity is significantly higher than the user's normal level, the confidence of the positive category can be increased. Similarly, if the candidate emotion category is negative and the speech rate decreases, the fundamental frequency drops, and the energy weakens, the confidence of the negative category can also be enhanced. Conversely, if the candidate emotion category given by the text is significantly inconsistent with the speech prosody, the confidence of that candidate category is reduced to avoid misjudgment based solely on the text content. The magnitude of a single correction can be limited to within 0.15. For example, if the original confidence of the positive emotion category is 0.72, and the acoustic emotion intensity is highly consistent with the positive emotion performance, the corrected confidence can be increased to 0.82. However, if the acoustic performance is significantly inconsistent with the positive emotion, it can be reduced to 0.62, and the corrected confidence of each emotion category is re-compared. The emotion category with the highest corrected confidence is determined as the final emotion category corresponding to the current user's voice. Positive emotions, neutral emotions, and negative emotions are mapped to 1, 0, and -1, respectively. The corrected confidence of this final emotion category is combined to determine the dialogue emotion state value, so that the dialogue emotion state value simultaneously represents the user's current emotional direction and the reliability of the emotion judgment.Before calculating the cognitive tracking correction amount of the feedback execution agent based on the dialogue emotion state value and the local emotion state value of the feedback execution agent, the following steps are included: constructing the directed communication topology of the agents within the desktop companion robot, using the dialogue cognitive agent as the global baseline state generation node, configuring communication connection relationships between each feedback execution agent and the dialogue cognitive agent, and with other neighboring feedback execution agents, generating an agent adjacency matrix, where the elements of the adjacency matrix represent the connection weights between agents; calculating the Laplace matrix of the agent communication topology based on the adjacency matrix, extracting the minimum non-zero eigenvalue of the Laplace matrix, and configuring a constraint gain parameter for each feedback execution agent, where the constraint gain parameter represents the state tracking strength between the feedback execution agent and the dialogue cognitive agent; setting convergence constraints for consistency correction based on the minimum non-zero eigenvalue and the constraint gain parameter, ensuring that after the superposition and update of the cognitive tracking correction amount and the neighbor collaborative correction amount, the local emotion state value of each feedback execution agent can stably converge to the global baseline state of the dialogue cognitive agent, and the state deviation between each feedback execution agent converges to a preset bounded interval.

[0032] In one embodiment, such as Figure 4 As shown, step S102 specifically includes the following steps: S301: Calculate the cognitive tracking correction amount of the feedback execution agent based on the dialogue emotion state value and the local emotion state value of the feedback execution agent. The cognitive tracking correction amount is used to enable the feedback execution agent to track the global emotion state of the dialogue cognitive agent. S302: Calculate the neighbor cooperation correction amount based on the local emotional state value of the feedback execution agent and the local emotional state value of each neighboring feedback execution agent at the most recent synchronization time. The neighbor cooperation correction amount is used to keep the emotional state of multiple feedback execution agents consistent. S303: The cognitive tracking correction amount and the neighbor collaborative correction amount are superimposed, and the local emotional state value of the feedback execution agent is consistent and collaboratively corrected to obtain the target emotional state value.

[0033] In this embodiment, the dialogue emotion state value output by the dialogue cognitive agent is used as the robot's current global emotion reference state. The local emotion state value saved by the feedback execution agent to be corrected is also read. This local emotion state value corresponds to the emotion state maintained by the feedback execution agent after its most recent state synchronization and companionship feedback update. To ensure that different feedback execution agents can stably track the same global emotion state, a directed communication topology is pre-constructed within the robot before consistency correction. The dialogue cognitive agent is set as the global baseline state generation node. Communication connections are established between the display screen expression feedback agent, head servo feedback agent, eye servo feedback agent, ambient light feedback agent, and speaker feedback agent, and between these agents and the dialogue cognitive agent and adjacent feedback execution agents. Corresponding adjacency weights and constraint gain parameters are also configured. For example, the constraint gain between the feedback execution agent and the dialogue cognitive agent can be preset to 0.6, and the adjacency weight between the feedback execution agent and each adjacent feedback execution agent can be preset to 0.2, making the corrective effect of the global emotion state on the feedback execution agent higher than the collaborative effect of a single adjacent agent. The current dialogue emotion state value is compared with the local emotion state value of the feedback execution agent to determine the direction and degree of deviation between the two. This deviation is then combined with the corresponding constraint gain to form a cognitive tracking correction quantity. For example, if the dialogue cognitive agent has switched from a neutral state to a strongly positive state, while a feedback execution agent remains in the neutral state of the previous moment, the cognitive tracking correction quantity points towards the positive state. Conversely, if the local emotion state of the feedback execution agent exceeds the current global emotion state, the correction direction is reversed, thus causing the feedback execution agent to continuously approach the global emotion state output by the dialogue cognitive agent. Simultaneously, to avoid the problem of local response asynchrony caused by each feedback execution agent independently tracking the global emotion state, the local emotion state values ​​saved at the most recent synchronization moment of each neighboring feedback execution agent with which the current feedback execution agent has a communication connection are read. These most recent synchronization states are then compared with the local emotion state of the current feedback execution agent to obtain multiple neighbor deviations. These deviations are then weighted and summarized according to the pre-configured agent communication topology adjacency weights to form a neighbor collaborative correction quantity. By using the state at the most recent synchronization moment instead of requiring neighboring feedback agents to continuously send the current state in real time, multiple feedback agents such as displays, servos, ambient lights, and speakers can exchange necessary emotional state data only when an event is triggered, thereby reducing the number of intra-machine communications and the amount of collaborative computation. The role of neighbor collaborative correction is mainly to eliminate the relative state deviation between different feedback agents, so that even if a certain agent is affected by local response delays, actuator lags, or communication time differences, it can gradually recover to a consistent emotional state with the help of the state information of neighboring feedback agents.

[0034] In this embodiment, both cognitive tracking correction and neighbor collaborative correction are applied to the local emotional state of the current feedback execution agent. The cognitive tracking correction primarily ensures that the feedback execution agent does not deviate from the global emotional state given by the dialogue cognitive agent, while the neighbor collaborative correction primarily suppresses local state differences between multiple feedback execution agents. The two are superimposed to form a complete consistent collaborative correction, and the local emotional state is updated based on this correction result to obtain the target emotional state value. To prevent abrupt changes in facial expressions, servo movements, or lighting effects due to excessively large correction magnitudes, the upper limit of a single emotional state correction magnitude can be preset to 0.20 of the normalized emotional state range. When the superimposed correction magnitude exceeds 0.20, it is executed according to 0.20; otherwise, it is executed according to the actual correction magnitude, allowing the emotional state to converge smoothly in stages. When, for example, the global emotional state has already shown a positive trend, while the local emotional state of a certain head servo feedback agent is still biased towards neutral, and its neighboring facial expression feedback agents and lighting effect feedback agents have already updated to a positive state, cognitive tracking correction and neighbor collaboration correction will jointly promote the head servo feedback agent to adjust to a positive state, so that the final target emotional state value not only follows the current user's dialogue emotion, but also maintains coordination with other companion feedback units of the robot.

[0035] In one embodiment, step S302 specifically includes the following steps: Calculate multiple neighbor deviation values ​​based on the local emotional state value of the feedback execution agent and the local emotional state value of each neighboring feedback execution agent at the most recent synchronization time. Based on the preset adjacency weights of the agent communication topology, the neighbor deviation values ​​are weighted and summed to obtain the neighbor collaborative correction amount.

[0036] In this embodiment, based on the pre-established directed communication topology of agents within the desktop companion robot, the set of neighboring feedback agents corresponding to the current feedback agent is determined. The adjacency relationship is determined by whether state synchronization communication is allowed between agents. For example, the display screen expression feedback agent can be adjacent to the head servo feedback agent, eye servo feedback agent, and ambient light feedback agent, without requiring all feedback agents to be directly connected to each other. This ensures multimodal feedback coordination while controlling internal communication volume. The local emotional state value saved by the current feedback agent is read, and the local emotional state value saved by each neighboring feedback agent at the most recent synchronization moment when the event trigger condition was met and state synchronization was completed is also read. The local emotional state of the current feedback agent is then compared one by one with the most recent synchronization states of each neighboring feedback agent, thereby obtaining multiple neighbor deviation values ​​corresponding to each neighboring feedback agent. The neighbor deviation value not only represents the magnitude of the difference in emotional state between two feedback execution agents, but also preserves the direction of state adjustment. For example, if the current feedback execution agent's local emotional state is biased towards neutral, while the most recently synchronized state of the neighboring display expression feedback agent is biased towards positive, the corresponding neighbor deviation should push the current feedback execution agent to correct in a positive direction. Conversely, if the current feedback execution agent's emotional state is already higher than the most recently synchronized state of the neighboring agent, the corresponding deviation will have a corrective effect in the opposite direction. Since the comparison involves the state saved at the most recently synchronized moment of the neighboring agents, it is not necessary for each feedback execution agent to continuously broadcast its real-time state. It is only necessary to update the state data available for neighbors to read after the event is triggered and synchronization is completed. This allows the use of historical synchronization states to maintain the collaborative relationship between multiple execution units, thereby reducing the number of master wake-up calls and data transmission power consumption caused by continuous communication. For the same current feedback execution agent, if it has three neighboring feedback execution agents, the three neighbor deviation values ​​are calculated sequentially, and these deviation results are simultaneously retained within the current consistency and collaborative correction cycle. In this embodiment, adjacency weights are configured for different adjacent connections based on a pre-established agent communication topology, allowing different neighboring feedback execution agents to have varying degrees of influence on the collaborative correction of the current agent. For example, the adjacency weights of neighboring agents with a high degree of correlation to the current feedback execution function can be preset to 0.4, the adjacency weights of the next highest correlation neighboring agents can be preset to 0.35, and the adjacency weights of the remaining neighboring agents can be preset to 0.25. The sum of the adjacency weights corresponding to the same feedback execution agent is set to 1, which facilitates control over the overall magnitude of neighbor collaboration. For example, for head servo feedback agents, the adjacency weight of eye servo feedback agents, which are strongly correlated with their action and emotional expression, can be set to 0.4; the adjacency weight of display screen expression feedback agents can be set to 0.35; and the adjacency weight of ambient light feedback agents can be set to 0.25. For ambient light feedback agents, the corresponding adjacency weights can be reset based on their correlation with display screen expressions and speaker emotional expressions. After configuring the weights, each neighbor deviation value is multiplied by its corresponding agent communication topology adjacency weight. The weighted neighbor deviations are then summed to obtain the neighbor collaborative correction amount for the current feedback agent. When the most recent synchronized states of most neighboring feedback agents are changing in the same emotional direction, the weighted neighbor deviations will form a significant unidirectional correction effect, propelling the current feedback agent to quickly approach the overall feedback state. Conversely, when the state changes of different neighboring agents differ, the positive and negative effects of each deviation will cancel each other out during the weighted summation, reducing the neighbor collaborative correction amount and preventing excessive interference from a single local feedback state. Furthermore, to avoid excessively large single neighbor collaborative corrections due to abnormal states or communication delays of a particular neighboring agent, the maximum single correction magnitude can be preset to 0.15 of the normalized emotional state range. When the weighted summation exceeds this range, it is limited to within 0.15, ensuring gradual convergence of states among multiple feedback agents rather than sudden jumps. The obtained neighbor collaboration correction mainly reflects the degree of deviation of the current feedback agent from the state of its local neighboring agent group.

[0037] In one embodiment, such as Figure 5 As shown, step S103 specifically includes the following steps: S401: Calculate the threshold amplification factor based on the dialogue silence duration and remaining battery value of the intelligent agent based on the feedback, and adjust the preset benchmark threshold based on the threshold amplification factor to obtain the dynamic event triggering threshold. S402: Generate an event trigger judgment result based on the target emotional state value, the local emotional state value of the feedback execution agent, and the dynamic event trigger threshold.

[0038] In this embodiment, the feedback execution agent continuously records the end time of the most recent valid user voice message and uses the time difference between the current time and that end time as the dialogue silence duration. Simultaneously, it reads the remaining battery level output by its own power management module to reflect the current level of interaction activity and available energy status. To avoid different time scales and battery level scales directly affecting threshold adjustment, the dialogue silence duration and remaining battery level are normalized separately. For example, 30 seconds is used as the upper limit for normalization of a silence duration; when the silence duration reaches or exceeds 30 seconds, the silence level is treated as the maximum value. The remaining battery level is mapped to a uniform range according to the battery state of charge from 0% to 100%. Subsequently, the actual silence state can be further confirmed by combining the recent voice energy envelope sequence. For example, using the most recent 5 seconds as the detection window, the variance of the voice energy envelope within this window is calculated, and a preset activity threshold is set to 0.015. When the normalized energy envelope variance is lower than 0.015, it is considered that the current user's voice change is weak, and the silence threshold amplification logic can be entered. When the variance is not lower than the threshold, it is considered that the robot is still in a relatively active dialogue state, reducing the amplification effect of silence duration on the trigger threshold. At the same time, the threshold sensitivity is adjusted according to the remaining battery power. For example, when the remaining battery power is higher than 60%, the normal trigger sensitivity is maintained; when the remaining battery power is between 30% and 60%, the threshold is moderately increased; and when the remaining battery power is lower than 30%, the threshold is further increased. This ensures that only significant changes in emotional state trigger new state synchronization and feedback action updates when the battery power is low, thereby reducing unnecessary calculations and actuator drives. Based on the normalized silence duration and remaining battery power, a first threshold amplification coefficient can be formed to reflect the combined influence of interaction state and energy state on trigger sensitivity.

[0039] In this embodiment, the change in state between the current target emotional state value and the local emotional state value at the previous synchronization time is calculated, and the error change rate is obtained based on the absolute value of the change in state between two consecutive calculation times. When the error change rate is negative, it is determined that the state deviation is in a decreasing trend, and the trigger threshold is appropriately amplified; when the error change rate is positive, it is determined that the state deviation is in an increasing trend, and the threshold amplification is weakened. The threshold adjustment direction is determined based on the sign of the error change rate, and the adjustment amplitude is determined based on the absolute value of the error change rate, forming a second threshold amplification coefficient. Then, the first threshold amplification coefficient and the second threshold amplification coefficient are weighted and fused. For example, the fusion weight of the first threshold amplification coefficient can be preset to 0.6, and the fusion weight of the second threshold amplification coefficient can be preset to 0.4, so that the silence duration and remaining battery power play the main adjustment role, while the error convergence trend plays an auxiliary adjustment role, thereby obtaining a comprehensive threshold amplification coefficient. The absolute value of the state change is normalized to fall within the range of 0 to 1. The dynamic event trigger threshold for the current feedback agent is calculated according to the formula: "Dynamic event trigger threshold = Comprehensive threshold amplification factor × Absolute value of state change + Preset absolute threshold benchmark". In this formula, the state change is the normalized state change, the comprehensive threshold amplification factor reflects the combined influence of the current level of silence, battery status, and error trend, and the preset absolute threshold benchmark is used to prevent the dynamic threshold from approaching zero infinitely; this benchmark value can be preset to 0.08. When the calculated dynamic event trigger threshold is greater than 1, it is limited to 1, maintaining the same 0-1 range as the difference level. When the user has not spoken for a long time, the feedback agent has low remaining battery, and the state has stabilized, the comprehensive threshold amplification factor increases accordingly, reducing the system's trigger frequency. Conversely, when the user is continuously interacting, the battery is sufficient, or the emotional state changes rapidly, the dynamic event trigger threshold decreases relatively, allowing the robot to respond more promptly to new emotional changes. Simultaneously, it can record the last event trigger time of the feedback execution agent and preset the minimum trigger interval to 200 ms. When the time interval between the current moment and the last trigger time is less than 200 ms, even if the state difference temporarily meets the trigger requirement, a new synchronization operation will not be executed, thereby avoiding repeated wake-ups of the main control chip and frequent refreshes of the servo motor, display screen, and lighting effects caused by continuous triggering in a short period of time. The degree of emotional deviation between the target emotional state value and the currently saved local emotional state value of the feedback execution agent is determined, and the deviation is normalized so that it is within a comparable numerical range to the obtained dynamic event trigger threshold. Then, the normalized degree of difference is compared with the dynamic event trigger threshold.When the degree of difference reaches or exceeds the dynamic event trigger threshold, and the minimum trigger interval constraint is also met, an event trigger judgment result that meets the trigger conditions is generated, causing the feedback execution agent to enter the process of state synchronization, local emotional state update, and recalculation of companionship feedback actions. Conversely, when the degree of difference is below the dynamic event trigger threshold, or although it reaches the threshold but has not yet met the minimum trigger interval, a judgment result that does not meet the trigger conditions is generated, and the current companionship feedback execution action continues. This allows the dynamic event triggering mechanism to be simultaneously regulated by the level of dialogue silence, remaining battery power, the magnitude and trend of emotional state changes, enabling the robot to maintain high responsiveness during active interaction phases, while actively reducing the frequency of state synchronization and execution updates during silent, low-battery, and stable phases. This ensures consistent emotional feedback while reducing overall power consumption.

[0040] In this embodiment, a threshold amplification coefficient is calculated based on the dialogue silence duration and remaining battery value of the feedback execution agent, and a preset benchmark threshold is adjusted based on the threshold amplification coefficient to obtain a dynamic event triggering threshold. This includes: extracting the energy envelope sequence of user voice signals received by the feedback execution agent within a recent time window; calculating the variance of the energy envelope sequence as a dialogue activity index; determining that the current state is a dialogue silence state when the variance is less than a preset activity threshold; calculating a first threshold amplification coefficient based on the normalized values ​​of the dialogue silence duration and remaining battery value; calculating the difference between the target emotional state value of the feedback execution agent at the current moment and the local emotional state value at the previous synchronization moment to obtain the state change amount; and calculating the rate of change of the absolute value of the state change amount with respect to time to obtain the error change rate; and based on the error change... The sign of the rate determines the direction of threshold adjustment, and the adjustment range is determined based on the absolute value of the error change rate, forming a second threshold amplification coefficient. The first threshold amplification coefficient and the second threshold amplification coefficient are weighted and fused to obtain a comprehensive threshold amplification coefficient. The dynamic event trigger threshold of the feedback execution agent at the current moment is calculated according to the formula "Dynamic event trigger threshold = comprehensive threshold amplification coefficient × absolute value of state change + preset absolute threshold benchmark". The moment of the last trigger event of the feedback execution agent is recorded, and the time interval between the current moment and the last trigger event moment is calculated. When the time interval is less than the minimum trigger interval threshold determined by the change rate of the comprehensive threshold amplification coefficient and the error change rate, the event trigger judgment result is forcibly maintained as not meeting the trigger condition, avoiding power consumption oscillation caused by frequent triggering.

[0041] In one embodiment, step S402 specifically includes the following steps: The emotion deviation value is calculated based on the target emotion state value and the local emotion state value of the feedback execution agent, and the emotion deviation value is normalized to obtain the degree of difference. The degree of difference is compared with the dynamic event triggering threshold to obtain the event triggering judgment result.

[0042] In this embodiment, the target emotional state value obtained by the current feedback execution agent after consistency and collaborative correction is read, and the local emotional state value maintained by the feedback execution agent after the most recent effective state synchronization is also read. These two states represent "the emotional feedback state that should be achieved" and "the emotional feedback state that the execution layer is currently maintaining," respectively, and are compared on the same scale. For example, the aforementioned three types of emotions—positive, neutral, and negative—can be pre-mapped to a unified emotional state scale. For instance, the baseline state corresponding to positive emotions can be set to 1, neutral emotions to 0, and negative emotions to -1. The specific position of the current emotional state on this scale is determined by combining the corresponding corrected category confidence, thus preserving both the emotional direction and the intensity of the emotional judgment. The target emotional state value is compared with the local emotional state value to obtain an emotional deviation value that characterizes the degree of deviation between the two. The direction of the deviation indicates whether the current feedback execution agent should adjust towards a positive or negative direction, while the absolute magnitude of the deviation indicates the degree of difference between the current execution state and the target state. For example, if the target emotional state has shifted from neutral to a more positive state, but the display screen's facial expression feedback agent still maintains the local emotional state corresponding to a neutral expression, the resulting emotional deviation is large, indicating that the current display feedback cannot adequately match the latest user emotion. If there is only a slight change in confidence between the target emotional state and the local emotional state, the corresponding emotional deviation is small, and there is no need to immediately re-drive the actuator. To avoid inconsistent deviation scales caused by different emotional categories and different confidence ranges, the emotional deviation value is normalized. The normalization result can be limited to the range of 0 to 1, where 0 indicates that the target emotional state and the local emotional state are basically consistent, and 1 indicates that the two have reached the preset maximum effective difference. Emotional deviations that reach or exceed the preset maximum effective deviation can be uniformly mapped to 1, while those less than the maximum effective deviation are mapped to the range of 0 to 1 according to their relative deviation. This yields a degree of difference that can be directly used for event triggering determination. For example, the difference between two complete emotional levels can be used as the maximum effective deviation, so that the change from a negative state to a positive state is close to the normalization upper limit, while smaller changes such as from neutral to slightly positive fall in the middle range, thus avoiding the original emotional state numerical scale directly affecting the trigger judgment.

[0043] In this embodiment, the degree of difference is compared with the dynamic event trigger threshold corresponding to the current feedback execution agent. The dynamic event trigger threshold comprehensively reflects the dialogue silence duration, remaining battery power, state change amplitude, and error change trend. Therefore, this comparison process is actually used to determine whether "the current emotional state change is significant enough to warrant waking up the feedback execution agent and re-outputting the action." When the degree of difference is greater than or equal to the dynamic event trigger threshold, an event trigger judgment result that meets the triggering conditions is generated. The feedback execution agent then enters the next stage, using the target emotional state value as the new feedback control basis. Based on the execution power level corresponding to the remaining battery power, the agent readjusts the accompanying feedback actions such as the display expression, head or eye servo posture, ambient light, and speaker. Simultaneously, after state synchronization is completed, the target emotional state value is updated to the local emotional state value at the most recent synchronization moment. When the degree of difference is less than the dynamic event trigger threshold, an event trigger judgment result that does not meet the triggering conditions is generated. The accompanying feedback action is not recalculated, but the current output state is maintained. To reduce repeated triggering caused by minor fluctuations near the threshold, a preset stabilization time of 50 ms can be set in actual control. This means that the triggering condition is only confirmed if the difference remains above the dynamic event triggering threshold for 50 consecutive ms; if it falls back below the threshold within this time, the triggering condition is still not met. Simultaneously, the aforementioned minimum trigger interval constraint remains as an additional restriction on triggering. Even if the difference has reached the dynamic event triggering threshold, as long as the set minimum trigger interval has not been reached since the previous triggering time, the system will temporarily remain in a non-triggered state.

[0044] In one embodiment, such as Figure 6 As shown, step S104 specifically includes the following steps: S501: When the event trigger judgment result is that the trigger condition is met, the corresponding execution power level is matched according to the remaining power value of the intelligent agent based on the feedback; S502: Drive the companionship feedback execution layer according to the target emotional state value and execution power level, output the companionship feedback execution action, and use the target emotional state value as the local emotional state value at the most recent synchronization moment; S503: When the event trigger judgment result is that the trigger condition is not met, maintain the current output of the companion feedback execution action of the companion feedback execution layer.

[0045] In this embodiment, when the event trigger judgment result is that the trigger condition is met, the feedback execution agent reads the current remaining power value from the power management module and matches the remaining power value with the preset power level range to determine the current execution power level. The remaining power value can be preset as high power level when it is greater than 60%, as medium power level when it is between 30% and 60%, and as low power level when it is less than 30%. These settings are used to control execution resources such as display refresh intensity, servo motor movement amplitude, lighting brightness, and speaker output gain. When the remaining battery power is high, the feedback execution agent prioritizes the integrity and subtlety of the companionship feedback. For example, it allows the head and eye servos to perform continuous movements according to the complete target posture, the display screen to switch according to the complete expression sequence, and the ambient light to smoothly adjust according to the target brightness and color temperature. When the remaining battery power is in the middle range, it appropriately compresses the movement amplitude, reduces the expression switching frame rate, and limits the upper limit of the lighting effect brightness. For example, the maximum movement amplitude of the servos can be limited to 80% of the high-power level, and the upper limit of the lighting effect brightness can be limited to 80%. When the remaining battery power is below 30%, it further enters a low-power execution state, where the maximum movement amplitude of the servos can be limited to 60%, the expression refresh rate can be reduced to about 50% of the normal state, and the upper limit of the ambient light brightness can be limited to 60%, thereby reducing the current consumption of the actuator while still retaining the core companionship feedback capability. The above specific tiered values ​​represent an implementable parameter configuration. In this embodiment, the target emotional state value, which has already undergone consistency and collaborative correction, and the execution power level are jointly sent to the companion feedback execution layer. The target emotional state value determines the emotional direction and basic action content of the feedback action, while the execution power level limits the amplitude, frequency, and finesse of the corresponding action. For example, a positive emotional state can correspond to a smiling expression, a slight head tilt, eye following, and a brighter ambient light effect. At a low power level, the above feedback direction remains unchanged, only the servo motion amplitude, expression refresh frequency, and light output intensity are reduced to avoid changing the direction of emotional expression due to energy saving. After the companion feedback execution layer completes the corresponding action output, it writes the current target emotional state value into the local state storage area of ​​the feedback execution agent and updates it to the local emotional state value at the most recent synchronization moment. At the same time, the corresponding synchronization moment is recorded, so that this state can be used as a comparison benchmark when the feedback execution agent makes the next event trigger judgment, and can also be used as the most recent synchronization state to participate in the neighbor deviation calculation when performing consistency and collaboration with neighboring feedback execution agents.When the event trigger judgment result indicates that the trigger condition is not met, it means that the difference between the current target emotional state and the local emotional state maintained by the feedback execution agent has not yet reached the point where re-driving is necessary, or it is still within the minimum trigger interval constraint. In this case, the calculation of action parameters such as facial expressions, servos, ambient lights, and speakers is not re-executed, nor are execution control commands reissued. Instead, the companion feedback execution actions currently output by the companion feedback execution layer are maintained. For example, the display screen continues to maintain the current facial expression frame, the head and eye servos maintain the current target angle, the ambient lights maintain the current brightness and color temperature, and the speaker-related gain parameters are not refreshed. At the same time, the local emotional state value at the most recent synchronization moment remains unchanged, thereby avoiding frequent wake-ups of the main control chip and execution drive circuit due to subtle emotional fluctuations or short-term state changes.

[0046] In one embodiment, step S502 specifically includes the following steps: The target feedback action parameter set is obtained by matching the basic feedback action parameter set with the target emotional state value and adjusting the amplitude of the basic feedback action parameter set based on the execution power level. Input the target feedback action parameter group into the companion feedback execution layer, output the companion feedback execution action, and use the target emotional state value as the local emotional state value at the most recent synchronization moment.

[0047] In this embodiment, the target emotional state value obtained after consistency and collaborative correction is input into a pre-established emotional state-feedback action mapping table. The basic feedback action parameter group is determined according to the emotional category corresponding to the target emotional state value and the corrected emotional confidence. The basic feedback action parameter group can be composed of the display screen expression number, head servo target angle, eye servo target angle, ambient light brightness parameter, ambient light color temperature parameter, and speaker volume gain parameter, so that the same target emotional state can synchronously constrain the action performance of different feedback actuators of the robot. For example, positive emotions can be associated with smiling emoticons, with the head servo tilted upwards by 8° relative to a neutral posture, the eye servo tilted 4° towards the user's sound source, the ambient light's base brightness set to 70%, and the color temperature set to 4000 K, while maintaining the speaker volume gain at a normal output level. Neutral emotions can be associated with natural emoticons, with the head and eye servos kept in a neutral posture, and the ambient light's base brightness set to 50% and the color temperature set to 3500 K. For negative emotions, more convergent emoticons and gentler postures can be matched, such as tilting the head servo downwards by 5° relative to a neutral posture and setting the ambient light's base brightness to 40%, thus ensuring that the displayed emoticons, mechanical postures, lighting effects, and sound feedback are coordinated in the emotional direction. Simultaneously, the emotional confidence score in the target emotional state value can be used to select basic action parameters of different intensities within the same emotional category. For example, when the confidence score for a positive category reaches 0.80 or higher, a more pronounced smiling expression and larger posture adjustments are selected, while when the confidence score is between 0.60 and 0.80, smaller positive feedback is selected to avoid overly exaggerated companionship actions when only slight changes occur in the textual or acoustic emotion judgment. This mapping relationship is consistent with the conversion relationship between dialogue emotional state, semantic intent, dialogue turn, and energy envelope to parameters such as the display screen, head and eye servos, ambient lighting, and speakers. In this embodiment, the amplitude of parameters directly related to power consumption is adjusted according to the execution power level already matched by the current feedback execution agent, without changing the action direction and basic expressive meaning determined by the target emotional state. For example, the amplitude coefficient corresponding to the high power level can be set to 100%, so that the head servo, eye servo, light brightness, and facial expression changes fully adopt the basic feedback motion parameters; the amplitude corresponding to the medium power level can be set to 80%, that is, while keeping the facial expression type and motion direction unchanged, the basic deflection amplitude of the head servo of 8° is adjusted to about 6.4°, and the basic light brightness of 70% is adjusted to about 56%; the amplitude corresponding to the low power level can be set to 60%, which further reduces the same head motion to about 4.8° and the light brightness to about 42%, while reducing the frequency of dynamic facial expression switching and the frequency of continuous servo movement.For discrete parameters like display expression numbers, instead of directly scaling the numbers themselves, expression sequences with different frame counts and dynamic complexities within the same emotion category are selected based on the execution power level. For example, a high-power level uses a 12-frame complete transition animation, a medium-power level uses an 8-frame simplified animation, and a low-power level uses a 4-frame key expression transition. This allows the power level to primarily change the execution intensity and refresh granularity without altering the direction of emotional expression, such as positive, neutral, or negative. After the above amplitude adjustments, the adjusted display expression parameters, servo attitude parameters, ambient light parameters, and volume gain parameters are recombined to form a target feedback action parameter group.

[0048] In this embodiment, the target feedback action parameter set is transmitted to the companion feedback execution layer, where corresponding execution drive modules complete the actual action output. The display drive module refreshes the display screen based on the target facial expression parameters, the servo drive module generates corresponding control signals based on the target head and eye angles and drives the execution mechanism to rotate, the lighting effect drive module adjusts the ambient lights according to the target brightness and color temperature, and the audio drive module controls the speaker output according to the target volume gain. This forms a multimodal companion feedback execution action that matches the current user's emotional state. To avoid significant asynchrony caused by differences in hardware response times between different execution mechanisms, a unified execution time marker can be added to the same target feedback action parameter set, and the allowed start-up time difference between multiple execution units is preset to 50 ms, allowing the display, servo, and lighting effects to begin updating within the same short time window. After the target feedback action parameter set is successfully sent, the current target emotional state value is written into the state cache of the feedback execution agent and used as the new local emotional state value at the most recent synchronization moment. Simultaneously, the corresponding synchronization moment is recorded, so that this state can provide the most recent valid synchronization state to neighboring feedback execution agents when calculating neighbor deviation next time, and can also serve as a comparison benchmark for the local emotional state when judging the next event trigger.

[0049] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0050] In one embodiment, a desktop companion robot interactive control chip is provided, which is used to perform the steps of the desktop companion robot interactive control method described above.

[0051] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the chip can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0052] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0053] In the embodiments provided in this application, it should be understood that the disclosed chips / devices and methods can be implemented in other ways. For example, the chip / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between chips or units may be electrical, mechanical, or other forms.

[0054] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0055] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for interactive control of a desktop companion robot, characterized in that, A multi-agent system for desktop companion robots, integrating one dialogue cognitive agent and multiple feedback execution agents, wherein the dialogue cognitive agent has only a dialogue semantic perception layer and each of the feedback execution agents has only a companion feedback execution layer, includes the following steps: The user voice signal collected by the dialogue cognitive agent is input into the dialogue semantic perception layer for voice recognition and emotion classification to obtain the dialogue emotion state value. Based on the dialogue emotion state value, the local emotion state value of the feedback execution agent is consistent and collaboratively corrected to obtain the target emotion state value. The dynamic event triggering threshold is calculated based on the dialogue silence duration and remaining battery value of the intelligent agent according to the feedback, and an event triggering judgment result is generated based on the target emotional state value and the dynamic event triggering threshold. When the event trigger judgment result is that the trigger condition is met, the target emotional state value and the remaining battery value are input into the companion feedback execution layer for hierarchical driving, and the companion feedback execution action is output, and the target emotional state value is used as the local emotional state value at the most recent synchronization time; when the event trigger judgment result is that the trigger condition is not met, the companion feedback execution action currently output by the companion feedback execution layer is maintained.

2. The desktop companion robot interactive control method as described in claim 1, characterized in that, The step of inputting the user's voice signal collected by the dialogue cognitive agent into the dialogue semantic perception layer for voice recognition and emotion classification to obtain the dialogue emotion state value includes: The dialogue cognitive agent collects user voice signals and performs voice endpoint detection and voice recognition on the user voice signals to obtain dialogue text. Calculate the fundamental frequency, speech rate, and energy parameters of the user's speech signal, and use the fundamental frequency, speech rate, and energy parameters as speech prosody parameters; Based on the prosodic parameters, the emotional tendency of the dialogue text is analyzed to obtain the emotional state value of the dialogue.

3. The desktop companion robot interactive control method as described in claim 2, characterized in that, The step of performing sentiment analysis on the dialogue text based on the prosodic parameters to obtain the dialogue emotional state value includes: The dialogue text is classified into sentiment tendencies to obtain candidate sentiment categories and category confidence scores. The speech prosody parameters are weighted and summed to calculate the acoustic sentiment intensity value. The confidence level of the candidate emotion category is adjusted based on the acoustic emotion intensity value to determine the dialogue emotion state value.

4. The desktop companion robot interactive control method as described in claim 1, characterized in that, The step of performing consistent collaborative correction on the local emotional state value of the feedback execution agent based on the dialogue emotional state value to obtain the target emotional state value includes: Based on the dialogue emotional state value and the local emotional state value of the feedback execution agent, the cognitive tracking correction amount of the feedback execution agent is calculated. The cognitive tracking correction amount is used to enable the feedback execution agent to track the global emotional state of the dialogue cognitive agent. The neighbor collaboration correction amount is calculated based on the local emotional state value of the feedback execution agent and the local emotional state value of each neighboring feedback execution agent at the most recent synchronization time. The neighbor collaboration correction amount is used to keep the emotional state of multiple feedback execution agents consistent. The cognitive tracking correction amount and the neighbor collaborative correction amount are superimposed, and the local emotional state value of the feedback execution agent is subjected to consistent collaborative correction to obtain the target emotional state value.

5. The desktop companion robot interactive control method as described in claim 4, characterized in that, The step of calculating the neighbor collaboration correction based on the local emotional state value of the feedback execution agent and the local emotional state values ​​of each neighboring feedback execution agent at the most recent synchronization time includes: Based on the local emotional state value of the feedback execution agent and the local emotional state value of each neighboring feedback execution agent at the most recent synchronization time, multiple neighbor deviation values ​​are calculated. Based on the preset adjacency weights of the intelligent agent communication topology, the multiple neighbor deviation values ​​are weighted and summed to obtain the neighbor collaborative correction amount.

6. The desktop companion robot interactive control method as described in claim 1, characterized in that, The step of calculating a dynamic event trigger threshold based on the dialogue silence duration and remaining battery value of the intelligent agent according to the feedback, and generating an event trigger judgment result based on the target emotional state value and the dynamic event trigger threshold, includes: Based on the feedback, the dialogue silence duration and remaining battery value of the intelligent agent are used to calculate the threshold amplification factor, and the preset benchmark threshold is adjusted based on the threshold amplification factor to obtain the dynamic event triggering threshold. An event triggering judgment result is generated based on the target emotional state value, the local emotional state value of the feedback execution agent, and the dynamic event triggering threshold.

7. The desktop companion robot interactive control method as described in claim 6, characterized in that, The step of generating an event triggering judgment result based on the target emotional state value, the local emotional state value of the feedback execution agent, and the dynamic event triggering threshold includes: The emotional deviation value is calculated based on the target emotional state value and the local emotional state value of the feedback execution agent, and the emotional deviation value is normalized to obtain the degree of difference. The degree of difference is compared with the dynamic event triggering threshold to obtain the event triggering judgment result.

8. The desktop companion robot interactive control method as described in claim 1, characterized in that, When the event trigger judgment result is that the trigger condition is met, the target emotional state value and the remaining battery value are input into the companion feedback execution layer for hierarchical driving, the companion feedback execution action is output, and the target emotional state value is used as the local emotional state value at the most recent synchronization time. When the event triggering judgment result is that the triggering condition is not met, the companion feedback execution action currently output by the companion feedback execution layer is maintained, including: When the event triggering judgment result is that the triggering condition is met, the corresponding execution power level is matched according to the remaining power value of the intelligent agent based on the feedback; The companionship feedback execution layer is driven according to the target emotional state value and the execution power level, and the companionship feedback execution action is output. The target emotional state value is used as the local emotional state value at the most recent synchronization time. When the event triggering judgment result is that the triggering condition is not met, the companion feedback execution action currently output by the companion feedback execution layer is maintained.

9. The desktop companion robot interactive control method as described in claim 8, characterized in that, The step of driving the companionship feedback execution layer according to the target emotional state value and the execution power level, outputting companionship feedback execution actions, and using the target emotional state value as the local emotional state value at the most recent synchronization moment includes: The target feedback action parameter set is obtained by matching the target emotional state value with a basic feedback action parameter set and adjusting the amplitude of the basic feedback action parameter set based on the execution power level. The target feedback action parameter group is input into the companion feedback execution layer, the companion feedback execution action is output, and the target emotional state value is used as the local emotional state value at the most recent synchronization moment.

10. A desktop companion robot interactive control chip, characterized in that, The steps are for performing the desktop companion robot interactive control method as described in any one of claims 1 to 9.

Images