An interactive robot control method and control system

Abstract

The application discloses an interactive robot control method and a control system, comprising: dynamically regulating a semantic understanding process based on a dialogue context dynamic model; performing weight self-adaptive optimization adjustment on a multi-modal input signal; constructing a feedback correction mechanism for cross-modal information fusion; and generating a continuous and consistent response to maintain dialogue coherence. The application has the beneficial effects that the complex changes of the dialogue context are dynamically regulated to solve the problems of semantic misunderstanding and discontinuity, and the weight distribution of the multi-modal input is optimized and adjusted to solve the problem of inaccurate information fusion.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, specifically to an interactive robot control method and control system. Background Technology

[0002] With the rapid development of artificial intelligence technology, interactive robots are increasingly being used in smart homes, education, and healthcare. However, existing technologies have many shortcomings. On the one hand, in terms of semantic understanding, most robots struggle to cope with complex changes in dialogue context, often misunderstanding due to ignoring early information, and failing to accurately capture user intent, especially during topic transitions where semantic gaps easily occur. On the other hand, for multimodal input, there is a lack of effective adaptive weight optimization, poor coordination of sensory information, difficulty in accurately interpreting ambiguous expressions, and unstable cross-modal information fusion, leading to data asynchrony and large deviations, resulting in poor overall performance and limited user experience. These shortcomings restrict the further development and popularization of interactive robots. Summary of the Invention

[0003] The purpose of this invention is to provide an interactive robot control method to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] In a first aspect, embodiments of the present invention provide an interactive robot control method, comprising:

[0006] Real-time adjustment of the semantic understanding process based on a dynamic model of dialogue context;

[0007] Adaptive weight optimization adjustment of multimodal input signals;

[0008] Construct a feedback correction mechanism for cross-modal information fusion;

[0009] Generate consistent responses to maintain dialogue coherence.

[0010] Preferably, the real-time adjustment of the semantic understanding process based on the dynamic model of the dialogue context further includes:

[0011] Based on historical dialogue data and current input, calculate the importance weight of each dialogue round to identify key dialogue nodes;

[0012] Semantic features are extracted using natural language processing techniques, and model parameters are optimized by combining the results of dialogue intent prediction.

[0013] The context window size is dynamically adjusted based on the current dialogue state in order to more accurately capture the trend of dialogue changes.

[0014] For semantic misunderstandings in the dialogue, corrections are made based on pre-defined correction rules.

[0015] Preferably, the real-time adjustment of the semantic understanding process based on the dynamic model of the dialogue context further includes:

[0016] Based on the current dialogue content and the user's emotional state, a sentiment analysis model is constructed to enhance the accuracy of understanding;

[0017] If the sentiment score of the current dialogue node is E and the importance weight is W, the enhanced semantic parsing function is activated when E>θ1 and W<θ2; where E represents the sentiment score, θ1 represents the lower threshold, W represents the node weight, and θ2 represents the upper weight.

[0018] By combining long-term historical dialogue records with deep learning training, the predictive ability of the model can be gradually improved.

[0019] Use a dialogue interruption detection mechanism to identify breakpoints in a timely manner and prevent misunderstandings from continuing into subsequent processes.

[0020] Preferably, the step of basing the current dialogue content and the user's emotional state further includes:

[0021] Based on historical data from multi-turn dialogues, analyze users' expression habits and tone preferences;

[0022] The Bayesian algorithm is used to assess the user's sentiment tendency, and the output confidence interval is used as the basis for adjustment.

[0023] If the semantic inconsistency of the current dialogue node is D and the confidence interval width is CI, a recalibration mechanism is triggered when D>μ and CI>ν; where D represents the inconsistency score, μ represents the inconsistency threshold, CI represents the interval width, and ν represents the width threshold.

[0024] Implement emotional compensation strategies to provide additional informational support for possible semantic misunderstandings or emotional deviations.

[0025] Preferably, the historical data based on multi-turn dialogue further includes:

[0026] A dialogue fingerprint map is generated based on the actual content of each round of dialogue, forming a multimodal comprehensive reference framework;

[0027] Using deep learning algorithms to identify key patterns in the graph provides support for understanding complex contexts;

[0028] If the graph consistency score for each dialogue is C and the average dialogue length is L, it is recommended to use the extended context window length when C≤γ and L≥λ; where C represents the graph consistency score, γ represents the graph consistency threshold, L represents the average dialogue length, and λ represents the length threshold.

[0029] By comparing and matching the feature information in the graph with historical cases, we can ensure the effective identification of new dialogue structures.

[0030] Preferably, the generation of a dialogue fingerprint map based on the actual content of each round of dialogue further includes:

[0031] High-quality dialogue fingerprints are generated based on a combination of natural language understanding and image feature extraction techniques.

[0032] Expert rules are introduced to handle high-risk error points specially, ensuring accurate understanding;

[0033] If the integrity score of the generated dialogue fingerprint is P and the confidence level is T, a manual verification prompt signal is triggered when P < τ and T < ρ; where P represents the fingerprint integrity score, τ represents the integrity baseline, T represents the confidence score, and ρ represents the confidence threshold.

[0034] By simulating human experts in manual verification, the final confirmation rate is improved on the basis of automated processes.

[0035] Preferably, the real-time adjustment of the semantic understanding process based on the dynamic model of the dialogue context further includes:

[0036] Constructing multi-layer neural networks to simulate the brain's processing of dialogue information;

[0037] Apply machine learning to automatically adjust model hyperparameters to improve the ability to adapt to new scenarios;

[0038] If the model performance index M changes significantly between [a, b] after each adjustment, the adaptive feedback mechanism is activated; where M represents the model performance score, a represents the lower bound of performance, and b represents the upper bound of performance.

[0039] It combines a speech synthesis engine to generate natural and fluent speech responses.

[0040] In a second aspect, embodiments of the present invention provide an interactive robot control system for implementing the method described in any of the above embodiments, comprising:

[0041] A control unit that controls the semantic understanding process in real time based on a dynamic dialogue context model.

[0042] An optimization unit is used to adaptively optimize and adjust the weights of the multimodal input signal.

[0043] A correction unit, wherein the correction unit is used to construct a feedback correction mechanism for cross-modal information fusion;

[0044] A generation unit is used to generate a continuous and consistent response to maintain dialogue coherence.

[0045] Thirdly, embodiments of the present invention provide an electronic device including at least one processor, the processor being communicatively connected to at least one memory, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform an interactive robot control method.

[0046] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that are used to cause a processor to execute the above-described interactive robot control method.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] 1. By dynamically adjusting the complex changes in the dialogue context, the problems of semantic misunderstanding and discontinuity can be solved.

[0049] 2. Optimize and adjust the weight allocation of multimodal inputs to solve the problem of inaccurate information fusion. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0051] Figure 2 This invention enables real-time control of the semantic understanding process based on a dynamic dialogue context model;

[0052] Figure 3 This invention enables real-time control of the semantic understanding process based on a dynamic dialogue context model;

[0053] Figure 4 This is a further flowchart based on the current dialogue content and the user's emotional state in an embodiment of the present invention;

[0054] Figure 5 These are the specific steps involved in optimizing based on multi-turn dialogue history data in an embodiment of the present invention;

[0055] Figure 6 This is a process diagram of generating a dialogue fingerprint map based on the actual content of each round of dialogue in an embodiment of the present invention;

[0056] Figure 7 This is a flowchart illustrating the real-time control of semantic understanding based on a dynamic dialogue context model, as described in an embodiment of the present invention.

[0057] Figure 8 This is a system block diagram of an embodiment of the present invention;

[0058] Figure 9 This is a block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Please see Figure 1 This invention provides a technical solution: an interactive robot control method, which mainly includes the following steps: real-time regulation of the semantic understanding process based on a dynamic model of dialogue context; adaptive optimization and adjustment of the weights of multimodal input signals; construction of a feedback correction mechanism for cross-modal information fusion; and generation of continuous and consistent responses to maintain dialogue coherence.

[0061] Real-time adjustment of the semantic understanding process based on a dynamic dialogue context model is achieved by establishing a complex dialogue management system. This process not only focuses on the meaning of individual user statements but also emphasizes understanding the intent and emotion within the entire conversational context. For example, in one embodiment, when a user asks about the location of a specific restaurant, the robot not only parses the geographic location query but also integrates information from previous conversations, such as a preference for quiet environments, and recommends a highly-rated restaurant away from the city center. This solves the problem of maintaining accuracy in dynamic and complex environments, avoids misunderstandings caused by ignoring early dialogue information, and ensures consistency between the initial reception and the final output.

[0062] Specifically, adaptive weighting of multimodal input signals means that the robot can intelligently analyze information from multiple senses, such as text, speech, or vision, and automatically assign importance scores to each data source. This effectively solves the problem of potential inconsistencies or biases between information. For example, when faced with a task that includes both gestures and verbal descriptions, the system can quickly determine which is more authoritative and accurate, thus making a more appropriate decision. For instance, if a child points to a menu and says "I want that," but it's unclear which item they are referring to, facial expressions and other body language can help correctly interpret the child's intentions, improving the ability to handle ambiguous expressions.

[0063] The feedback correction mechanism for cross-modal information fusion aims to establish stable connections between data from different modes, enabling collaborative work among components to achieve higher overall performance. During operation, it continuously monitors the synchronization of various input channels and immediately corrects any discrepancies, forming a closed-loop control. Specifically, after receiving a user command, the robot immediately captures the surrounding environment using its camera and microphone. It then compares this real-time situational information with previously recorded similar scenarios. If a significant discrepancy is detected between a new situation and the old pattern, the self-learning function is activated to reassess the current situation and better respond to the next action. For example, suppose an elderly user is directing a home assistance robot to move a calendar on a table. Due to hand tremors, the robot will promptly correct the path based on a pre-stored gesture library to prevent misoperation.

[0064] Finally, to ensure greater temporal consistency in the dialogue, it is essential to design features that consistently generate logical and realistic responses. This functionality emphasizes maintaining a clear and unambiguous chain of answers throughout the entire conversation, avoiding contradictions or gaps even in the event of sudden shifts. In one embodiment, assuming the conversation transitions from the weather to a sports event, the technology enables the device to recognize the transition and seamlessly connect the two topics, providing a smooth and natural transition while continuing to offer precise feedback based on user preferences. For example, upon learning that the user supports their local basketball team, the technology appropriately shares recent game results and related information to ensure the quality of the interaction.

[0065] In summary, this innovative human-computer interaction model effectively addresses a long-standing challenge for researchers in this field by designing the aforementioned core elements: how to dynamically adjust to complex changes in dialogue context, thus mitigating the possibility of semantic misunderstandings and gaps in understanding. Optimizing information fusion through weight allocation strategies improves the efficiency of integrating various types of perceptual data, ensuring effective human-computer communication and significantly enhancing the user experience. Therefore, this new technological solution undoubtedly brings significant improvements to AI assistants and is of great value in promoting the rapid development of intelligent applications in fields such as smart homes, education, and healthcare.

[0066] In an embodiment of the present invention, please refer to Figure 2 Based on a dynamic dialogue context model, the semantic understanding process is dynamically adjusted in real time, including:

[0067] First, this process calculates the importance weight of each dialogue round based on historical dialogue data and current input to identify key dialogue nodes. Specifically, this step analyzes each back-and-forth dialogue between the user and the bot, and uses a pre-trained probabilistic model to assign corresponding scores to different dialogue content, ultimately forming a weighted score for each round. For example, suppose the formula is defined as (W_i=alpha times F_{Hist}+(1-alpha)times F_{Input}), where (W_i) represents the importance weight of the i-th round of dialogue, (F_{Hist}) represents the feature factor derived from historical dialogues, and (F_{Input}) is the relevance index calculated based on the newly received input in this round; the coefficient α (range: 0<α≤1) is used as a balancing factor to optimize the combination of these two parts. Setting the optimal value α=0.7 maximizes the balance between historical influence and immediate feedback, improving the accuracy of the judgment.

[0068] The second step involves extracting semantic features using natural language processing (NLP) techniques and combining them with the obtained dialogue intent prediction results to update and optimize the model's internal parameters. This allows the robot to respond to dialogues more closely to the user's true expressions and intentions. For example, in one embodiment, if a user is identified as expressing dissatisfaction after repeatedly asking questions about travel preparations, this information is labeled, used for reinforcement learning, and fed back into the training samples to readjust the weight allocation ratio, resulting in better performance for similar questions in subsequent processing.

[0069] Furthermore, the window size used to store context fragments is adjusted appropriately based on the current conversation status to ensure more accurate tracking of conversation progression without losing important information or excessively piling up irrelevant background content. For example, if the topic is detected to shift from leisure and entertainment to business discussions, the previously collected family and hobby-related chat logs will be reduced to focus on professional experience sharing, etc.

[0070] For any communication misunderstandings caused by ambiguity, a pre-programmed correction logic is employed to rectify the situation. This involves identifying the error based on a built-in dictionary and relevant contextual information, quickly switching to the correct parsing mode, and providing an appropriate response to the user. For example, if a user asks an open-ended question with a broad timeframe, such as "Approximately what time can we meet tomorrow afternoon?", an external notification function is used to refine the timeframe before providing a clear answer, minimizing the risk of ambiguity.

[0071] The above four links are closely linked and are designed to continuously improve and maintain a highly sensitive response mechanism to ensure good human-machine communication.

[0072] In an embodiment of the present invention, please refer to Figure 3 The semantic understanding process is dynamically controlled in real time based on the dialogue context model, which includes the following four steps:

[0073] In the first step, during interaction with the user, the robot dynamically adjusts its responses based on the specific content of the current conversation and the user's immediate emotional state. The constructed sentiment analysis model can capture the user's positive or negative emotions and improve the accuracy of understanding the user's intentions accordingly. In this way, it ensures that the robot's responses are more in line with the user's actual situation.

[0074] For example, in one embodiment, if a user expresses dissatisfaction with company policies, sentiment analysis can quickly capture and convey this to subsequent processing steps, helping the robot choose a more soothing response.

[0075] In the formula, E represents the sentiment score; this parameter is typically normalized from 0 to 1, with its optimal range depending on the application scenario. θ1 represents the lower threshold for sentiment activation, a crucial critical point set to trigger specific behaviors. W reflects the importance of dialogue nodes, with integers from 0 to 5 representing different levels of importance. θ2 is a weighting constraint to avoid unnecessary computational burden or overemphasis on less critical information. When both E > θ1 and W < θ2 are true, enhanced semantic parsing is enabled to improve dialogue accuracy and response speed. This configuration is designed to enhance the system's ability to deeply understand dialogue when necessary without consuming excessive resources processing non-core dialogue segments.

[0076] The third step emphasizes learning from long-term historical dialogue records to improve predictive ability. This process utilizes a large dataset of accumulated back-and-forth communication for deep learning training, enabling the algorithm to better predict future interaction paths and outcomes.

[0077] In a real-world scenario, if the robot has encountered numerous customer inquiries about the return process before, it can now provide accurate answers more quickly when similar scenarios arise again.

[0078] The final step outlines the use of a conversation interruption detection mechanism to prevent misinterpretations from further impacting future communication. This design ensures that any brief communication disruptions caused by technical issues, external interference, or other unforeseen circumstances are not treated as normal input, preventing erroneous commands from being sent.

[0079] Specifically, assuming a video conference suddenly loses connection and then reconnects, the robot can recognize this special interruption and readjust its understanding and response to restore the normal conversation.

[0080] In an embodiment of the present invention, please refer to Figure 4 Based on the current conversation content and the user's emotional state, this further includes:

[0081] First, in interactive robot control methods, historical data from multiple dialogues is incorporated to optimize dialogue quality and service effectiveness, aiming for a deeper understanding of the user. This step involves collecting and processing all past records of user interactions, encompassing not only the dialogue text itself but also nonverbal communication methods such as the emotional tone and intensity of voice. Through this cumulative observation, the machine can learn the user's expression habits, tone preferences, and frequently used vocabulary. For example, if a user consistently uses specific internet slang to express positive emotional responses, the system can, based on this data, selectively adopt or respond to such wording in future interactions to increase intimacy and identification. In one embodiment, this process may include building a keyword database containing a large number of language samples labeled with emotional polarity and tone features, which is continuously updated and iterated to gradually improve recognition accuracy.

[0082] Next, the system uses a Bayesian algorithm to assess the user's sentiment tendency and provides a corresponding confidence interval for subsequent adjustment decisions. Here, each user's sentiment is considered a parameter to be estimated. Bayes' theorem uses the prior probability P(A), the likelihood ratio P(B|A), and the normalization factor to calculate the posterior probability, i.e., P(A|B) = P(B|A) * P(A) / ΣP(B|i). It assumes a series of predefined scenarios or states (i). When a new input arrives, the probability distribution of the most likely scenario is calculated according to this rule. Then, a measure of uncertainty—the so-called confidence interval—is added to each prediction, using upper and lower boundaries to represent the magnitude of the result's variation. For example, when faced with an answer that seems to be on the verge of frustration but is vaguely expressed, "I think it's okay" might be judged to have a relatively wide confidence interval; meaning this judgment is not certain enough. Typically, a reasonable μ value is set around [0.5, 0.75], indicating that slight to moderate inconsistency is sufficient to trigger the mechanism. The width threshold ν is generally between (0,1), and it tends to be optimally set to a smaller value, such as 0.2-0.4, to ensure timely response to abnormal signals without causing interference from frequent alarms. This setting aims to detect potential misunderstandings early on, while preventing unnecessary resource waste due to oversensitivity.

[0083] If the semantic deviation at the current node exceeds a predetermined limit μ, and the confidence interval is above the tolerable level ν, a recalibration process will be initiated. This measure aims to reduce the accumulation of misunderstandings caused by poor communication and maintain smooth and directional communication. Specifically, it checks whether the dialogue accurately reflects the other party's true meaning. D is part of a quantitative scoring system used to characterize the difference between the literal meaning and the inferred intent. The larger the value, the greater the difference between the two parts. The criteria for establishing the relationship between D and μ can vary depending on the application scenario. For example, requirements may be more lenient in customer service, while stricter controls may be needed for scenarios such as medical consultation.

[0084] The final step involves introducing emotional compensation strategies. Whenever potential misinterpretations or signs of widening gaps in perception are identified, additional information should be proactively provided to clarify the situation and enhance empathy in the interaction. This approach helps build a better user experience, correcting potential cognitive biases while demonstrating a human-centered communication style. In one embodiment, if the chatbot detects obvious disappointment in the other party's tone and fails to respond appropriately, it should try sending a short message, such as, "I noticed you sounded a bit unhappy just now, is it because...?" This demonstrates that the other party's feelings are respected and valued while guiding the conversation in a more constructive direction. Such measures not only alleviate negative atmospheres but also help solidify long-term trust relationships.

[0085] In an embodiment of the present invention, please refer to Figure 5 Optimization based on multi-turn dialogue history data involves several specific steps:

[0086] First, for each round of dialogue, a dialogue fingerprint map is automatically generated to form a multimodal comprehensive reference framework. A dialogue fingerprint map refers to a unique identifier containing information such as speech and text, constructed by extracting the structured and unstructured elements of the dialogue using algorithms. This map allows each dialogue segment to be accurately captured, laying the foundation for subsequent in-depth analysis. For example, in a human-computer dialogue system, when a user inquires about weather information and continues to ask about other related details, this map can effectively reflect the main characteristics of this dialogue.

[0087] The second step involves using deep learning algorithms to identify key patterns in the dialogue fingerprint. Machine learning models, fine-tuned with extensive training data, are employed here to extract meaningful signals from complex linguistic environments, thereby enhancing contextual understanding. In one embodiment, a natural language processing module applies convolutional neural networks (CNNs) or long short-term memory (LSTMs) to the generated map, searching for recurring segments that reveal the user's intent and emotional state. This step helps the robot respond more quickly and accurately to various types of input.

[0088] For each dialogue segment, if the graph consistency score is C and the average dialogue length is L, where C represents the graph consistency score from 1 to 100 (a larger C means greater consistency between graphs), and the score does not exceed a certain threshold γ (the consistency score threshold value), and the average dialogue length is greater than or equal to another threshold λ, it is recommended to extend the context window's duration to capture more information for decision support. Specifically, if a particular interaction is found to involve multiple jumps in discussion topics, leading to an unstable overall structure and accompanied by a long communication record, then the number of previous dialogue segments should be increased to better understand the current dialogue context. Generally, the optimal range for γ is around 70, while λ depends on the specific application scenario; for example, customer service chat might involve two lines of text, while psychological counseling might involve 100 characters. This design is because an excessively short history window might overlook important prior context, while excessive magnification can introduce noise and affect parsing accuracy.

[0089] The final step compares and matches the feature information obtained from the dialogue graph with previously saved classic historical cases. This ensures accurate interpretation even when encountering new or less common dialogue formats. Specifically, if the robot encounters a previously unseen technical troubleshooting process, this process allows it to find similar problems and their solutions, helping to quickly and appropriately respond. This method ensures good adaptability and efficiency regardless of changes in dialogue format. In summary, the entire process is interconnected, forming a closed-loop system from collecting dialogue data to delivering accurate responses.

[0090] In an embodiment of the present invention, please refer to Figure 6 The process of generating a dialogue fingerprint map based on the actual content of each round of dialogue consists of the following steps:

[0091] First, in the initial step, a combination of natural language understanding and image feature extraction techniques will be employed to generate high-quality dialogue fingerprints. This method combines textual semantics and contextual information with visual cues (such as images and emojis sent by the user), enabling the generated dialogue fingerprints to more accurately reflect the actual interaction content. This not only improves the understanding of natural language but also better captures non-verbal information from multimodal data. In one embodiment, when the robot receives a screenshot of a user's face expressing emotion during a dialogue, it creates a dialogue fingerprint by fusing the facial expression recognition results of this image with the accompanying verbal description, allowing the robot to more accurately determine the other party's emotional state and intentions.

[0092] Next, to ensure high accuracy in dialogue processing, expert rules are introduced to specifically handle potential high-risk error points—this is the second step. This mechanism aims to supplement scenarios where automated processes struggle or are prone to errors, such as difficulties understanding dialects or sensitive topics. For example, when encountering regional accents or uncommon terminology, the default algorithm may produce significant errors; in such cases, based on predefined expert guidelines, the system will make targeted adjustments to the corresponding content to ensure accurate understanding of the user's true meaning.

[0093] If the generated dialogue fingerprint fails to meet expectations, a manual verification signal will be triggered—this is the third step. Specifically, let P represent the integrity score, which measures whether the generated dialogue fingerprint comprehensively and completely records the key features of the current dialogue. Its value is a real number between 0 and 1, and it aims to be as close to 1 as possible. τ serves as the numerical standard for the integrity assessment baseline. Additionally, T represents the credibility score, reflecting the reliability of the extracted fingerprint, also within the (0,1) range, striving for a larger number closer to 1. ρ is the threshold parameter that determines the triggering condition. The verification signal is sent only when both P < τ and T < ρ are met because this effectively filters out issues with genuinely high potential risks for further confirmation. For example, in scenarios involving monetary transactions, even slight deviations from normal indicators require careful handling. Once the above formula logic determines an abnormal situation, the system will immediately notify relevant personnel for review.

[0094] Finally, the fourth step simulates the role of a human expert for manual review. This leverages the advantages of the automated process while ensuring accuracy in the final stage. This dual-safety system helps maintain an efficient and stable workflow while significantly reducing the error rate. Specifically, even after automation, some ambiguities remain that require human review and approval. As mentioned earlier in financial business scenarios, any subtle changes, such as alterations to payment instructions, must be carefully reviewed by a human before execution to avoid unnecessary financial losses. In this way, automation enhances the probability of making correct decisions in complex situations.

[0095] In embodiments of the present invention, please refer to the appendix. Figure 7 The real-time regulation of the semantic understanding process based on a dynamic dialogue context model further includes:

[0096] First, a multi-layered neural network is constructed to simulate the brain's processing of conversational information. Specifically, this multi-layered neural network learns features such as vocabulary usage patterns, sentence structure, and semantic connections in human communication. This allows it to efficiently process and understand complex dialogue content, laying the foundation for natural and fluent human-computer interaction. This network's ability to mimic the complex connections between different neurons in the brain helps enhance the understanding of ambiguous or unclear utterances. For example, during communication with users, if dialects or trendy slang are used, the network can identify the underlying meanings and respond appropriately.

[0097] Building upon the aforementioned foundation, applying machine learning to automatically adjust model hyperparameters to enhance their adaptability to new scenarios is a crucial step. Automatic adjustment aims to enable the model to better adapt to various environments and changing needs without relying on pre-set, fixed parameter configurations. The optimal hyperparameters may differ significantly for each specific application environment or newly introduced functional module; therefore, introducing a dynamic adjustment mechanism ensures stable and optimal performance regardless of changing external conditions. For example, in an educational application scenario, adjusting hyperparameter values ​​related to sensitivity and response style is necessary when switching teaching topics or targeting different age groups of students.

[0098] If the model performance metric M changes significantly after each adjustment, falling within the range of [a, b], an adaptive feedback mechanism is activated. In this case, 'a' represents the minimum acceptable performance threshold of the model, and 'b' is the maximum tolerable variation limit. These two limits are set because excessive fluctuations indicate potential overfitting or insufficient training of certain key elements. To maintain system robustness and continuous improvement, an adaptive process is triggered for targeted optimization when such anomalies are detected. In this example, 'a' is set to 85 points and 'b' to 93 points; if the evaluation score exceeds this range, immediate measures are taken to adjust the model to restore stability.

[0099] The final step in the process is to combine a speech synthesis engine to generate natural and fluent voice responses. This technology uses text-to-speech (TTS) to convert the previously calculated response information into an audio format and output it to the user's auditory senses, making it sound like a real person speaking. High-quality synthesis not only enhances the user experience but also helps to create a more friendly and intuitive product image. A concrete example is in the scenario of a family companionship interactive assistant, when a child says: "Mom didn't spend time with me today," the robot can gently reply: "I know you missed Mom today."

[0100] The above four steps enable more precise and effective control of the interactive robot, which not only promotes a better human-computer communication experience but also increases the system's intelligence and flexibility.

[0101] The construction of a multi-layer neural network includes the following steps:

[0102] The first step utilizes a bidirectional long short-term memory (BiLSTM) network to enhance the understanding of complex sentence meanings. Specifically, this step uses two parallel LSTM networks: one scans the sentence from beginning to end, while the other operates in reverse order, thus comprehensively grasping the context.

[0103] The second step involves integrating a reinforcement learning module to adapt to new forms of intent expression. Here, a reinforcement learning mechanism is employed, enabling the system to optimize its decision-making process through reward signals, thereby automatically adjusting internal parameters to capture constantly changing user needs.

[0104] The third step is to expand the number and types of training samples when the intent recognition accuracy is (I) and the satisfaction score (S) is greater than the threshold (delta). In the formula, (I in[0,1]) is used to evaluate the model's prediction effect on user commands; (Sin[0,1]) represents the user's satisfaction score; and (delta) is usually taken as a high value close to 1 to determine the acceptable threshold.

[0105] The fourth step is to design an end-to-end dialogue architecture to ensure that all components work collaboratively. Specifically, this means that all tasks, such as natural language processing and context maintenance, operate smoothly within an integrated framework and cooperate to form an efficient loop. This helps improve efficiency, reduce latency, and enhance overall service quality.

[0106] In the embodiments of this invention, a bidirectional long short-term memory (BiLSTM) network technique is employed for enhancement. Specifically, measures are taken at each step to ensure that the model can effectively understand and respond to user needs.

[0107] A key part of this method is collecting a large amount of diverse feature information during each round of dialogue to aid the application during the training phase. These features include not only explicit inputs provided by the user, but also implicit indicators such as tone of voice and dialogue length.

[0108] Next, the number of hidden units is adjusted based on personalized considerations. This step aims to achieve optimal generalization capabilities while maintaining the best matching degree, tailored to different application scenarios and individual differences.

[0109] Contextual relevance is used as an important parameter to judge the tightness of content. When the calculated result exceeds a predefined relevance threshold β, it is considered to be of decisive significance for the development of the dialogue. R is an indicator that quantifies the influence of the current information point in the entire semantic sequence; it generally ranges from 0 to 1, with values ​​close to 1 indicating high relevance and values ​​close to 0 indicating almost no connection.

[0110] The coordination between the forward and backward propagation mechanisms has been further improved to achieve optimal overall architectural performance. This goal primarily aims to ensure efficient learning and reduce error propagation. On the one hand, the algorithm needs to be sensitive enough to capture new knowledge and patterns; on the other hand, overfitting must be suppressed to maintain good stability. This framework achieves a delicate balance between these two aspects through careful tuning of the weight update strategy and the selection of activation functions. For example, error rate fluctuations can be periodically reviewed, and the gradient descent speed can be fine-tuned accordingly.

[0111] In embodiments of the present invention, the first step is to collect rich feature data for use in the training phase. This process further includes four specific steps, each with unique objectives and technical means.

[0112] First, language materials from different user groups are collected to build a comprehensive and detailed corpus. This means integrating a large amount of real-world language dialogues, texts, and commands from a wide range of user sources. This not only enriches the training materials but also helps the interactive robot understand and respond to user needs more naturally and accurately. For example, in one embodiment, the system might collect voice commands and text chat logs covering users of different ages and regions, and compile them into a multi-dimensional corpus. This large corpus, as the foundational data for training, can significantly enhance the model's learning performance and generalization ability.

[0113] Secondly, to support accurate parsing for certain industry applications, a set of specialized terminology parsing dictionaries specific to these fields has been developed to assist in corpus analysis. This includes creating specialized vocabularies for industries such as finance, law, and medicine to ensure that the machine can recognize the unique expressions and professional terminology of these fields. In one example, if the interactive robot is applied in a healthcare environment, a large number of medical terms, such as disease names, drug types, and treatment methods, need to be added to the parsing dictionary.

[0114] When the amount of newly added data reaches a certain scale, special cleaning and organization measures are required. The specific condition is determined by the formula N*Q>η. Here, N represents the amount of newly added data (usually a positive integer), Q represents the proportion of newly added word categories to all word categories (a decimal between 0 and 1), and η is a threshold (set according to actual needs), with an optimal value of approximately 0.5 based on experience. This formula means that when the newly introduced data contains enough new and unique content (N is large and Q is close to 1), targeted cleaning and organization work is triggered to update and improve existing training data in a timely manner, ensuring the system continues to operate efficiently.

[0115] Specialized high-performance preprocessing tools have been developed and applied to improve overall computing speed and efficiency. These tools can perform basic operations such as removing noisy data and standardizing text and symbol formats, while leveraging distributed computing resources to accelerate the batch processing of big data. Specifically, the preprocessing stage optimizes the data storage structure to speed up index lookup time and reduce additional time overhead caused by data redundancy.

[0116] In summary, by making improvements to multiple details and establishing clear technical solutions and parameter standards, the above methods enable the robot control system to maintain efficient operation while continuously acquiring new knowledge, thereby providing a more intelligent service experience.

[0117] The method of the embodiments of the present invention includes the following key steps, which are intended to effectively solve problems such as dynamic control of complex changes in the dialogue process and weight optimization and adjustment of multimodal inputs, and ensure the accuracy and consistency of semantic understanding and information fusion in diverse application scenarios.

[0118] First, to handle the complex changes in dialogue context and avoid potential semantic misunderstandings or gaps in understanding, this invention introduces a dynamic model real-time adjustment mechanism based on dialogue context. Specifically, during each user interaction, the system performs semantic analysis and sentiment recognition on the current and historical dialogue content. Based on the analysis results, the system dynamically adjusts its dialogue semantic understanding strategy. This strategy is not statically set but continuously tracks dialogue trends, topic shifts, and background knowledge accumulation throughout the dialogue, and adjusts its semantic judgment parameters in real time accordingly. For example, when encountering ambiguous or easily misinterpreted utterances, the algorithm can accurately predict and correct misreadings by combining the specific scenarios of previous rounds of conversation, ensuring that the machine can accurately capture the user's true intentions. Furthermore, it constructs a long-term memory module to store similar scenario cases that have been successfully processed in the past for future reference, further strengthening the system's ability to maintain a stable and accurate response in sudden and complex situations, thereby achieving efficient and accurate resolution of uncertainties in dialogue context.

[0119] Secondly, this invention employs adaptive optimization measures for multimodal input signals to ensure accurate information transmission. Multimodal input refers to the process of multiple sensory data sources, such as text, speech, and images, interacting together. These different types of perceptual information have varying importance for dialogue management and fluctuate with changing scenarios. This method introduces machine learning algorithms to automatically calculate the weight distribution of each input modality in the entire interaction chain: on the one hand, it flexibly adjusts the proportion of each modality based on the actual input type (such as plain text input in a silent environment or information containing a large number of emojis); on the other hand, considering specific task requirements, such as customer service where there is a greater focus on customer emotional expression, it correspondingly enhances the weight of audio feature extraction to help better evaluate and simulate behavioral responses that conform to natural human communication habits. Meanwhile, in order to achieve organic unity across modalities rather than a simple superposition effect, this paper emphasizes the construction of a cross-modal feedback correction mechanism as a connecting point. After each round of dialogue iteration, a post-validation and evaluation feedback is immediately carried out on all output behaviors in that round. If any single-modal information is detected to deviate abnormally from the expected output standard, an error correction plan is quickly activated for fine-tuning and optimization. This ensures that each response maintains a high degree of logical rigor and consistent style, which not only fully meets the personalized user experience requirements under different conditions, but also greatly reduces the risk of system disorder caused by errors in individual sensory data.

[0120] In summary, the method provided by the embodiments of the present invention not only innovatively designs an intelligent decision-making architecture based on a context model, fundamentally overcoming the shortcomings of previous technologies in adapting to the dynamic and ever-changing requirements of today's increasingly rich artificial intelligence communication environment, but also pioneers a new era of interactive AI control ideas and technical means with higher intelligence, stronger robustness, and wider applicability; this brings more possibilities to the future field of human-computer dialogue and is of great significance in promoting technological progress in the industry.

[0121] Please see Figure 8 Embodiments of the present invention also provide an interactive robot control system for implementing the methods described in any of the above embodiments, the system comprising:

[0122] Control unit 100, which controls the semantic understanding process in real time based on a dialogue context dynamic model;

[0123] Optimization unit 200, the optimization unit is used to perform adaptive weight optimization adjustment on the multimodal input signal;

[0124] Correction unit 300, the correction unit is used to construct a feedback correction mechanism for cross-modal information fusion;

[0125] Generation unit 400 is used to generate a continuous and consistent response to maintain dialogue coherence.

[0126] Please see Figure 9 , Figure 9 A schematic diagram of the mechanism of an electronic device 20 that can implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of control devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0127] Electronic device 20 includes at least one processor 21 and a memory, such as read-only memory (ROM) 22 and random access memory (RAM) 23, communicatively connected to at least one processor 21. The memory stores computer programs executable by at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 22 or loaded from storage unit 28 into the RAM 13. The RAM 23 may also store various programs and data required for the operation of electronic device 20. The processor 21, ROM 22, and RAM 23 are interconnected via bus 24. Input / output (I / O) interface 25 is also connected to bus 24.

[0128] Multiple components in electronic device 20 are connected to I / O interface 25, including: input unit 26, such as keyboard, mouse, etc.; output unit 27, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 29, such as network card, modem, wireless transceiver, etc. Communication unit 29 allows electronic device 20 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0129] Processor 21 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 21 performs the various methods and processes described above.

[0130] In some embodiments, the methods described above can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 28. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 20 via ROM 22 and / or communication unit 29. When the computer program is loaded into RAM 23 and executed by processor 21, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, processor 21 can be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).

[0131] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0133] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and

[0135] Input from the user can be received in any form (including voice input, speech input, or tactile input).

[0136] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0137] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0138] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An interactive robot control method, characterized by, include: Acquire the multimodal input signal of the interactive robot in the current round; Based on historical dialogue data and current input, the importance weight of the i-th round of dialogue is calculated according to the formula W_i=α×F_Hist+(1-α)×F_Input to determine key dialogue nodes; where F_Hist represents the feature factor inferred from historical dialogue, F_Input represents the relevance index calculated based on current input, and α is a balancing factor. Semantic features are extracted using natural language processing techniques, and model parameters are optimized by combining the results of dialogue intent prediction. The context window size is dynamically adjusted based on the current dialogue state, and the weights of the multimodal input signals are adaptively optimized and adjusted. Based on the current dialogue content and the user's emotional state, an emotional score E and a node weight W are obtained. When E > θ1 and W < θ2, the enhanced semantic parsing function is activated; where θ1 represents the lower limit of the threshold and θ2 represents the upper limit of the weight. The Bayesian algorithm is used to assess the user's sentiment tendency and output a confidence interval. When the semantic inconsistency D of the current dialogue node is greater than μ and the confidence interval width CI is greater than ν, a recalibration mechanism is triggered. Here, μ represents the inconsistency threshold and ν represents the width threshold. A dialogue fingerprint map is generated based on the actual content of each round of dialogue. When the integrity score P < τ and the credibility T < ρ of the generated dialogue fingerprint, a manual verification prompt signal is triggered. Here, τ represents the integrity baseline and ρ represents the credibility threshold. Based on the semantic understanding results that have been recalibrated or verified, a consistent response is generated to maintain the coherence of the dialogue.

2. The interactive robot control method of claim 1, wherein, The importance weight of the i-th round of dialogue is calculated based on historical dialogue data and current input according to the formula W_i=α×F_Hist+(1-α)×F_Input, including: The back-and-forth dialogue information between the user and the robot in each round is analyzed, and a pre-trained probability model is used to assign scores to different dialogue contents to form a weighted score for the corresponding round; key dialogue nodes are determined based on the weighted score. The scope of context calls in subsequent semantic understanding processes is adjusted based on the key dialogue nodes. 3.The interactive robot control method of claim 1, wherein, The step of extracting semantic features using natural language processing techniques and optimizing model parameters by combining dialogue intent prediction results includes: Extract semantic features from the current input; The dialogue intent prediction result is obtained based on the semantic features; User feedback information is labeled and fed back into the training samples to adjust model parameters and weight allocation ratios. 4.The interactive robot control method of claim 1, wherein, The method of dynamically adjusting the size of the context window based on the current dialogue state includes: detecting changes in the topic in the current dialogue state; When a topic is detected to have shifted from the first category to the second category, the call ratio of early context fragments that are less relevant to the second category is shrunk, while the call ratio of context fragments that are more relevant to the second category is increased. Capture the changing trends of the conversation based on the adjusted context window.

5. The interactive robot control method of claim 1, wherein, The process of using a Bayesian algorithm to assess a user's sentiment tendency and output confidence intervals includes: The posterior probability of a user's sentiment tendency is calculated based on prior probability, likelihood ratio, and normalization factor. The probability distribution corresponding to the user's emotional tendency is determined based on the posterior probability. The probability distribution outputs a confidence interval to characterize the prediction uncertainty; When the semantic inconsistency degree D exceeds the inconsistency threshold μ and the confidence interval width CI exceeds the width threshold ν, a recalibration mechanism is initiated to reduce the accumulation of semantic misunderstandings.

6. The interactive robot control method of claim 1, wherein, The generation of a dialogue fingerprint map based on the actual content of each round of dialogue includes: Extract the structured and unstructured elements from each round of dialogue to construct a dialogue fingerprint graph that includes voice and text information; Key patterns in the dialogue fingerprint were identified using deep learning algorithms. When the graph consistency score C ≤ γ for each dialogue segment and the average dialogue length L ≥ λ, historical dialogue segments are invoked by extending the context window length; where γ represents the graph consistency threshold and λ represents the length threshold. The feature information in the dialogue fingerprint is compared and matched with historical cases to identify new dialogue structures.

7. The interactive robot control method of claim 6, wherein, The generation of a dialogue fingerprint map based on the actual content of each round of dialogue also includes: Dialogue fingerprints are generated by combining natural language understanding and image feature extraction. Expert rules are introduced to handle high-risk error points; When the integrity score P of the generated dialogue fingerprint is less than τ and the confidence level T is less than ρ, a manual verification prompt signal is triggered. The semantic understanding results corresponding to the dialogue fingerprint are confirmed based on the results of manual verification.

8. An interactive robot control system for implementing the method of any one of claims 1 to 7, characterized by include: The input acquisition unit is used to acquire the multimodal input signal of the interactive robot in the current round; The round weight calculation unit is used to calculate the importance weight of the i-th round of dialogue based on historical dialogue data and current input, according to the formula W_i=α×F_Hist+(1-α)×F_Input, in order to determine key dialogue nodes; The semantic control unit is used to extract semantic features through natural language processing techniques and optimize model parameters by combining the results of dialogue intent prediction. The context adjustment unit is used to dynamically adjust the size of the context window based on the current dialog state. The multimodal optimization unit is used to adaptively optimize and adjust the weights of multimodal input signals. An enhanced parsing trigger unit is used to activate the enhanced semantic parsing function when the sentiment score E>θ1 and the node weight W<θ2; The recalibration unit is used to evaluate the user's sentiment tendency using a Bayesian algorithm and output a confidence interval, and triggers the recalibration mechanism when the semantic inconsistency degree D>μ and the confidence interval width CI>ν. The dialogue fingerprint verification unit is used to generate a dialogue fingerprint map based on the actual content of each round of dialogue, and to trigger a manual verification prompt signal when the integrity score P < τ and the credibility T < ρ of the generated dialogue fingerprint. The response generation unit is used to generate a continuous and consistent response based on the semantic understanding results after recalibration or verification to maintain the coherence of the dialogue.

9. An electronic device, characterized in that, The method includes at least one processor, which is communicatively connected to at least one memory, wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the method according to any one of claims 1 to 7.

Images