Interactive robot control method and control system

Abstract

The invention discloses an interactive robot control method and system. The method comprises the steps that a semantic understanding process is regulated and controlled in real time based on a dialogue context dynamic model; carrying out weight adaptive optimization adjustment on the multi-modal input signal; constructing a feedback correction mechanism of cross-modal information fusion; a continuous consistent response is generated to maintain dialog coherence. The method has the beneficial effects that the complex change of the dialogue context is dynamically regulated and controlled to solve the problems of semantic misunderstanding and fault, and the weight distribution of multi-modal input is optimized and adjusted to solve the problem of inaccurate information fusion.

Description

Technical Field

[0001] The present invention relates to the technical field of robot control, and in particular to an interactive robot control method and control system. Background Art

[0002] With the rapid development of artificial intelligence technology, interactive robots are increasingly used in smart homes, education, medical care and other fields. However, existing technologies have many shortcomings. On the one hand, in terms of semantic understanding, most robots find it difficult to cope with the complex changes in the context of conversations. They often ignore early information and lead to misunderstandings, and are unable to accurately capture user intentions, especially when the topic is switched. Semantic faults are prone to occur. On the other hand, for multimodal input, there is a lack of effective weight adaptive optimization, poor coordination of sensory information, and it is difficult to accurately interpret ambiguous expressions. In addition, cross-modal information fusion is unstable, and data asynchrony and large deviations are prone to occur, resulting in poor overall performance and limited user experience. These shortcomings restrict the further development and popularization of interactive robots. Summary of the invention

[0003] The object of the present invention is to provide an interactive robot control method to solve the problems raised in the above background technology.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] In a first aspect, an embodiment of the present invention provides an interactive robot control method, comprising:

[0006] Real-time control of semantic understanding process based on dynamic model of conversation context;

[0007] Perform weight adaptive optimization and adjustment on multimodal input signals;

[0008] Construct a feedback correction mechanism for cross-modal information fusion;

[0009] Generate consistent responses to maintain conversational coherence.

[0010] Preferably, the real-time regulation of semantic understanding process based on the dialogue context dynamic model further includes:

[0011] Based on historical conversation data and current input, calculate the importance weight of each conversation turn to determine the key conversation nodes;

[0012] Extract semantic features through natural language processing technology, and optimize model parameters based on conversation intent prediction results;

[0013] Dynamically adjust the context window size based on the current conversation state to more accurately capture conversation change trends;

[0014] Semantic misunderstandings in conversations are corrected based on pre-set correction rules.

[0015] Preferably, the real-time regulation of semantic understanding process based on the dialogue context dynamic model further includes:

[0016] Based on the current conversation content and the user's emotional state, a sentiment analysis model is built to enhance the accuracy of understanding;

[0017] If the sentiment score of the current dialogue node is E and the importance weight is W, when E>θ1 and W<θ2, the enhanced semantic parsing function is activated; where E represents the sentiment score, θ1 represents the lower limit of the threshold, W represents the node weight, and θ2 represents the upper limit of the weight;

[0018] Combine long-term historical conversation records for deep learning training to gradually improve the model's predictive ability;

[0019] Use the conversation interruption detection mechanism to promptly identify fault points and prevent conversation misunderstandings from continuing into subsequent processes.

[0020] Preferably, the method based on the current conversation content and the user's emotional state further includes:

[0021] Analyze users’ expression habits and intonation preferences based on historical data from multiple rounds of conversations;

[0022] Use Bayesian algorithm to evaluate users' sentiment tendency and output confidence interval as the basis for adjustment;

[0023] If the semantic inconsistency of the current dialogue node is D and the confidence interval width is CI, when D>μ and CI>ν, the recalibration mechanism is triggered; where D represents the inconsistency score, μ represents the inconsistency threshold, CI represents the interval width, and ν represents the width threshold;

[0024] Implement emotional compensation strategies to provide additional information interpretation support for possible semantic misunderstandings or emotional deviations.

[0025] Preferably, the historical data based on multiple rounds of dialogue further includes:

[0026] Generate a conversation fingerprint based on the actual content of each round of conversation to form a multi-modal comprehensive reference framework;

[0027] Use deep learning algorithms to identify key patterns in graphs and provide support for understanding complex contexts;

[0028] If the graph consistency score of each conversation is C and the average conversation length is L, when C≤γ and L≥λ, it is recommended to use the method of extending the context window length; where C represents the graph consistency score, γ represents the graph consistency threshold, L represents the average conversation length, and λ represents the length threshold;

[0029] Compare and match the feature information in the graph with historical cases to ensure effective recognition of new dialogue structures.

[0030] Preferably, generating a conversation fingerprint map based on the actual content of each round of conversation further includes:

[0031] Generate high-quality conversation fingerprints based on a combination of natural language understanding and image feature extraction technology;

[0032] Introduce expert rules to handle high-risk error points specially to ensure the accuracy of understanding;

[0033] If the integrity score of the generated conversation fingerprint is P and the credibility is T, when P<τ and T<ρ, a manual verification prompt signal is triggered; where P represents the fingerprint integrity score, τ represents the integrity baseline, T represents the credibility score, and ρ represents the credibility threshold;

[0034] Simulate human experts to perform manual verification and improve the final confirmation rate based on the automated process.

[0035] Preferably, the real-time regulation of semantic understanding process based on the dialogue context dynamic model further includes:

[0036] Build a multi-layer neural network to simulate the way the brain processes conversation information;

[0037] Apply machine learning to automatically adjust model hyperparameters to improve the ability to adapt to new scenarios;

[0038] If the model effect index M after each adjustment varies greatly between [a, b], the adaptive feedback mechanism is activated; where M represents the model performance score, a represents the lower bound of performance, and b represents the upper bound of performance;

[0039] Combined with speech synthesis engine to generate natural and fluent voice responses.

[0040] In a second aspect, an embodiment of the present invention provides an interactive robot control system, which is used to implement the method described in any of the above embodiments, including:

[0041] A control unit, wherein the control unit controls the semantic understanding process in real time based on a dynamic model of the dialogue context;

[0042] An optimization unit, the optimization unit is used to perform weight adaptive optimization adjustment on the multimodal input signal;

[0043] A correction unit, wherein the correction unit is used to construct a feedback correction mechanism for cross-modal information fusion;

[0044] A generating unit is used to generate a response with continuous consistency to maintain the continuity of the conversation.

[0045] In a third aspect, an embodiment of the present invention provides an electronic device comprising at least one processor, wherein the processor is communicatively connected to at least one memory, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute an interactive robot control method.

[0046] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the above-mentioned interactive robot control method when executed by a processor.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. Solve the problems of semantic misunderstanding and disconnection by dynamically regulating the complex changes in the conversation context.

[0049] 2. Optimize and adjust the weight distribution of multimodal input to solve the problem of inaccurate information fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is an overall flow chart of the method of the present invention;

[0051] Figure 2 The present invention regulates the semantic understanding process in real time based on the dynamic model of the conversation context;

[0052] Figure 3 The present invention regulates the semantic understanding process in real time based on the dynamic model of the conversation context;

[0053] Figure 4 A further flow chart of an embodiment of the present invention based on the current conversation content and the user's emotional state;

[0054] Figure 5 The specific steps of the embodiment of the present invention when optimizing based on multi-round dialogue history data;

[0055] Figure 6 A process diagram of generating a conversation fingerprint map based on the actual content of each round of conversation according to an embodiment of the present invention;

[0056] Figure 7 This is a flowchart of real-time regulation of semantic understanding based on a dynamic model of a conversation context according to an embodiment of the present invention;

[0057] Figure 8 is a system module diagram of an embodiment of the present invention;

[0058] Fig. 9 2 is a module diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] See also Figure 1 The present invention provides a technical solution: an interactive robot control method which mainly includes the following steps: real-time regulation of the semantic understanding process based on a dynamic model of the dialogue context; adaptive weight optimization and adjustment of multimodal input signals; construction of a feedback correction mechanism for cross-modal information fusion; and generation of a continuous and consistent response to maintain dialogue coherence.

[0061] The semantic understanding process is regulated in real time based on the dynamic model of the conversation context by establishing a complex conversation management system. This process not only focuses on the meaning of the user's single sentence expression, but also pays more attention to understanding the intention and emotion in the entire conversation context. For example, in one embodiment, the user asks about the location of a specific restaurant. The robot not only parses the geographical location query, but also integrates the information mentioned in the previous conversation that the robot likes a quiet environment and recommends a highly rated restaurant away from the downtown area. This solves the problem of maintaining accuracy in a dynamic and complex environment, avoids the problem of misunderstanding caused by ignoring early conversation information, and ensures the consistency from the initial reception to the final output.

[0062] Specifically, adaptive weight optimization and adjustment of multimodal input signals means that the robot can intelligently analyze information from multiple senses such as text, voice or vision, and automatically set the importance score of each data source according to the situation. This effectively solves the problem of inconsistency or deviation between information. For example, when faced with a task that includes both gesture instructions and oral descriptions, the system can quickly determine which one is more authoritative and accurate, so as to make a more appropriate decision. For example, a child points to a menu and says "I want that" but is not sure which item he is referring to. With the help of facial expressions and other body language, the system helps to correctly interpret the child's intentions and improves the ability to deal with ambiguous expressions.

[0063] The feedback correction mechanism for cross-modal information fusion is designed to establish a stable connection between data in different modes so that all parts can work together to achieve a higher level of overall performance. During operation, the synchronization of various input channels will be continuously monitored, and corrections will be made immediately when differences occur, forming a closed-loop control. Specifically, when the robot receives user instructions, it will immediately capture the state of the surrounding environment through its own camera and microphone, and then compare these real-time scene information with similar scenes previously recorded. If it detects that there is an obvious discrepancy between the new situation and the old mode, the self-learning function will be activated to re-evaluate the current situation in order to better respond to the next action. For example, suppose an elderly user is directing a home assistance robot to move a calendar on the desktop. Due to slight movements caused by hand tremors, the robot will promptly correct the path according to the pre-stored gesture library to prevent misoperation.

[0064] Finally, in order to make the conversation more consistent in time, it is necessary to design a function that can continuously generate logical and practical answers. This part of the function emphasizes that a clear and definite answer chain should be maintained throughout the conversation cycle, and there should be no inconsistency or disconnection even in the case of sudden changes. In one embodiment, suppose the conversation turns from the weather to the topic of sports events. This technology enables the device to recognize the moment of transition and smoothly connect the two topics. The transition is smooth and natural, and it continues to provide accurate information feedback based on user preferences. For example, after learning that the user supports the local basketball team, it is appropriate to share the latest game results and related information to ensure the quality of interaction.

[0065] In summary, this innovative human-computer interaction model effectively addresses the problem that has always troubled researchers in this field through the design of the above-mentioned core links, namely, how to dynamically regulate the complex changes in the dialogue context and solve the possibility of semantic misunderstandings and faults. The problem of inaccurate information fusion is solved by optimizing the weight distribution strategy, which improves the efficiency of integrating various types of perception data, ensures the efficiency of human-computer communication, and greatly enhances the user experience. Therefore, this new technical solution will undoubtedly bring great improvements to artificial intelligence assistants and promote the rapid development of intelligent applications in smart home, education, and medical fields, which is of great value.

[0066] In the embodiments of the present invention, please refer to Figure 2 , based on the dynamic model of the conversation context, real-time control of the semantic understanding process, including:

[0067] First, in this process, the importance weight of each dialogue round is calculated based on the historical dialogue data and the current input to determine the key dialogue nodes. Specifically, this link analyzes each reciprocal dialogue message between the user and the robot, and uses the pre-trained probability model to assign corresponding scores to different dialogue contents, and finally forms a weight score in one round. For example, suppose the formula is defined as (W_i=alpha times F_{Hist}+(1-alpha)times F_{Input}), where (W_i) represents the importance weight of the i-th dialogue, (F_{Hist}) represents the characteristic factor calculated based on the historical dialogue; and (F_{Input}) is the correlation index calculated based on the newly received input in this round; the coefficient α (range: 0<α≤1) is used as a balancing factor to optimize the degree of integration of the two parts. When the optimal value α=0.7 is set, the historical influence and immediate feedback can be taken into account to the greatest extent, improving the accuracy of judgment.

[0068] In the second step, semantic features are extracted through natural language processing technology and combined with the obtained dialogue intention prediction results to update and tune the internal parameters of the model, so that the robot can respond to the dialogue more closely to the user's real expression and intention. For example, in one embodiment, if a user is identified to be dissatisfied after continuously asking questions about travel preparation suggestions, this information is marked for reinforcement learning and fed back into the training samples to readjust the weight distribution ratio so that similar questions can be better expressed in subsequent processing.

[0069] Furthermore, the window size used to store context fragments is reasonably changed based on the current conversation status to ensure more accurate tracking of the changing trend of the conversation without losing important information or over-accumulating invalid background content. For example, if it is detected that the topic has shifted from leisure and entertainment to business talks, the chat records of family and hobbies that were collected too extensively in the early stage will be reduced to narrow the focus area to professional experience sharing, etc.

[0070] For any communication deviation caused by language ambiguity, the correction logic that has been prepared in advance is used to perform correction actions. That is, the wrong interpretation point is judged based on the built-in vocabulary and the relevance conditions that can be referenced in the context, and the correct interpretation mode is quickly switched to provide appropriate responses to the interactive partners. If a user uses unclear expressions such as "When can we meet tomorrow afternoon?" This open-ended question has a wide time range, the external reminder function is used to refine and confirm the specific time period before replying with a clear answer to reduce the risk of ambiguity.

[0071] The above four links are closely linked, aiming to continuously improve and maintain a highly sensitive response mechanism to ensure good human-computer communication effects.

[0072] In the embodiments of the present invention, please refer to Figure 3 , based on the dynamic model of the conversation context, the semantic understanding process is regulated in real time, which includes the following four steps:

[0073] In the first step, when interacting with the user, the robot will dynamically adjust its response based on the specific content of the current conversation and the user's immediate emotional state. The constructed sentiment analysis model can capture the user's positive or negative emotions and improve the accuracy of understanding the user's intentions accordingly. In this way, the robot's response is more in line with the user's actual situation.

[0074] For example, in one embodiment, if a user expresses dissatisfaction with a company policy, sentiment analysis can quickly capture and convey it to subsequent processing links, helping the robot choose a more soothing response plan.

[0075] In the formula, E represents the sentiment score; this parameter is generally normalized from 0 to 1, and its optimal range depends on the needs of the application scenario. θ1 represents the lower threshold of sentiment activation, which is an important critical point set to trigger specific behaviors. W reflects the importance of the dialogue node, and integers from 0 to 5 represent different levels of importance, while θ2 is a restriction on the weight to avoid unnecessary computational burden or excessive attention to less critical information. When both conditions E>θ1 and W<θ2 are met, the enhanced semantic parsing function is enabled to improve the accuracy of the dialogue and the speed of response. The reason for this configuration is to increase the system's ability to deeply understand the dialogue when necessary without consuming too many resources to process non-core dialogue fragments.

[0076] The third step emphasizes learning from long-term historical conversation records to improve predictive power. This process uses a large amount of accumulated data sets of back-and-forth communication for deep learning training, so that the algorithm can better predict future interaction paths and results.

[0077] In a real-world example, if the robot has encountered multiple customers asking about the return process before, it can now give accurate answers more quickly when a similar scenario occurs again.

[0078] The last step is to use a conversation interruption detection mechanism to prevent misinterpretations from affecting future communication processes. This design ensures that any brief communication interruptions due to technical problems, external interference, or other unexpected events will not be treated as normal input and cause incorrect instructions to be sent.

[0079] Specifically, assuming that a video conference suddenly loses connection and then reconnects, the robot can recognize this special interruption and readjust its understanding and response methods to restore normal conversation.

[0080] In the embodiments of the present invention, please refer to Figure 4 , based on the current conversation content and the user's emotional state, further includes:

[0081] First, in the interactive robot control method, in order to optimize the quality of dialogue and service effects, the historical data of multiple rounds of dialogue are taken into consideration, aiming to understand the user more deeply. This step involves collecting and processing all past records of interaction with the user, covering not only the dialogue text itself, but also non-verbal communication methods such as the emotional color and intensity of the voice. Through this cumulative observation, the machine can learn the user's expression habits, intonation preferences, and common vocabulary. For example, if a user always uses specific Internet buzzwords to express positive emotional reactions, then based on this data, the system can use or echo such wording in future communications to increase intimacy and identity. In one embodiment, the process can include establishing a keyword database containing a large number of language samples with labeled emotional polarity and tone characteristics, and gradually improve the recognition accuracy through continuous updates and iterations.

[0082] Next, the system uses the Bayesian algorithm to evaluate the user's emotional tendency and gives the corresponding confidence interval for subsequent adjustment decisions. Here, each user's emotion is regarded as a parameter to be estimated. Bayes' theorem uses the prior probability P(A), the likelihood ratio P(B|A) and the normalization factor to calculate the posterior probability, that is, P(A|B)=P(B|A)*P(A) / ΣP(B|i). Here, it is assumed that there are a series of predefined scenarios or states (i). When new input arrives, the probability distribution of the most likely scenario is calculated according to this rule. Then, a measure of uncertainty is added to each prediction - the so-called confidence interval, with upper and lower boundaries indicating the size of the space for the result to change. For example, when faced with an answer that seems to be on the verge of depression but is vague, I think it's okay. This type of answer may be judged to have a relatively wide confidence interval; meaning that the judgment is not certain enough. Usually, a reasonable μ value is set around [0.5,0.75], indicating that slight to moderate inconsistency is sufficient to trigger the mechanism. The width threshold ν is generally between (0,1), and tends to be set to a smaller value such as 0.2-0.4 to set the optimal point, so as to ensure timely response to abnormal signals without interference caused by frequent alarms. The reason for this setting is to catch possible misunderstandings at an early stage and prevent unnecessary waste of resources caused by excessive sensitivity.

[0083] Once the degree of semantic deviation at the current node exceeds the established limit μ, and the amplitude of the confidence interval is also above the tolerable level ν, the recalibration process will be initiated. The purpose of this measure is to reduce the accumulation of misunderstandings caused by poor communication and maintain the smoothness and direction of communication. Specifically, it is to detect whether the content of the conversation accurately reflects the other party's true meaning. D is part of a quantitative scoring system that is used to characterize the difference between literal meaning and inferred intention. The larger the value, the greater the difference between the two parts. The criteria for establishing the relationship between D and μ can vary depending on the application scenario. For example, the requirements may be relaxed in the field of customer service, while more stringent control may be required for scenarios such as medical consultation.

[0084] The final step involves introducing emotional compensation strategies. Whenever potential misunderstandings or signs of widening gaps in feelings are identified, additional information should be proactively provided to clarify and enhance empathy in the interaction. This approach helps build a better user experience, correcting possible cognitive biases while also demonstrating a humane communication attitude. In one embodiment, if the chatbot finds that the other party's tone is obviously frustrated and the chatbot fails to respond appropriately, it should try to send a short message, such as "I noticed that you sounded unhappy just now. Is it because..." In this way, the other party's feelings are respected and valued, while guiding the conversation in a more constructive direction. Such measures can not only alleviate negative atmospheres but also help consolidate long-term trust relationships.

[0085] In the embodiments of the present invention, please refer to Figure 5 , optimization based on multi-round dialogue history data involves several specific steps:

[0086] First, for each round of actual conversation content, a conversation fingerprint map is automatically generated to form a multimodal comprehensive reference framework. The conversation fingerprint map refers to the extraction of structural and non-structural elements of the conversation through an algorithm to construct a unique identifier containing voice, text and other information. This map allows each conversation link to be accurately captured and lays the foundation for subsequent in-depth analysis. For example, in a human-computer dialogue system, when a user asks for weather information and continues to inquire about other relevant details, the map can effectively reflect the main characteristics of this conversation.

[0087] The second step is to use deep learning algorithms to identify key patterns in the conversation fingerprint map. Machine learning models are used here, which are tuned with a large amount of training data so that they can extract meaningful signals from complex language environments to improve contextual understanding. In one embodiment, a natural language processing module performs a convolutional neural network (CNN) or long short-term memory (LSTM) on the generated map to find important fragments that appear repeatedly but reveal the user's intentions and emotional state. This step helps the robot respond to various types of input more quickly and accurately.

[0088] For each conversation, if the consistency score of the graph is C and the average conversation length is L, and C represents the graph consistency score between 1 and 100 (the larger the C, the more consistent the graphs), when the score does not exceed a certain set threshold γ, that is, the consistency score threshold, and the average conversation length is greater than or equal to another threshold λ, it is recommended to extend the context window time span to capture more information points for decision support. Specifically, if it is determined that there are multiple jump-like discussion topic changes in a specific interaction process, resulting in an unstable overall structure, accompanied by a long communication record, then the number of previous conversation segments to focus on is appropriately increased to better understand the current conversation context. Generally speaking, the optimal range of γ may be around 70, and λ depends on the needs of the specific application scenario. For example, customer service chat may be 2 lines of text, while psychological counseling may reach 100 characters. This design is because a too short historical window may lead to the neglect of important previous connections, while excessive amplification is easy to introduce noise and affect the parsing accuracy.

[0089] The last step is to compare and match the feature information obtained from the conversation graph with the classic historical cases that have been saved before, in order to ensure that even if a new or rare type of conversation format is encountered, a good interpretation can be achieved. Specifically, assuming that the robot encounters a technical troubleshooting process consultation that has not been seen before, this process can find similar problems and their solutions in the past, helping to quickly make appropriate responses. This method ensures good adaptability and efficiency regardless of how the conversation format changes. In summary, the entire process is closely linked, forming a closed-loop system from collecting conversation data to completing accurate response output.

[0090] In the embodiments of the present invention, please refer to Figure 6 ,The process of generating a conversation fingerprint based on the actual content of each round of conversation is divided into the following steps:

[0091] First, in the first step, a technology combining natural language understanding and image feature extraction is used to generate high-quality conversation fingerprints. This method combines text semantics and contextual environment, as well as visual clue information (such as pictures and expressions sent by users), so that the generated conversation fingerprints can more accurately reflect the actual interaction content. Doing so can not only improve the understanding of natural language, but also better capture non-verbal information in multimodal data. In one embodiment, when the robot receives a facial screenshot expressing emotions sent by the user during a conversation, it creates a conversation fingerprint by fusing the expression recognition results of the image and the corresponding language description, so that the robot can more accurately judge the other party's emotional state and intentions.

[0092] Next, to ensure the high accuracy of dialogue processing, expert rules are introduced to handle high-risk error points that may occur, which is the second step. This part of the mechanism is designed to supplement scenarios that are difficult to solve or prone to errors in automatic processes, such as difficulty understanding dialects or sensitive topics. For example, when encountering some situations with local accents or uncommon terms, the default algorithm may have large errors; at this time, according to the predefined expert guidelines, the system will make targeted adjustments to the corresponding content to ensure its accurate grasp of the user's true meaning.

[0093] If the generated conversation fingerprint fails to reach a satisfactory level, a manual verification prompt signal will be triggered, which is the third step. Specifically, let P represent the integrity score, which measures whether the generated conversation fingerprint has fully and completely recorded the key features of the conversation. The value range is a real number between 0 and 1, and it is expected to be as close to 1 as possible; τ is used as the numerical standard of the integrity assessment baseline; in addition, T is used to represent the credibility score to reflect that the reliability of the extracted fingerprint is also in the interval (0,1) and pursues a larger number close to 1; and ρ is the threshold parameter that determines the trigger condition. Here, the verification signal is sent only when P<τ and T<ρ are met at the same time because this can effectively screen out those problems that do have higher potential risks for further confirmation. For example, in the scenario involving amount transactions, even if there is only a slight deviation from the normal indicators, it needs to be handled with caution. Once the above formula logic determines that it is an abnormal situation, the system will immediately notify the relevant personnel to review.

[0094] Finally, the fourth step simulates the role of human experts to conduct manual review, which not only retains the benefits of the previous automatic operation mode but also ensures the correctness of the results in the final stage. This dual guarantee system helps maintain an efficient and stable workflow while greatly reducing the rate of misjudgment. Specifically, after the automation is completed, there are still some gray areas that need to be reviewed and determined by real people. Just like in the financial business scenario mentioned earlier, any subtle changes such as changes in payment instructions must be carefully checked by real people before they can be released for execution to avoid unnecessary economic losses. In this way, the probability of making correct decisions in complex situations is improved on the basis of automation.

[0095] In the embodiments of the present invention, please refer to the attached Figure 7 , the real-time regulation of semantic understanding process based on the dynamic model of dialogue context further includes:

[0096] First, a multi-layer neural network is constructed to simulate the way the brain processes conversation information. Specifically, this multi-layer neural network can efficiently process and understand complex conversation content by learning features such as vocabulary usage patterns, sentence structure, and semantic associations in human communication, providing a basis for natural and smooth human-computer communication. The ability of this network to mimic the complex connection relationship between different neurons in the brain helps to enhance the ability to understand vague or ambiguous speech. For example, in the process of communicating with users, if dialects or trendy slang appear, the network can identify the hidden meaning behind them and respond correctly.

[0097] On the basis of the above, it is crucial to apply machine learning to automatically adjust the model hyperparameters to improve the ability to adapt to new scenarios. Automatic adjustment aims to enable the model to better adapt to various environments and changes in demand without relying on pre-set fixed parameter configurations. For each specific application environment or newly introduced functional module, the optimal hyperparameters may be completely different; therefore, the introduction of a dynamic adjustment mechanism can ensure stable and optimal performance regardless of how external conditions change. For example, in an educational application scenario, when switching teaching topics or targeting different age groups of students, it is necessary to adjust the hyperparameter values ​​of related sensitivity and response style.

[0098] If the model effect indicator M changes greatly after each adjustment, and the range is between [a, b], the adaptive feedback mechanism is activated. In this case, a represents the lowest acceptable performance threshold of the model, and b represents the maximum tolerable upper limit of the change. The reason for setting these two limits is that excessive fluctuations may indicate overfitting or that some key elements have not been fully trained. In order to keep the system running stably and continuously improving, when such anomalies are detected, the adaptive process is triggered for targeted optimization. In the example description, it is assumed that a is set to 85 points and b is set to 93 points. Once the evaluation score exceeds this range, measures will be taken immediately to adjust the model to restore stability.

[0099] The whole process is completed as the last step by combining the speech synthesis engine to generate a natural and fluent voice response. This technology converts the previously calculated answer information into a sound format and outputs it to the user's auditory perception organs through text-to-speech conversion (TTS), making it sound like a real person. High-quality synthesis can not only improve the user experience but also help to form a more friendly and intuitive product image. A specific example is in the scenario of a family companion interactive assistant, when the child says: Mom is not with me today. At this time, the robot can gently reply: I know you miss your mother today.

[0100] The above four steps enable more precise and effective control of the interactive robot, which not only promotes a better human-computer communication experience but also increases the intelligence and flexibility of the system.

[0101] Among them, building a multi-layer neural network includes the following steps:

[0102] The first step is to use a bidirectional long short-term memory network (BiLSTM) to enhance the understanding of complex sentence meanings. Specifically, this step uses two parallel LSTM networks, one to scan from the beginning to the end of the sentence, and the other to operate in reverse order, so that the context information can be fully grasped.

[0103] The second step is to integrate the reinforcement learning module to adapt to the new form of intent expression. The reinforcement learning mechanism used here allows the system to optimize the decision-making process through reward signals, thereby automatically adjusting internal parameters to capture the changing user needs.

[0104] The third step is to expand the number and type of training samples when the intent recognition accuracy is (I) and the satisfaction score (S) is greater than the threshold (delta). In the formula, (I in [0, 1]) is used to evaluate the model's prediction effect on user commands; (Sin [0, 1]) represents the user's satisfaction score; and (delta) usually takes a high value close to 1 to determine the qualified limit.

[0105] The fourth step is to design an end-to-end dialogue architecture to ensure that all components work together. Specifically, all tasks such as natural language processing and context maintenance functions operate smoothly within an integrated framework and cooperate with each other to form an efficient loop link. This helps improve efficiency, reduce latency, and improve overall service quality.

[0106] In the embodiment of the present invention, a bidirectional long short-term memory network (BiLSTM) technology is used for enhancement. Specifically, measures are taken at each step to ensure that the model can effectively understand and respond to user needs.

[0107] An important part of this approach is to collect a large amount of diverse feature information during each round of dialogue to assist the application of the training phase. These features include not only the explicit input provided by the user, but also implicit indicators such as tone of voice and length of the dialogue.

[0108] Then the number of hidden units is adjusted based on personal considerations. This step aims to achieve generalization capabilities at the best matching level according to different application scenarios and individual differences.

[0109] Context relevance is used as an important parameter to judge the closeness of content. When the calculated result exceeds the predefined relevance threshold level β, it is considered to be decisive for the development of the dialogue. R is an indicator that quantifies the influence of the current information point in the entire semantic sequence; it generally ranges from 0 to 1, with values ​​close to 1 indicating a high degree of relevance and values ​​close to 0 indicating almost no connection.

[0110] The coordination between the forward and back propagation mechanisms has been further improved to achieve the best overall architecture performance. The main goal is to ensure that the learning process is efficient and reduce error propagation. On the one hand, the algorithm must be sensitive enough to capture new knowledge and patterns, and on the other hand, it must suppress overfitting to maintain good stability. In this framework, the weight update strategy and the choice of activation function are carefully controlled to achieve a delicate balance between the two. For example, the error rate fluctuation trend can be regularly reviewed and the speed of gradient descent can be fine-tuned accordingly.

[0111] In the embodiment of the present invention, the first step is to collect rich feature data for use in the training phase. This process further includes four specific steps, each of which is designed with unique goals and technical means.

[0112] First, language materials from different user groups are collected to build a comprehensive and detailed large corpus. This means integrating a large number of real language conversations, texts, and commands from a wide range of user sources. This not only enriches the training materials, but also helps the interactive robot understand and respond to user needs more naturally and accurately. For example, in one embodiment, the system may collect voice commands and text chat records covering users of different age groups and regions, and compile them into a multi-dimensional corpus. This large corpus, as the basic data for training, can significantly enhance the learning effect and generalization ability of the model.

[0113] Secondly, in order to support accurate analysis of certain industry applications, a set of specialized terminology analysis dictionaries specific to these fields has been developed to assist in corpus analysis. This includes creating professional vocabularies for industries such as finance, law, and medicine to ensure that the machine can recognize the expressions and specialized terminology unique to these fields. In one example, if the interactive robot is used in a medical service environment, a large number of medical terms, such as disease names, types of drugs, and diagnosis and treatment methods, need to be added to the analysis dictionary.

[0114] When the amount of newly added data reaches a certain scale, special cleaning and sorting measures need to be taken. The specific conditions are determined by the formula N*Q>η. Here, the parameter N represents the amount of newly added data (usually a positive integer), Q represents the proportion of the newly added word category to the total word category, and is a decimal ranging from 0 to 1. η is a threshold (set according to actual needs), and its preferred value can be selected based on experience to be about 0.5. The meaning of this formula is that when the newly introduced data contains enough new and unique content (N is large and Q is close to 1), targeted cleaning and organization work is triggered to timely update and improve the existing training data to ensure that the system continues to operate efficiently.

[0115] We have developed and applied specialized high-performance preprocessing tools to improve overall computing speed and efficiency. These tools can perform basic operations such as removing noise data and standardizing text and symbol formats, while using distributed computing resources to speed up batch processing of big data. Specifically, the preprocessing stage will optimize the data storage structure to speed up index query time and reduce additional time overhead caused by data redundancy.

[0116] In summary, by making improvements in multiple details and formulating clear technical solutions and parameter standards, the above method enables the robot control system to maintain efficient operation while continuously acquiring new knowledge, thereby providing a more intelligent service experience.

[0117] The method of the embodiment of the present invention includes the following key steps, which aim to effectively solve problems such as dynamic regulation of complex changes in the dialogue process and weight optimization adjustment of multimodal input, so as to ensure the accuracy and consistency of semantic understanding and information fusion in various application scenarios.

[0118] First, in order to handle the complex changes in the conversation context and avoid possible semantic misunderstandings or faults, the present invention introduces a real-time control mechanism of a dynamic model based on the conversation context. Specifically, during each user interaction, the system will perform semantic analysis and emotion recognition on the current and historical conversation content. The conversation semantic understanding strategy is dynamically adjusted according to the analysis results. This strategy is not a static setting, but continuously tracks factors such as conversation trends, topic conversions, and background knowledge accumulation throughout the conversation, and corrects its semantic judgment parameters in real time accordingly. For example, when encountering ambiguous or ambiguous words, the algorithm can accurately predict and correct misreadings based on the specific scenarios of the previous rounds of conversations, ensuring that the machine can accurately capture the user's true intentions; in addition, it also stores similar scenario cases that have been successfully processed in the past by building a long-term memory module for future reference, further strengthening the system's ability to maintain a stable and accurate response to sudden complex situations, thereby achieving efficient and accurate solutions to uncertainty challenges in the conversation context.

[0119] Secondly, the present invention takes adaptive optimization measures on multimodal input signals to ensure accurate transmission of information. Multimodal input refers to the process in which multiple sources of sensory data such as text, voice, and images work together. The importance of these different types of perceptual information to dialogue management varies and fluctuates with the change of scenes. This method automatically calculates the weight distribution of each input modality in the entire interaction chain by introducing a machine learning algorithm: on the one hand, the proportion of each modality is flexibly adjusted according to the actual input type (such as pure text input in a sound environment or information containing a large number of emoticons); on the other hand, considering specific task requirements such as the customer service field paying more attention to customer emotional expression, the audio feature extraction weight is correspondingly enhanced to assist in better evaluating and simulating behavioral responses that conform to human natural communication habits. At the same time, in order to achieve an organic unity across modalities rather than a simple superposition and combination effect, special emphasis is placed here on building a cross-modal feedback and correction mechanism as a connection point - immediately after each round of dialogue iteration operations is completed, a posteriori verification and evaluation feedback are carried out on all output behaviors in that round. If any single-mode information is detected to be abnormally deviated from the expected output standard, the error correction plan is quickly initiated for fine-tuning and optimization, thereby ensuring that each answer maintains a consistent characteristic of highly logical rigor and coordinated style, which not only fully meets the personalized user experience demands under different conditions, but also greatly reduces the risk factor of system-wide disorder due to individual sensory data errors.

[0120] In summary, the method provided by the embodiment of the present invention not only innovatively designs an intelligent decision-making architecture based on a context model, fundamentally overcoming the shortcomings of previous technologies in adapting to the dynamic and changeable requirements of today's increasingly rich artificial intelligence communication environment, but also creates a new era of interactive AI control ideas and technical means with a higher level of intelligence, strong robustness and a wide range of applications; this brings more possibilities to the future field of human-computer dialogue and is also of great significance in promoting technological progress in the industry.

[0121] See also Figure 8 The embodiment of the present invention further provides an interactive robot control system for implementing the method described in any of the above embodiments, the system comprising:

[0122] A control unit 100, which controls the semantic understanding process in real time based on a dynamic model of the dialogue context;

[0123] An optimization unit 200, the optimization unit is used to perform weight adaptive optimization adjustment on the multimodal input signal;

[0124] A correction unit 300, which is used to construct a feedback correction mechanism for cross-modal information fusion;

[0125] The generating unit 400 is configured to generate a response with continuous consistency to maintain the continuity of the conversation.

[0126] See also Fig. 9 , Fig. 9 A schematic diagram of an electronic device 20 that can implement an embodiment of the present invention is shown, and the electronic device is intended to represent various forms of control devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0127] The electronic device 20 includes at least one processor 21, and a memory connected to the at least one processor 21, such as a read-only memory (ROM) 22, a random access memory (RAM) 23, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 22 or the computer program loaded from the storage unit 28 to the random access memory (RAM) 13. In the RAM 23, various programs and data required for the operation of the electronic device 20 can also be stored. The processor 21, ROM 22 and RAM 23 are connected to each other through a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0128] A number of components in the electronic device 20 are connected to the I / O interface 25, including: an input unit 26, such as a keyboard, a mouse, etc.; an output unit 27, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 29, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 29 allows the electronic device 20 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0129] The processor 21 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 21 performs the various methods and processes described above.

[0130] In some embodiments, the method of the above embodiment may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 20 via the ROM 22 and / or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the processor 21, one or more steps in the method described above may be performed. Alternatively, in other embodiments, the processor 21 may be configured to perform the method of the above embodiment in any other appropriate manner (e.g., by means of firmware).

[0131] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0133] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and

[0135] Input from the user may be received in any form, including acoustic input, voice input, or tactile input.

[0136] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0137] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0138] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An interactive robot control method, characterized in that: include: Real-time control of semantic understanding process based on dynamic model of conversation context; Perform weight adaptive optimization and adjustment on multimodal input signals; Construct a feedback correction mechanism for cross-modal information fusion; Generate consistent responses to maintain conversational coherence.

2. An interactive robot control method according to claim 1, characterized in that: The real-time regulation of semantic understanding process based on the dynamic model of the dialogue context further includes: Based on historical conversation data and current input, calculate the importance weight of each conversation turn to determine the key conversation nodes; Extract semantic features through natural language processing technology, and optimize model parameters based on conversation intent prediction results; Dynamically adjust the context window size based on the current conversation state to more accurately capture conversation change trends; Semantic misunderstandings in conversations are corrected based on pre-set correction rules.

3. An interactive robot control method according to claim 2, characterized in that: The real-time regulation of semantic understanding process based on the dynamic model of the dialogue context further includes: Based on the current conversation content and the user's emotional state, a sentiment analysis model is built to enhance the accuracy of understanding; If the sentiment score of the current dialogue node is E and the importance weight is W, when E>θ1 and W<θ2, the enhanced semantic parsing function is activated; where E represents the sentiment score, θ1 represents the lower limit of the threshold, W represents the node weight, and θ2 represents the upper limit of the weight; Combine long-term historical conversation records for deep learning training to gradually improve the model's predictive ability; Use the conversation interruption detection mechanism to promptly identify fault points and prevent conversation misunderstandings from continuing into subsequent processes.

4. An interactive robot control method according to claim 3, characterized in that: The method based on the current conversation content and the user's emotional state further includes: Analyze users’ expression habits and intonation preferences based on historical data from multiple rounds of conversations; Use Bayesian algorithm to evaluate users' sentiment tendency and output confidence interval as the basis for adjustment; If the semantic inconsistency of the current dialogue node is D and the confidence interval width is CI, when D>μ and CI>ν, the recalibration mechanism is triggered; where D represents the inconsistency score, μ represents the inconsistency threshold, CI represents the interval width, and ν represents the width threshold; Implement emotional compensation strategies to provide additional information interpretation support for possible semantic misunderstandings or emotional deviations.

5. An interactive robot control method according to claim 4, characterized in that: The historical data based on multiple rounds of dialogue further includes: Generate a conversation fingerprint based on the actual content of each round of conversation to form a multi-modal comprehensive reference framework; Use deep learning algorithms to identify key patterns in graphs and provide support for understanding complex contexts; If the graph consistency score of each conversation is C and the average conversation length is L, when C≤γ and L≥λ, it is recommended to use the method of extending the context window length; where C represents the graph consistency score, γ represents the graph consistency threshold, L represents the average conversation length, and λ represents the length threshold; Compare and match the feature information in the graph with historical cases to ensure effective recognition of new dialogue structures.

6. An interactive robot control method according to claim 5, characterized in that: Generating a conversation fingerprint map based on the actual content of each round of conversation further includes: Generate high-quality conversation fingerprints based on a combination of natural language understanding and image feature extraction technology; Introduce expert rules to handle high-risk error points specially to ensure the accuracy of understanding; If the integrity score of the generated conversation fingerprint is P and the credibility is T, when P<τ and T<ρ, a manual verification prompt signal is triggered; where P represents the fingerprint integrity score, τ represents the integrity baseline, T represents the credibility score, and ρ represents the credibility threshold; Simulate human experts to perform manual verification and improve the final confirmation rate based on the automated process.

7. An interactive robot control method according to claim 6, characterized in that: The real-time regulation of semantic understanding process based on the dynamic model of the dialogue context further includes: Build a multi-layer neural network to simulate the way the brain processes conversation information; Apply machine learning to automatically adjust model hyperparameters to improve the ability to adapt to new scenarios; If the model effect index M after each adjustment varies greatly between [a, b], the adaptive feedback mechanism is activated; where M represents the model performance score, a represents the lower bound of performance, and b represents the upper bound of performance; Combined with speech synthesis engine to generate natural and fluent voice responses.

8. An interactive robot control system for implementing the method according to any one of claims 1 to 7, characterized in that: include: A control unit, wherein the control unit controls the semantic understanding process in real time based on a dynamic model of the dialogue context; An optimization unit, the optimization unit is used to perform weight adaptive optimization adjustment on the multimodal input signal; A correction unit, wherein the correction unit is used to construct a feedback correction mechanism for cross-modal information fusion; A generating unit is used to generate a response with continuous consistency to maintain the continuity of the conversation.

9. An electronic device, characterized in that: The method comprises at least one processor, wherein the processor is communicatively connected to at least one memory, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 7 when the processor executes the computer instructions.

Images