Interactive robot intelligent man-machine interaction method

Abstract

The invention discloses an intelligent man-machine interaction method for an interactive robot. According to the method, the interaction accuracy and adaptability are remarkably improved through multi-dimensional technology fusion. On the intention understanding level, deep analysis of user requirements is achieved through dynamic feature weight distribution, and a strategy is generated by combining cross-domain knowledge graph embedding and emotional expression, so that response content can accurately match user core appeals and can also be naturally fused into professional knowledge and emotional temperature, and information transmission deviation is effectively reduced. According to the real-time feedback mechanism, a dynamic preference model is constructed by capturing facial expressions, voice intonation and behavior data, so that the system can adjust a content generation strategy according to instant emotional fluctuation and attention change of a user, personalized experience in an interaction process is enhanced, mechanical feeling brought by a fixed response mode is avoided, and user experience is improved. Cooperation of content generation and rhythm control is realized, so that the speed, pause and emotion expression of voice response form a natural rhythm, and the comfort level of information receiving is improved.

Description

Technical Field

[0001] This invention belongs to the field of interactive robot technology, specifically an interactive robot intelligent human-computer interaction method. Background Technology

[0002] Interactive robots are intelligent devices with the capabilities of perception, understanding, response, and interaction. Through technologies such as speech recognition, natural language processing, computer vision, and sensor fusion, they enable natural and fluent dialogue and collaboration with humans. They can be widely applied in customer service guidance, educational companionship, smart homes, medical assistance, and commercial services, providing various functions such as information retrieval, task execution, and emotional interaction. Interactive robots are typically equipped with AI algorithms that can adjust their response strategies in real time based on user input, supporting multi-turn dialogues and personalized services. Some products also possess capabilities such as movement, facial expression recognition, and gesture control, enhancing the user experience. With the development of artificial intelligence and IoT technologies, interactive robots are gradually integrating into daily life and work scenarios, becoming an important medium for human-computer interaction.

[0003] However, existing technologies in human-computer interaction generally suffer from superficial understanding of intent and delayed feedback responses. They rely heavily on fixed rules or static user profiles to generate responses, making it difficult to dynamically integrate cross-domain knowledge and emotional expression, resulting in mechanical and rigid interactive content. At the same time, they lack real-time multimodal feedback and closed-loop iteration mechanisms, making it impossible to adjust strategies according to changes in users' immediate emotions and attention. This results in insufficient system adaptability and personalized experience, making it difficult to form continuously evolving interactive capabilities. Summary of the Invention

[0004] The purpose of this invention is to provide an interactive robot intelligent human-computer interaction method to solve the problems mentioned above.

[0005] The technical solution adopted in this invention is as follows: an interactive robot intelligent human-computer interaction method, the method comprising the following steps: S1: Capture user interaction requests via voice or text input and pass the raw input data to S2 for processing; S2: Integrate the language input obtained by S1 with the user's non-verbal signals to form a multi-dimensional interactive information package and send it to S3; S3: Construct a real-time context model based on the interaction information package provided by S2, which includes the user's current needs and historical interaction characteristics, to provide a basis for decision-making in S4; S4: Based on the context model of S3, analyze the user's deep intent and synchronize the intent classification results to the S5 content generation module and the S7 task planning module; S5: Combining the user intent analyzed in S4 with the interaction preferences fed back in S8, generate response content that integrates cross-domain knowledge and emotional expression, and pass it to S6 for rhythm control; S6: Based on the complexity of the content generated by S5 and the task progress arrangement of S7, dynamically adjust the interval of interaction rounds and information density to ensure smooth dialogue. S7: After receiving the intention instruction from S4, it coordinates with the rhythm control parameters of S6 to plan the task execution path and synchronizes the phased results to S9 for synchronous behavior adjustment. S8: Capture user feedback signals to S5's responses in real time, analyze emotional tendencies and attention changes, and generate preference data to feed back to S5 to optimize the generation strategy. S9: By combining the task planning scheme of S7 and the feedback data of S8, the robot's language style, movement range and other interactive behavior parameters are dynamically adjusted to achieve personalized interaction adaptation.

[0006] In a preferred embodiment, in step S1, during the system startup phase, user interaction data is synchronously collected via a distributed sensor array. Text input uses a 16-bit Unicode encoding format, and the sampling frequency is set to 50Hz to ensure real-time performance. When handwriting input is supported, the handwriting sampling interval is controlled within 8ms. Voice signals are collected via a dual-microphone array with a fixed sampling rate of 44.1kHz and quantization precision of 24 bits. The preamplifier gain is dynamically adjusted according to ambient noise, and noise reduction mode is automatically activated when background noise exceeds 65dB. The environmental sensor group collects data at 500ms intervals, including triaxial acceleration (range ±16g), ambient light intensity (0-100000lux), and temperature and humidity (temperature 0-50℃, humidity 20%~90%RH) parameters. All raw data is transmitted to the preprocessing unit via a USB 3.0 interface, with transmission latency controlled within 20ms.

[0007] In a preferred embodiment, in step S2, the text data is first processed by word segmentation using a word segmentation model based on a bidirectional maximum matching algorithm. The dictionary covers 800,000 commonly used words, and the accuracy of out-of-vocabulary word recognition is no less than 92%. The speech signal is processed by Hamming windowing, with a frame length of 25ms and a frame shift of 10ms. After 2048-point FFT transformation, 40-dimensional Mel-frequency cepstral coefficients are extracted. At the same time, the fundamental frequency trajectory (range of 50-500Hz) and short-time energy (the average energy of each frame is normalized to the 0-1 range) are calculated. Sensor data is denoised using Kalman filtering. The acceleration signal is processed by second-order low-pass filtering (cutoff frequency 5Hz) to calculate the motion vector magnitude. Environmental parameters are normalized using Z-score. All modal features are finally uniformly converted into 256-dimensional floating-point vectors and stored in a shared memory buffer for fusion.

[0008] In a preferred embodiment, in step S3, a pre-trained scene classification model is loaded upon system startup. This model is trained based on the ResNet-50 architecture, with the fused feature vector output from S2 as input and a 128-dimensional scene probability distribution vector outputting 16 scene categories, including office, home, and travel. Historical interaction data is retrieved from a time-series database using records from the last 90 days, and compressed into 64-dimensional preference features using an improved LSTM network. The demand type distribution is normalized using softmax, and the response duration feature is scaled to the 0-1 range after logarithmic transformation. Current features and historical preferences are associated through a knowledge graph, and entity similarity is calculated using the TransE algorithm. When the similarity exceeds 0.75, a preference reinforcement mechanism is triggered. The context model is updated every 100ms, with the update threshold set to a cosine distance between the feature vectors greater than 0.15. Model parameters are stored in a Redis cache for millisecond-level access.

[0009] In a preferred embodiment, step S4 uses a bidirectional long short-term memory network as its basic architecture and employs a three-layer progressive processing approach to achieve context modeling. First, the algorithm transforms the multimodal data (text, speech, sensor data, etc.) transmitted in S2 into a unified feature vector: text information is semantically embedded using a pre-trained language model; speech signals are converted into an emotion feature matrix through Mel-spectrum analysis; and device sensor data are used to generate dynamic trajectory vectors through temporal differential encoding. These feature vectors are weighted at the fusion layer using an attention mechanism. For example, when the word "urgent" appears in the user's speech, the weight of the emotion feature temporarily increases to 0.6, while the text semantic weight remains at 0.4, forming a dynamically balanced feature combination.

[0010] The formula for dynamic feature weight allocation is: ; Where Wt represents the multimodal feature fusion weight at time t; Sem(T) is the semantic feature value of the text, which is calculated by word vector cosine similarity; Emo(V) is the emotional intensity of speech, with a value range of [0,1], and is determined by the Mel frequency cepstral coefficients and the rate of change of speech rate. Sen(S) represents the dynamic characteristics of the sensor, which includes normalized data of device motion acceleration and ambient light intensity.

[0011] α is the semantic sentiment balance coefficient, which is automatically increased to 0.7 when an interjection is detected in the user's speech; β is a sensor characteristic adjustment factor, which is 0.3 in outdoor scenes and decreases to 0.1 in indoor scenes; In the historical interaction feature integration stage, an improved gated recurrent unit is used to compress user interaction data from the past three months into a 128-dimensional preference feature vector. This vector includes implicit features such as demand type distribution (e.g., 60% for inquiries and 30% for commands) and response time patterns (average reply interval of 2.3 seconds). Current scenario features are linked to historical preference features using knowledge graph embedding technology. For example, when the system detects that a user has currently entered "recommend a mobile phone," it automatically activates the preference tag "focus on battery life parameters" from historical interactions and strengthens the weight of this feature in the context model through a graph attention network. The final generated context model includes core parameters such as scenario type probability distribution (e.g., 0.85 for shopping decision scenarios and 0.15 for casual conversation scenarios), demand urgency scores (out of 1 to 5), and historical preference matching degrees. These data are output to the S4 module in structured tensor form, providing quantitative decision-making basis for intent parsing.

[0012] The formula for historical preference fusion is: ; In the formula: Hf is the fused historical preference feature vector; Hp is a user's interaction preference matrix over the past 90 days, which includes the proportion of demand types and the distribution of response time. Sim(C,Hp) represents the cosine similarity between the current scene feature C and the historical preference. When the similarity is higher than 0.6, the preference reinforcement mechanism is triggered. γ is the time decay coefficient, which is dynamically adjusted according to the number of days since the interaction occurred. For example, for interaction data within 30 days, γ is 0.8, and for data older than 60 days, it is reduced to 0.4, ensuring that recent interaction features have a higher weight in the current context model.

[0013] In a preferred embodiment, step S5 uses a deep bidirectional Transformer architecture and a dual encoder-decoder structure to dynamically integrate multi-source information. First, the user intent vector parsed in S4 (containing elements such as need type, urgency, and topic keywords) and the preference feature matrix fed back in S8 are input to the intent encoder and preference encoder, respectively. The intent encoder employs a sparse self-attention mechanism to focus on capturing the association weights between "core needs" and "potential expectations." The preference encoder extracts high-frequency preference features through a convolutional neural network; for example, the user's preference intensity for "concise expression" is converted into a feature value of 0.8. The output vectors of the two encoders interact in the fusion layer through a cross-attention mechanism: the "technical consultation" label in the intent vector automatically activates the "acceptability of technical terms" feature in the preference matrix, calculating the cosine similarity between the two (with a threshold set to 0.75). If the similarity exceeds the threshold, the weight of this preference feature is increased, forming a fused context vector.

[0014] In a preferred embodiment, in step S6, after receiving the speech rhythm unit generated in S5, the system first parses the speech rate markers to generate a basic temporal sequence. The standard speech rate is set to 150 words / minute, increasing to 180 words / minute in fast mode and decreasing to 120 words / minute in slow mode. The pause duration is dynamically adjusted according to the punctuation type: commas correspond to 0.3–0.5 seconds, periods to 0.8–1.2 seconds, and exclamation marks are shortened by 20% for positive emotions and lengthened by 30% for negative emotions. The intonation curve is generated using third-order B-spline interpolation, with a fundamental frequency range of 100–300Hz. The frequency is increased by 50Hz at the end of a question and decreased by 20Hz at the end of a declarative sentence. For every 0.2 change in emotional polarity, the intonation fluctuation amplitude is adjusted by 15%. The rhythm control parameters are transmitted to the audio synthesis module via UDP protocol, with the transmission delay required to be controlled within 50ms to ensure lip-sync.

[0015] In a preferred embodiment, in step S7, the text-to-speech conversion uses a hybrid excitation LPC coding algorithm with a frame length of 20ms and an overlap of 5ms, a linear prediction coefficient order of 12, and the residual signal is generated by random pulse and noise mixing excitation. Audio compression uses MP3 format, with dynamically adjusted bitrate (128kbps in quiet environments and 192kbps in noisy environments), and a fixed sampling rate of 44.1kHz in stereo mode. For text output, an adaptive typesetting algorithm is used, with line width automatically adjusted according to screen size (32 characters / line on mobile devices and 48 characters / line on tablets). Font size and line spacing are set according to the golden ratio of 1:1.618, and key content is highlighted through weight variations (from 500 for regular to 700 for bold). Output latency is required to be ≤100ms for text and ≤300ms for speech.

[0016] In a preferred embodiment, in step S8, after the user receives the response content generated in S5, the system captures changes in facial micro-expressions (such as the angle of the corners of the mouth and the frequency of frowning) through the front-facing camera, extracts the intonation fluctuations in the voice response (such as the amplitude of tone changes and the fluctuation of speech rate), and records the behavioral data of the interactive interface (such as the screen click position, the duration of content stay, and the scrolling speed).

[0017] These multi-source feedback signals are first converted into standardized feature values: facial expressions are generated into emotion intensity values ​​of 0-10 using facial keypoint tracking technology; speech signals are converted into sentiment tendency parameters (positive values ​​represent positive emotions, negative values ​​represent negative emotions) through spectral analysis; and behavioral data are converted into attention distribution vectors through temporal coding (higher values ​​indicate higher attention to the content in that area). The system fuses these feature values ​​in real time. For example, when a user's mouth angle exceeds 15 degrees and they click the "favorite" button, the system automatically increases the positive feedback weight of the current response and compares this signal with historical interaction data: if the user has shown more than 80% attention dwell time for "step-by-step decomposition" responses in the past three times, the "structured expression" label will be strengthened in the preference model. The fused feedback data generates dynamic adjustment instructions. For example, when it is detected that the user's dwell time on a paragraph of technical terms is less than 2 seconds, an optimization signal to "reduce the density of technical terms" is automatically sent to S5, and the preference feature is stored in the user profile database for subsequent interaction strategy adjustments.

[0018] The formula for dynamically allocating emotional attention weights is: ; In the formula: Wa represents the overall attention weight value, which ranges from 0 to 10. The higher the value, the stronger the user's attention and positive emotions towards the current content.

[0019] wf, wv, and wb are the modal weights for facial expressions, voice intonation, and behavioral data, respectively. The initial value of each is 1, and the corresponding weight is automatically reduced to 0.2 when a modality of data is missing.

[0020] F is the facial emotion feature value, which is calculated through changes in eye corner curvature and lip shape. V is a speech emotion parameter, which is determined by the fundamental frequency change rate and the energy fluctuation amplitude; B represents the behavioral attention vector, which is calculated by combining click frequency and dwell time.

[0021] γ is the time decay coefficient, which is fixed at 0.15; Δt represents the time interval (in seconds) between user feedback and content presentation, used to mitigate the impact of delayed feedback on the current preference judgment.

[0022] In a preferred embodiment, in step S9, user interaction data is written to a time-series database in real time, using a sliding window mechanism to retain records from the most recent 30 days, with a data sampling frequency of 5Hz. The preference update algorithm employs an incremental learning framework, with an initial learning rate set to 0.001, a decay coefficient of 0.1 every 1000 interactions, and the feature space dimension remaining constant at 256. Feedback weights decay over time using an exponential function e^(-ε).(-t / τ) The calculation involves τ representing a 7-day half-life, with positive feedback weights set to 1.5 and negative feedback weights set to 0.8. Model parameters are updated in batches daily at 3 AM using the Adam optimizer with a batch size of 256, iterating until the loss function falls below 0.01. During the update process, the system switches to a backup model to ensure uninterrupted service.

[0023] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention significantly improves the accuracy and adaptability of interaction through the integration of multi-dimensional technologies. At the intent understanding level, dynamic feature weight allocation enables deep analysis of user needs. Combined with cross-domain knowledge graph embedding and emotional expression generation strategies, the response content accurately matches the user's core demands while naturally incorporating professional knowledge and emotional warmth, effectively reducing information transmission bias. The real-time feedback mechanism captures facial expressions, voice tone, and behavioral data to construct a dynamic preference model, allowing the system to adjust its content generation strategy based on the user's immediate emotional fluctuations and attention changes. This enhances the personalized experience during interaction and avoids the mechanical feeling brought by fixed response patterns.

[0024] 2. In this invention, the synergy between content generation and rhythm control creates a natural rhythm in the speech rate, pauses, and emotional expression of the voice response, enhancing the comfort of information reception. The closed-loop iterative mechanism consisting of S8 and S9 ensures that the system can continuously learn long-term user preferences, dynamically balancing the weight of historical data and real-time feedback through a time decay coefficient, allowing the interaction strategy to evolve with user habits and strengthening the system's scalability. This end-to-end design, from intent parsing to feedback optimization, ultimately realizes the transformation of human-computer interaction from passive response to active adaptation, significantly improving users' trust in the intelligent system and their stickiness to use it. Attached Figure Description

[0025] Figure 1 This is a schematic diagram illustrating the process principle of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0027] Reference Figure 1 , An interactive robot intelligent human-computer interaction method, the method includes the following steps: S1: Capture user interaction requests via voice or text input and pass the raw input data to S2 for processing; S2: Integrate the language input obtained by S1 with the user's non-verbal signals, form a multi-dimensional interactive information package, and send it to S3; S3: Construct a real-time context model based on the interaction information package provided by S2, which includes the user's current needs and historical interaction characteristics, to provide a basis for decision-making in S4; S4: Based on the context model of S3, analyze the user's deep intent and synchronize the intent classification results to the S5 content generation module and the S7 task planning module; S5: Combining the user intent analyzed in S4 with the interaction preferences fed back in S8, generate response content that integrates cross-domain knowledge and emotional expression, and pass it to S6 for rhythm control; S6: Based on the complexity of the content generated by S5 and the task progress arrangement of S7, dynamically adjust the interval of interaction rounds and information density to ensure smooth dialogue. S7: After receiving the intention instruction from S4, it coordinates with the rhythm control parameters of S6 to plan the task execution path and synchronizes the phased results to S9 for synchronous behavior adjustment. S8: Capture user feedback signals to S5's responses in real time, analyze emotional tendencies and attention changes, and generate preference data to feed back to S5 to optimize the generation strategy; S9: By combining the task planning scheme of S7 and the feedback data of S8, the robot's language style, movement range and other interactive behavior parameters are dynamically adjusted to achieve personalized interaction adaptation.

[0028] In step S1, during the system startup phase, user interaction data is synchronously collected via a distributed sensor array. Text input uses a 16-bit Unicode encoding format, and the sampling frequency is set to 50Hz to ensure real-time performance. When handwriting input is supported, the handwriting sampling interval is controlled within 8ms. Voice signals are collected via a dual-microphone array with a fixed sampling rate of 44.1kHz and quantization precision of 24 bits. The preamplifier gain is dynamically adjusted according to ambient noise, and noise reduction mode is automatically activated when background noise exceeds 65dB. The environmental sensor group collects data at 500ms intervals, including triaxial acceleration (range ±16g), ambient light intensity (0-100000lux), and temperature and humidity (temperature 0-50℃, humidity 20%~90%RH) parameters. All raw data is transmitted to the preprocessing unit via a USB 3.0 interface, with transmission latency controlled within 20ms.

[0029] In step S2, the text data is first processed through word segmentation using a segmentation model based on a bidirectional maximum matching algorithm. The dictionary covers 800,000 commonly used words, with an out-of-vocabulary word recognition accuracy of no less than 92%. The speech signal is processed using a Hamming window with a frame length of 25ms and a frame shift of 10ms. After a 2048-point FFT transformation, 40-dimensional Mel-frequency cepstral coefficients are extracted. Simultaneously, the fundamental frequency trajectory (range of 50–500Hz) and short-time energy are calculated (the average energy of each frame is normalized to the 0-1 range). Sensor data is denoised using Kalman filtering. The acceleration signal is processed by a second-order low-pass filter (cutoff frequency of 5Hz) to calculate the motion vector magnitude. Environmental parameters are normalized using Z-score. All modal features are ultimately converted into 256-dimensional floating-point vectors and stored in a shared memory buffer for fusion.

[0030] In step S3, the pre-trained scene classification model is loaded upon system startup. This model is trained based on the ResNet-50 architecture, with the fused feature vector output from S2 as input and a 128-dimensional scene probability distribution vector outputting 16 scene categories, including office, home, and travel. Historical interaction data is retrieved from a time-series database using records from the last 90 days, and compressed into 64-dimensional preference features using an improved LSTM network. The demand type distribution is normalized using softmax, and the response duration feature is scaled to the 0-1 range after logarithmic transformation. Current features and historical preferences are linked through a knowledge graph, and the TransE algorithm is used to calculate entity similarity. When the similarity exceeds 0.75, a preference reinforcement mechanism is triggered. The context model is updated every 100ms, with the update threshold set to a cosine distance between the feature vectors greater than 0.15. Model parameters are stored in a Redis cache for millisecond-level access.

[0031] In step S4, context modeling is achieved through a three-layer progressive processing based on a bidirectional long short-term memory network architecture. First, the algorithm transforms the multimodal data (text, speech, sensor data, etc.) transmitted in S2 into a unified feature vector: text information is semantically embedded using a pre-trained language model; speech signals are converted into an emotion feature matrix through Mel-spectrum analysis; and device sensor data are used to generate dynamic trajectory vectors through temporal differential encoding. These feature vectors are weighted in the fusion layer using an attention mechanism. For example, when the word "urgent" appears in the user's speech, the weight of the emotion feature temporarily increases to 0.6, while the text semantic weight remains at 0.4, forming a dynamically balanced feature combination.

[0032] The formula for dynamic feature weight allocation is: ; Where Wt represents the multimodal feature fusion weight at time t; Sem(T) is the semantic feature value of the text, which is calculated by word vector cosine similarity; Emo(V) is the emotional intensity of speech, with a value range of [0,1], and is determined by the Mel frequency cepstral coefficients and the rate of change of speech rate. Sen(S) represents the dynamic characteristics of the sensor, which includes normalized data of device motion acceleration and ambient light intensity.

[0033] α is the semantic sentiment balance coefficient, which is automatically increased to 0.7 when an interjection is detected in the user's speech; β is a sensor characteristic adjustment factor, which is 0.3 in outdoor scenes and decreases to 0.1 in indoor scenes; In the historical interaction feature integration stage, an improved gated recurrent unit is used to compress user interaction data from the past three months into a 128-dimensional preference feature vector. This vector includes implicit features such as demand type distribution (e.g., 60% for inquiries and 30% for commands) and response time patterns (average reply interval of 2.3 seconds). Current scenario features are linked to historical preference features using knowledge graph embedding technology. For example, when the system detects that a user has currently entered "recommend a mobile phone," it automatically activates the preference tag "focus on battery life parameters" from historical interactions and strengthens the weight of this feature in the context model through a graph attention network. The final generated context model includes core parameters such as scenario type probability distribution (e.g., 0.85 for shopping decision scenarios and 0.15 for casual conversation scenarios), demand urgency scores (out of 1 to 5), and historical preference matching degrees. These data are output to the S4 module in structured tensor form, providing quantitative decision-making basis for intent parsing.

[0034] The formula for historical preference fusion is: ; In the formula: Hf is the fused historical preference feature vector; Hp is a user interaction preference matrix over the past 90 days, which includes the proportion of demand types and the distribution of response time. Sim(C,Hp) represents the cosine similarity between the current scene feature C and the historical preference. When the similarity is higher than 0.6, the preference reinforcement mechanism is triggered. γ is the time decay coefficient, which is dynamically adjusted according to the number of days since the interaction occurred. For example, for interaction data within 30 days, γ is 0.8, and for data older than 60 days, it is reduced to 0.4, ensuring that recent interaction features have a higher weight in the current context model.

[0035] In step S5, a deep bidirectional Transformer architecture is used as the basic framework, and a dual encoder-decoder structure is employed to dynamically integrate multi-source information. First, the user intent vector parsed in S4 (containing elements such as need type, urgency, and topic keywords) and the preference feature matrix fed back in S8 are input to the intent encoder and preference encoder, respectively. The intent encoder uses a sparse self-attention mechanism to focus on capturing the association weights between "core needs" and "potential expectations." The preference encoder extracts high-frequency preference features through a convolutional neural network; for example, the user's preference intensity for "concise expression" is converted into a feature value of 0.8. The output vectors of the two encoders interact in the fusion layer through a cross-attention mechanism: the "technical consultation" label in the intent vector automatically activates the "acceptability of technical terms" feature in the preference matrix, calculating the cosine similarity between the two (with a threshold set to 0.75). If the similarity exceeds the threshold, the weight of this preference feature is increased, forming a fused context vector.

[0036] In step S6, after receiving the speech rhythm unit generated in S5, the system first parses the speech rate markers to generate a basic temporal sequence. The standard speech rate is set to 150 words / minute, increasing to 180 words / minute in fast mode and decreasing to 120 words / minute in slow mode. Pause duration is dynamically adjusted according to punctuation type: commas correspond to 0.3–0.5 seconds, periods to 0.8–1.2 seconds, and exclamation marks are shortened by 20% for positive emotions and lengthened by 30% for negative emotions. The intonation curve is generated using third-order B-spline interpolation, with a fundamental frequency range of 100–300Hz. The frequency is increased by 50Hz at the end of a question and decreased by 20Hz at the end of a declarative sentence. For every 0.2 change in emotional polarity, the intonation fluctuation amplitude is adjusted by 15%. Rhythm control parameters are transmitted to the audio synthesis module via UDP protocol, with transmission latency required to be controlled within 50ms to ensure lip-sync.

[0037] In step S7, the text-to-speech conversion uses a hybrid excitation LPC coding algorithm with a frame length of 20ms and an overlap of 5ms. The linear prediction coefficients are of order 12, and the residual signal is generated by a mixture of random impulses and noise. Audio compression uses MP3 format with dynamically adjusted bitrate (128kbps in quiet environments and 192kbps in noisy environments) and a fixed sampling rate of 44.1kHz in stereo mode. For text output, an adaptive typesetting algorithm is used, with line width automatically adjusted according to screen size (32 characters / line on mobile devices and 48 characters / line on tablets). Font size and line spacing are set according to the golden ratio of 1:1.618. Important content is highlighted through variations in font weight (from 500 for regular to 700 for bold). Output latency is required to be ≤100ms for text and ≤300ms for speech.

[0038] In step S8, after the user receives the response content generated in S5, the system captures changes in facial micro-expressions (such as the angle of the corners of the mouth and the frequency of frowning) through the front camera, extracts the intonation fluctuations in the voice response (such as the amplitude of tone changes and the fluctuation of speech rate), and records the behavioral data of the interactive interface (such as the screen click position, the duration of content stay, and the scrolling speed).

[0039] These multi-source feedback signals are first converted into standardized feature values: facial expressions are generated into emotion intensity values ​​of 0-10 using facial keypoint tracking technology; speech signals are converted into sentiment tendency parameters (positive values ​​represent positive emotions, negative values ​​represent negative emotions) through spectral analysis; and behavioral data are converted into attention distribution vectors through temporal coding (higher values ​​indicate higher attention to the content in that area). The system fuses these feature values ​​in real time. For example, when a user's mouth angle exceeds 15 degrees and they click the "favorite" button, the system automatically increases the positive feedback weight of the current response and compares this signal with historical interaction data: if the user has shown more than 80% attention dwell time for "step-by-step decomposition" responses in the past three times, the "structured expression" label will be strengthened in the preference model. The fused feedback data generates dynamic adjustment instructions. For example, when it is detected that the user's dwell time on a technical terminology paragraph is less than 2 seconds, the system automatically sends an optimization signal to S5 to "reduce the density of technical terms," ​​and simultaneously stores this preference feature in the user profile database for subsequent interaction strategy adjustments.

[0040] The formula for dynamically allocating emotional attention weights is: ; In the formula: Wa represents the overall attention weight value, which ranges from 0 to 10. The higher the value, the stronger the user's attention and positive emotions towards the current content.

[0041] wf, wv, and wb are the modal weights for facial expressions, voice intonation, and behavioral data, respectively. The initial value of each is 1, and the corresponding weight is automatically reduced to 0.2 when a modality of data is missing.

[0042] F is the facial emotion feature value, which is calculated through changes in eye corner curvature and lip shape. V is a speech emotion parameter, which is determined by the fundamental frequency change rate and the energy fluctuation amplitude; B represents the behavioral attention vector, which is calculated by combining click frequency and dwell time.

[0043] γ is the time decay coefficient, which is fixed at 0.15; Δt represents the time interval (in seconds) between user feedback and content presentation, used to mitigate the impact of delayed feedback on the current preference judgment.

[0044] In step S9, user interaction data is written to the time-series database in real time, using a sliding window mechanism to retain records from the most recent 30 days, with a data sampling frequency of 5Hz. The preference update algorithm employs an incremental learning framework, with an initial learning rate set to 0.001, a decay coefficient of 0.1 every 1000 interactions, and the feature space dimension remaining constant at 256. Feedback weights decay over time using an exponential function e^(-ε). (-t / τ) The calculation involves τ representing a 7-day half-life, with positive feedback weights set to 1.5 and negative feedback weights set to 0.8. Model parameters are updated in batches daily at 3 AM using the Adam optimizer with a batch size of 256, iterating until the loss function falls below 0.01. During the update process, the system switches to a backup model to ensure uninterrupted service.

[0045] From the above, we can conclude that: This invention significantly improves the accuracy and adaptability of interaction through the integration of multi-dimensional technologies. At the intent understanding level, step S4 achieves deep analysis of user needs through dynamic feature weight allocation. Combined with cross-domain knowledge graph embedding and emotional expression generation strategies, the response content accurately matches the user's core demands while naturally incorporating professional knowledge and emotional warmth, effectively reducing information transmission bias. The real-time feedback mechanism captures facial expressions, voice tone, and behavioral data to construct a dynamic preference model, allowing the system to adjust its content generation strategy based on the user's immediate emotional fluctuations and attention changes. This enhances the personalized experience during interaction and avoids the mechanical feeling of fixed response patterns.

[0046] This invention achieves synergy between content generation and rhythm control, creating a natural rhythm in the speech rate, pauses, and emotional expression of voice responses, thus enhancing the comfort of information reception. The closed-loop iterative mechanism consisting of S8 and S9 ensures the system can continuously learn long-term user preferences, dynamically balancing the weight of historical data and real-time feedback through a time decay coefficient. This allows the interaction strategy to evolve with user habits, strengthening the system's scalability. This end-to-end design, from intent parsing to feedback optimization, ultimately realizes a shift in human-computer interaction from passive response to proactive adaptation, significantly improving user trust in the intelligent system and user engagement.

[0047] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0048] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An interactive robot intelligent human-computer interaction method, characterized in that: The method includes the following steps: S1: Capture user interaction requests via voice or text input and pass the raw input data to S2 for processing; S2: Integrate the language input obtained by S1 with the user's non-verbal signals, form a multi-dimensional interactive information package, and send it to S3; S3: Construct a real-time context model based on the interaction information package provided by S2, which includes the user's current needs and historical interaction characteristics, to provide a basis for decision-making in S4; S4: Based on the context model of S3, analyze the user's deep intent and synchronize the intent classification results to the S5 content generation module and the S7 task planning module; S5: Combining the user intent analyzed in S4 with the interaction preferences fed back in S8, generate response content that integrates cross-domain knowledge and emotional expression, and pass it to S6 for rhythm control; S6: Based on the complexity of the content generated by S5 and the task progress arrangement of S7, dynamically adjust the interval of interaction rounds and information density to ensure smooth dialogue. S7: After receiving the intention instruction from S4, it coordinates with the rhythm control parameters of S6 to plan the task execution path and synchronizes the phased results to S9 for synchronous behavior adjustment. S8: Capture user feedback signals to S5's responses in real time, analyze emotional tendencies and attention changes, and generate preference data to feed back to S5 to optimize the generation strategy; S9: By combining the task planning scheme of S7 and the feedback data of S8, the robot's language style, movement range and interaction behavior parameters are dynamically adjusted to achieve personalized interaction adaptation.

2. The interactive robot intelligent human-computer interaction method as described in claim 1, characterized in that: In step S1, during the system startup phase, user interaction data is synchronously collected through a distributed sensor array. Text input uses a 16-bit Unicode encoding format, and the sampling frequency is set to 50Hz to ensure real-time performance. When handwriting input is supported, the handwriting sampling interval is controlled within 8ms. Voice signals are collected through a dual-microphone array with a fixed sampling rate of 44.1kHz and a quantization accuracy of 24 bits. The preamplifier gain is dynamically adjusted according to the ambient noise, and noise reduction mode is automatically activated when the background noise exceeds 65dB. The environmental sensor group collects data with a period of 500ms, including triaxial acceleration, ambient light intensity, and temperature and humidity parameters. All raw data is transmitted to the preprocessing unit through a USB 3.0 interface, and the transmission delay is controlled within 20ms.

3. The interactive robot intelligent human-computer interaction method as described in claim 1, characterized in that: In step S2, the text data is first processed by word segmentation using a word segmentation model based on the bidirectional maximum matching algorithm. The dictionary covers 800,000 commonly used words, and the accuracy of out-of-vocabulary word recognition is no less than 92%. The speech signal is processed by Hamming windowing, with a frame length of 25ms and a frame shift of 10ms. After 2048-point FFT transformation, 40-dimensional Mel frequency cepstral coefficients are extracted, and the fundamental frequency trajectory and short-time energy are calculated simultaneously. The sensor data is denoised using Kalman filtering, and the acceleration signal is calculated by second-order low-pass filtering. The environmental parameters are normalized using Z-score. All modal features are finally uniformly converted into 256-dimensional floating-point vectors and stored in a shared memory buffer for fusion.

4. The interactive robot intelligent human-computer interaction method as described in claim 1, characterized in that: In step S3, when the system starts, a pre-trained scene classification model is loaded. This model is trained based on the ResNet-50 architecture. The input is the fusion feature vector output by S2, and the output is a 128-dimensional scene probability distribution vector containing office, home, and travel scenarios. Historical interaction data is queried from the time series database for the most recent 90 days of records and compressed into 64-dimensional preference features using an improved LSTM network. The demand type distribution is normalized by softmax, and the response duration feature is scaled to the 0-1 range after logarithmic transformation. Current features and historical preferences are linked through a knowledge graph. The TransE algorithm is used to calculate entity similarity. When the similarity exceeds 0.75, a preference reinforcement mechanism is triggered. The context model is updated every 100ms, and the update threshold is set to a feature vector cosine distance greater than 0.

15. The model parameters are stored in a Redis cache to achieve millisecond-level access.

5. The interactive robot intelligent human-computer interaction method as described in claim 1, characterized in that: In step S4, a bidirectional long short-term memory network is used as the basic architecture to achieve context modeling through a three-layer progressive processing. First, the algorithm transforms the text, speech, and sensor multimodal data transmitted in S2 into a unified feature vector: the text information is extracted with semantic embedding through a pre-trained language model, the speech signal is transformed into an emotion feature matrix through Mel spectrum analysis, and the device sensor data is generated into a dynamic trajectory vector through temporal differential encoding. These feature vectors are weighted in the fusion layer through an attention mechanism. The formula for dynamic feature weight allocation is: ； Where Wt represents the multimodal feature fusion weight at time t; Sem(T) is the semantic feature value of the text, which is calculated by word vector cosine similarity; Emo(V) is the emotional intensity of speech, with a value range of [0,1], and is determined by the Mel frequency cepstral coefficients and the rate of change of speech rate. Sen(S) represents the dynamic characteristics of the sensor, which includes normalized data of device motion acceleration and ambient light intensity; α is the semantic sentiment balance coefficient, which is automatically increased to 0.7 when an interjection is detected in the user's speech; β is a sensor characteristic adjustment factor, which is 0.3 in outdoor scenes and decreases to 0.1 in indoor scenes; In the historical interaction feature integration stage, an improved gated recurrent unit is used to compress user interaction data from the past three months into a 128-dimensional preference feature vector, which includes implicit features such as demand type distribution and response time patterns. Current scenario features and historical preference features are associated through knowledge graph embedding technology, and the weight of the feature in the context model is strengthened through graph attention network. The final generated context model includes scenario type probability distribution, demand urgency score, and core parameters of historical preference matching degree. These data are output to the S4 module in the form of structured tensors to provide quantitative decision-making basis for intent parsing. The formula for historical preference fusion is: ； In the formula: Hf is the fused historical preference feature vector; Hp is a user interaction preference matrix over the past 90 days, which includes the proportion of demand types and the distribution of response time. Sim(C,Hp) represents the cosine similarity between the current scene feature C and the historical preference. When the similarity is higher than 0.6, the preference reinforcement mechanism is triggered. γ is the time decay coefficient, which is dynamically adjusted according to the number of days since the interaction occurred. For example, for interaction data within 30 days, γ is 0.8, and for data older than 60 days, it is reduced to 0.4, ensuring that recent interaction features have a higher weight in the current context model.

6. The interactive robot intelligent human-computer interaction method as described in claim 1, characterized in that: In step S5, based on the deep bidirectional Transformer architecture, the dynamic integration of multi-source information is achieved through a dual encoder-decoder structure. First, the user intent vector parsed in S4 and the preference feature matrix fed back in S8 are respectively input to the intent encoder and the preference encoder—the intent encoder adopts a sparse self-attention mechanism.

7. The interactive robot intelligent human-computer interaction method as described in claim 1, characterized in that: In step S6, after receiving the speech rhythm unit generated in S5, the system first parses the speech rate markers to generate a basic temporal sequence. The standard speech rate is set to 150 words / minute, increasing to 180 words / minute in fast mode and decreasing to 120 words / minute in slow mode. The pause duration is dynamically adjusted according to the punctuation type: 0.3-0.5 seconds for commas, 0.8-1.2 seconds for periods, and 20% shorter for exclamation marks during positive emotions and 30% longer during negative emotions. The intonation curve is generated using third-order B-spline interpolation, with a fundamental frequency range of 100-300Hz. The end of a question is increased by 50Hz, and the end of a declarative sentence is decreased by 20Hz. For every 0.2 change in emotional polarity, the intonation fluctuation amplitude is adjusted by 15%. The rhythm control parameters are transmitted to the audio synthesis module via UDP protocol, with the transmission delay required to be controlled within 50ms to ensure lip-phonetic synchronization.

8. The interactive robot intelligent human-computer interaction method as described in claim 1, characterized in that: In step S7, the text-to-speech conversion uses a hybrid excitation LPC coding algorithm with a frame length of 20ms and an overlap of 5ms. The linear prediction coefficients are of order 12, and the residual signal is generated by a mixture of random pulses and noise. The audio compression uses MP3 format with dynamic bit rate adjustment and a fixed sampling rate of 44.1kHz in stereo mode. When outputting text, an adaptive typesetting algorithm is used, with the line width automatically adjusted according to the screen size. The font size and line spacing are set according to the golden ratio of 1:1.618, and important content is highlighted by changes in font weight. The output latency is required to be ≤100ms for text and ≤300ms for speech.

9. The interactive robot intelligent human-computer interaction method as described in claim 1, characterized in that: In step S8, after the user receives the response content generated in S5, the system captures changes in facial micro-expressions through the front camera, extracts the intonation fluctuations in the voice response, and records the behavioral data of the interactive interface. These multi-source feedback signals are first converted into standardized feature values: facial expressions are generated into emotion intensity values ​​of 0-10 through facial key point tracking technology, speech signals are converted into emotional tendency parameters through spectrum analysis, and behavioral data are converted into attention distribution vectors through temporal coding; the system will fuse these feature values ​​in real time; the fused feedback data will generate dynamic adjustment instructions, for example, when it is detected that the user's dwell time on a paragraph of technical terms is less than 2 seconds, an optimization signal of "reducing the density of professional terms" will be automatically sent to S5, and the preference feature will be stored in the user profile database for subsequent interaction strategy adjustments; The formula for dynamically allocating emotional attention weights is: ； In the formula: Wa represents the overall attention weight value, which ranges from 0 to 10. The higher the value, the stronger the user's attention and positive emotions towards the current content. wf, wv, and wb are the modal weights for facial expressions, voice intonation, and behavioral data, respectively. The initial value of each is 1. When a modality of data is missing, the corresponding weight is automatically reduced to 0.

2. F is the facial emotion feature value, which is calculated through changes in eye corner curvature and lip shape. V is a speech emotion parameter, which is determined by the fundamental frequency change rate and the energy fluctuation amplitude; B represents the behavioral attention vector, which is calculated by combining click frequency and dwell time; γ is the time decay coefficient, which is fixed at 0.15; Δt represents the time interval between user feedback and content presentation, used to mitigate the impact of delayed feedback on the current preference judgment.

10. The interactive robot intelligent human-computer interaction method as described in claim 1, characterized in that: In step S9, user interaction data is written to the time-series database in real time, and the sliding window mechanism is used to retain the records of the most recent 30 days. The data sampling frequency is 5Hz. The preference update algorithm adopts an incremental learning framework, with an initial learning rate of 0.001 and a decay coefficient of 0.1 for every 1000 cumulative interactions. The feature space dimension remains unchanged at 256 dimensions. Feedback weights decay over time using an exponential function e. (-t / τ) The calculation involves τ representing a 7-day half-life, with positive feedback weights set to 1.5 and negative feedback weights set to 0.

8. Model parameters are updated in batches daily at 3 AM using the Adam optimizer with a batch size of 256 and iterations until the loss function falls below 0.

01. During the update process, the system switches to a backup model to ensure uninterrupted service.

Images