Man-machine interaction method and system of mobile robot

Abstract

The invention discloses a man-machine interaction method and system of a mobile robot. According to the method, the response content is dynamically adjusted through the narrative style fusion formula, and the fitting degree of the interaction process is directly improved; weighted fusion of a task type template, an environment constraint factor and a user preference parameter in the formula enables the robot to flexibly adjust and express key points according to a specific scene, for example, an instruction structure is automatically simplified in a complex environment, redundant instructions are reduced when a user is familiar with operation, and the communication rhythm better meets actual requirements. Mechanical repetition of response content is avoided by adding a random disturbance item, the natural feeling and the freshness feeling in the interaction process are enhanced, the response speed of the robot can be increased through long-term use, effective transmission of key information is ensured through product calculation of risk consequences and spatial distances, distraction caused by information overload is avoided, and the safety of the robot is improved. And the operation error rate is reduced.

Description

Technical Field

[0001] This invention belongs to the field of robot interaction technology, specifically a human-computer interaction method and system for mobile robots. Background Technology

[0002] Mobile robots are intelligent robots with autonomous mobility. Equipped with sensors, actuators, and control systems, they can perform path planning, obstacle avoidance, and task execution in complex environments. Their core technologies include environmental perception, localization and navigation, decision-making and control, and multi-sensor fusion. They typically employ wheeled, tracked, or legged structures to adapt to different terrains. Mobile robots are widely used in industrial manufacturing, warehousing and logistics, medical services, home companionship, and security patrols, characterized by high efficiency, flexibility, and intelligence. With the development of artificial intelligence, 5G communication, and edge computing, mobile robots are gradually achieving higher levels of autonomy and collaboration, providing intelligent solutions for various industries and driving changes in production and lifestyles. Human-robot interaction in mobile robots refers to the exchange of information and transmission of commands between humans and mobile robots through various sensing and communication technologies, aiming to improve the robot's intelligence level and user experience. Interaction methods include voice recognition, touchscreen operation, gesture control, facial expression recognition, and remote control via mobile apps, integrating natural language processing, computer vision, and sensor technologies to enable the robot to understand user intentions and provide corresponding responses.

[0003] However, existing technologies often suffer from fixed interaction modes, redundant or insufficient feedback information, lack of personalized learning capabilities, and weak path planning flexibility and real-time obstacle avoidance response, resulting in low efficiency of human-machine collaboration and poor user experience. Summary of the Invention

[0004] The purpose of this invention is to provide a human-computer interaction method and system for mobile robots in order to solve the problems mentioned above.

[0005] The technical solution adopted in this invention is as follows: A human-computer interaction method for a mobile robot, the method comprising the following steps: S1: After the robot starts, it automatically scans the surrounding space, builds a basic environmental map and identifies static obstacles, providing spatial context for subsequent interactions. The map data will be synchronized to S2 and S5 in real time.

[0006] S2: Integrates voice commands, body movements, and touch signals, and dynamically adjusts the input priority based on the environmental noise level in S1. The parsing results are transmitted to S3 in real time for intent determination.

[0007] S3: Based on the input signal parsed by S2 and combined with the user's historical interaction habits (from the optimized model of S8), a structured task description is generated, and the task parameters are sent to S4 for feedback confirmation.

[0008] S4: Generate a humanized, scenario-based response based on the task description in S3, prompt relevant precautions for the task and confirm the user's willingness to perform, and simultaneously synchronize the user's confirmation result to S5 to start the execution process.

[0009] S5: Utilizes the environmental map of S1 to plan the optimal movement path, receives obstacle update information from S6 in real time during execution, completes the physical task through wheel speed adjustment and steering control, and provides real-time feedback on the execution status to S6.

[0010] S6: During the execution of S5, status indicators are provided through voice progress announcements, changes in body lights, and micro-movements of joints. The feedback content synchronously affects the timing of evaluation collection in S7.

[0011] S7: After the task is completed, the results are presented proactively and user feedback is collected via voice. The evaluation data will be stored in an encrypted manner and used for model updates in S8.

[0012] S8: Based on user reviews and historical interaction data collected in S7, dynamically adjust the input parsing weights of S2, the narrative style of S4, and the feedback frequency of S6.

[0013] In a preferred embodiment, in step S1, after the robot starts, it performs spatial scanning using a 3D LiDAR and a vision sensor array. The scanning range covers 360 degrees horizontally and -15 to +45 degrees vertically, constructing a rasterized environment map with a resolution of 0.05 meters per pixel. Simultaneously, an ultrasonic sensor is activated to extract the outlines of static obstacles, and the recognition confidence threshold is set to 0.85. The map data is sent to the input parsing module in S2 and the path planning module in S5 via the CAN bus at a frequency of 10 Hz. The initialization phase lasts no more than 15 seconds. If the timeout occurs, a low-power scanning mode is triggered.

[0014] In a preferred embodiment, in step S2, a dual-channel processing architecture is used to receive voice commands, body movements, and touch signals respectively. The sampling rate of the voice commands is 16kHz, and the dynamic range is 80dB. The body movements are based on skeletal keypoint detection with a frame rate of 25fps. The touch signals use capacitive sensing and support simultaneous recognition of 10 points. Voice features are extracted using Mel-frequency cepstral coefficients and combined with the ambient noise decibel value transmitted in S1. The sampling interval is 500ms, and the input weight matrix is ​​dynamically adjusted—when the noise value is higher than 65dB, the voice weight is reduced by 30% and the gesture recognition weight is increased. The parsing results are processed by a feature fusion network based on an attention mechanism and then pushed to the intent understanding module in S3 in real time in JSON format.

[0015] In a preferred embodiment, in step S3, a sequence labeling model combining bidirectional LSTM and CRF is used. The input layer receives the multimodal feature vector parsed in S2, with a dimension of 256. At the same time, it calls the user's historical interaction records in the local encrypted database, which include data from the last 90 days. The data is automatically anonymized daily. The task similarity score is generated by calculating cosine similarity, with a threshold set to 0.7. The structured task description includes action type, divided into 6 major categories and 32 sub-items, target attributes, including shape, color, and material triples and spatial coordinates, and an absolute coordinate system based on the map in S1. The task parameters are encapsulated using the protobuf protocol and sent to the feedback generation unit in S4.

[0016] In a preferred embodiment, in step S4, the core actions, target objects, and spatial attributes are first extracted by the task semantic parsing module, while simultaneously generating contextual constraint factors by calling the environmental dynamic database (linking the static map of S1 with the real-time obstacle data of S5). Subsequently, the narrative style decision layer, combined with the personalized model parameters of S8 (including the user's preference coefficient for interaction simplicity and historical emotional feedback weights), matches a basic framework from a predefined narrative template library. The precaution extraction submodule scans potential risk points in the task path (such as dynamic obstacle density and ground flatness threshold) using a rule engine and converts them into natural language warning elements. Finally, the response generator uses a multi-dimensional fusion algorithm to integrate the core task instructions, contextual constraint factors, precautions, and user preference parameters into coherent text, synchronously triggering an execution confirmation request; the user confirmation signal is transmitted in real-time to the execution queue initiator of S5 via the state synchronization bus, while the unconfirmed state returns to S3 to re-enter the intent correction process.

[0017] The formula for fusing narrative styles is: ; In the formula: S represents the final generated narrative text vector (dimension = 512, based on the BERT pre-trained word embedding space). T represents the task type basic template vector (such as the predefined templates "grab" and "navigate", which are determined by the S3 task description); E represents the context constraint factor vector (containing standardized features of environmental complexity and obstacle risk level, with values ​​ranging from [0,1]). P represents the user preference weight vector (output by S8's personalized model, including simplicity coefficient, sentiment feedback coefficient, etc.). ω1,ω 2, ω3 represents the dynamically adjusted weight matrix (the initial values ​​are learned from the interactive corpus through transfer learning and corrected in real time by the S7 evaluation data). ε represents a random perturbation term (controlling narrative text diversity, with values ​​following a normal distribution N(0,0.02)); The priority sorting formula for the precautions is as follows: ; In the formula: Ri represents the broadcast priority of the i-th note (normalized value, ranging from [0,1]); C i Indicates the severity of the risk consequences (based on pre-calculation of physical collision simulation, such as "step edge" corresponding to Ci=0.8, "carpet wrinkle" corresponding to Ci=0.3); D i Indicates the spatial distance from the task path (unit: meters, calculated using Euclidean distance from S1 map coordinates); n represents the total number of notices detected in the current task path (dynamically selected, with a maximum of 5).

[0018] In a preferred embodiment, in step S5, an improved A is used. The algorithm performs global path planning, with the heuristic function weight factor set to 1.2 to balance path length and smoothness. Local obstacle avoidance uses the DWA algorithm with a sampling period of 0.1 seconds. It receives dynamic obstacle velocity vectors transmitted from S6 in real time, with an update frequency of 20Hz. The wheel speed is adjusted by a PID controller, with an output range of -1.5 to +1.5 rad / s and a steering angle range of -30 to +30 degrees. During execution, trajectory correction is performed through encoder feedback, with a position error threshold of ±0.05 meters. The task execution status is synchronized to the feedback scheduling module of S6 in the form of a pose matrix.

[0019] In a preferred embodiment, in step S6, the base layer generates a progress report every 5 seconds, the text-to-speech uses the WaveNet model, and the speech rate is 120 words / minute. The advanced layer controls the color gradient of the RGB ring light strip according to the task completion rate of 0% to 100%, with 0% corresponding to blue, 50% to cyan, and 90% to green. The enhancement layer triggers joint micro-motion at key nodes such as path change and target locking, with the servo rotation angle being ±8 degrees and lasting for 300ms. The priority of the feedback content is dynamically calibrated through the evaluation data in S7, and the weight update cycle is 1 hour. In abnormal conditions, it automatically switches to the sound and light composite alarm mode.

[0020] In a preferred embodiment, in step S7, after the task completion signal is triggered, a dual-modal confirmation process is initiated after a 2-second delay: the visual end displays a summary of the task results on an OLED screen with a resolution of 128×64 pixels; the voice end plays evaluation guidance statements with the volume dynamically adjusted based on ambient noise, ranging from 50-85dB; user feedback is parsed in real time using keyword spotting technology, supporting 10 categories of evaluation terms; the evaluation data is encrypted using SHA-256 hash and stored in a local SQLite database, and is simultaneously pushed to the model optimization module in S8 via the MQTT protocol; if no feedback is received, it is automatically marked as a "neutral" evaluation after 30 seconds.

[0021] In a preferred embodiment, in step S8, an online gradient descent algorithm is deployed with a learning rate of 0.01. The model is updated daily at 3:00 AM. The input layer receives the evaluation data from S7, using nearly 100 sliding windows and interaction logs, including the parsing accuracy of S2, the feedback acceptance rate of S4, and the number of interruptions in S6. The parameters are adjusted through a multi-objective optimization function: the input parsing weight matrix, with each modality weight ranging from 0.1 to 0.9; the narrative style coefficient, with a simplicity value of 0.2 to 0.8; and the feedback frequency factor, which is 0.5 to 2.0 times the baseline value. The optimization results are stored in non-volatile memory and automatically loaded and applied upon the next startup.

[0022] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. In this invention, the narrative style fusion formula dynamically adjusts the response content, directly improving the relevance of the interaction process. The weighted fusion of task type templates, environmental constraint factors, and user preference parameters in the formula allows the robot to flexibly adjust its expression focus according to specific scenarios. For example, it automatically simplifies the instruction structure in complex environments and reduces redundant explanations when the user is familiar with the operation, making the communication rhythm more in line with actual needs. The addition of random perturbation terms avoids mechanical repetition of response content, enhancing the naturalness and novelty of the interaction process, and long-term use can improve the robot's response speed.

[0023] 2. In this invention, the effective communication of key information is ensured by calculating the product of risk consequences and spatial distance. This mechanism allows the robot to automatically filter out the most critical issues in multi-risk scenarios, prioritizing alerts to high-risk obstacles at close range rather than low-impact factors at a distance, thus reducing the information processing burden on users. The normalized priority values ​​ensure the simplicity of the prompts, avoiding distractions caused by information overload. In actual use, a noticeable improvement in the accuracy of potential risk prediction and a reduction in operational error rates can be observed. Attached Figure Description

[0024] Figure 1This is a schematic diagram illustrating the process principle of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0026] Example: Reference Figure 1 , A human-computer interaction method for a mobile robot, the method comprising the following steps: S1: After the robot starts, it automatically scans the surrounding space, builds a basic environmental map and identifies static obstacles, providing spatial context for subsequent interactions. The map data will be synchronized to S2 and S5 in real time.

[0027] S2: Integrates voice commands, body movements, and touch signals, and dynamically adjusts the input priority based on the environmental noise level in S1. The parsing results are transmitted to S3 in real time for intent determination.

[0028] S3: Based on the input signal parsed by S2 and combined with the user's historical interaction habits (from the optimized model of S8), a structured task description is generated, and the task parameters are sent to S4 for feedback confirmation.

[0029] S4: Generate a humanized, scenario-based response based on the task description in S3, prompt relevant precautions for the task and confirm the user's willingness to perform, and simultaneously synchronize the user's confirmation result to S5 to start the execution process.

[0030] S5: Utilizes the environmental map of S1 to plan the optimal movement path, receives obstacle update information from S6 in real time during execution, completes the physical task through wheel speed adjustment and steering control, and provides real-time feedback on the execution status to S6.

[0031] S6: During the execution of S5, status indicators are provided through voice progress announcements, changes in body lights, and micro-movements of joints. The feedback content synchronously affects the timing of evaluation collection in S7.

[0032] S7: After the task is completed, the results are presented proactively and user feedback is collected via voice. The evaluation data will be stored in an encrypted manner and used for model updates in S8.

[0033] S8: Based on user reviews and historical interaction data collected in S7, dynamically adjust the input parsing weights of S2, the narrative style of S4, and the feedback frequency of S6.

[0034] In step S1, after the robot starts, it performs spatial scanning using a 3D LiDAR and a vision sensor array. The scanning range covers 360 degrees horizontally and -15 to +45 degrees vertically, constructing a rasterized environment map with a resolution of 0.05 meters per pixel. Simultaneously, ultrasonic sensors are activated to extract the outlines of static obstacles, with the recognition confidence threshold set to 0.85. The map data is sent to the input parsing module in S2 and the path planning module in S5 via the CAN bus at a frequency of 10Hz. The initialization phase lasts no more than 15 seconds; if the timeout occurs, a low-power scanning mode is triggered.

[0035] In step S2, a dual-channel processing architecture is used to receive voice commands, body movements, and touch signals respectively. The sampling rate of voice commands is 16kHz with a dynamic range of 80dB. Body movements are based on skeletal keypoint detection with a frame rate of 25fps. Touch signals use capacitive sensing and support simultaneous recognition of 10 points. Voice features are extracted using Mel-frequency cepstral coefficients and combined with the ambient noise decibel value transmitted in S1. The sampling interval is 500ms, and the input weight matrix is ​​dynamically adjusted—when the noise value is higher than 65dB, the voice weight is reduced by 30% and the gesture recognition weight is increased. The parsing results are processed by a feature fusion network based on an attention mechanism and then pushed to the intent understanding module in S3 in real time in JSON format.

[0036] In step S3, a sequence labeling model combining bidirectional LSTM and CRF is used. The input layer receives the multimodal feature vector parsed in S2, with a dimension of 256. At the same time, it calls the user's historical interaction records in the local encrypted database, which contain data from the last 90 days. The data is automatically anonymized daily. The task similarity score is generated by calculating cosine similarity, with a threshold set to 0.7. The structured task description includes action type, divided into 6 major categories and 32 sub-items, target attributes, including shape, color, and material triples and spatial coordinates, and an absolute coordinate system based on the map in S1. The task parameters are encapsulated using the protobuf protocol and sent to the feedback generation unit in S4.

[0037] In step S4, the core actions, target objects, and spatial attributes are first extracted by the task semantic parsing module. Simultaneously, a dynamic environmental database (linking the static map from S1 with real-time obstacle data from S5) is invoked to generate contextual constraint factors. Subsequently, the narrative style decision layer combines the personalized model parameters from S8 (including user preference coefficients for interaction simplicity and historical emotional feedback weights) to match the basic framework from a predefined narrative template library. The precaution extraction submodule scans potential risk points in the task path (such as dynamic obstacle density and ground flatness thresholds) using a rule engine and converts them into natural language warning elements. Finally, the response generator employs a multi-dimensional fusion algorithm to integrate the core task instructions, contextual constraint factors, precautions, and user preference parameters into coherent text, synchronously triggering an execution confirmation request. The user confirmation signal is transmitted in real-time to the execution queue initiator in S5 via a state synchronization bus; an unconfirmed state returns to S3 to re-enter the intent correction process.

[0038] The formula for fusing narrative styles is: ; In the formula: S represents the final generated narrative text vector (dimension = 512, based on the BERT pre-trained word embedding space). T represents the basic template vector for the task type (such as the predefined templates for "grab" and "navigate", which are determined by the S3 task description). E represents the context constraint factor vector (containing standardized features of environmental complexity and obstacle risk level, with values ​​ranging from [0,1]). P represents the user preference weight vector (output by S8's personalized model, including simplicity coefficient, sentiment feedback coefficient, etc.). ω1,ω 2, ω3 represents the dynamically adjusted weight matrix (initial values ​​are learned from the interactive corpus through transfer learning and corrected in real time by the S7 evaluation data). ε represents a random perturbation term (controlling the diversity of narrative text, with values ​​following a normal distribution N(0,0.02)). The priority sorting formula for the precautions is: ; In the formula: Ri represents the broadcast priority of the i-th note (normalized value, ranging from [0,1]). C i Indicates the severity of the risk consequences (based on pre-calculation of physical collision simulation, such as "step edge" corresponding to Ci=0.8, "carpet wrinkle" corresponding to Ci=0.3). D i Indicates the spatial distance from the task path (unit: meters, calculated using Euclidean distance from S1 map coordinates). n represents the total number of notices detected in the current task path (dynamically selected, with a maximum of 5).

[0039] In step S5, improved A is adopted. The algorithm performs global path planning, with the heuristic function weight factor set to 1.2 to balance path length and smoothness. Local obstacle avoidance uses the DWA algorithm with a sampling period of 0.1 seconds. It receives dynamic obstacle velocity vectors transmitted from S6 in real time, with an update frequency of 20Hz. The wheel speed is adjusted by a PID controller, with an output range of -1.5 to +1.5 rad / s and a steering angle range of -30 to +30 degrees. During execution, trajectory correction is performed through encoder feedback, with a position error threshold of ±0.05 meters. The task execution status is synchronized to the feedback scheduling module of S6 in the form of a pose matrix.

[0040] In step S6, the base layer generates a progress report every 5 seconds. The text-to-speech conversion uses the WaveNet model with a speech rate of 120 words / minute. The advanced layer controls the color gradient of the RGB ring light strip according to the task completion rate of 0% to 100%, with 0% corresponding to blue, 50% to cyan, and 90% to green. The enhancement layer triggers joint micro-movements at key nodes such as path changes and target locking, with the servo motor rotating at ±8 degrees for 300ms. The priority of the feedback content is dynamically calibrated using the evaluation data from S7, and the weight update cycle is 1 hour. In abnormal conditions, it automatically switches to the sound and light composite alarm mode.

[0041] In step S7, after the task completion signal is triggered, a dual-modal confirmation process is initiated after a 2-second delay: the visual end displays a summary of the task results on an OLED screen with a resolution of 128×64 pixels; the voice end plays evaluation guidance statements with the volume dynamically adjusted based on ambient noise, ranging from 50-85dB; user feedback is parsed in real time using keyword spotting technology, supporting 10 categories of evaluation terms; the evaluation data is encrypted using SHA-256 hash and stored in a local SQLite database, and is simultaneously pushed to the model optimization module in S8 via the MQTT protocol; if no feedback is received, it is automatically marked as a "neutral" evaluation after 30 seconds.

[0042] In step S8, an online gradient descent algorithm is deployed with a learning rate of 0.01. The model is updated daily at 3:00 AM. The input layer receives the evaluation data from S7, using nearly 100 sliding windows and interaction logs, including the parsing accuracy of S2, the feedback acceptance rate of S4, and the number of interruptions in S6. The parameters are adjusted through a multi-objective optimization function: the input parsing weight matrix, with the weight of each modality ranging from 0.1 to 0.9; the narrative style coefficient, with a simplicity of 0.2 to 0.8; and the feedback frequency factor, which is 0.5 to 2.0 times the baseline value. The optimization results are stored in non-volatile memory and automatically loaded and applied upon the next startup.

[0043] A human-computer interaction system for a mobile robot, which is applied to the aforementioned human-computer interaction method for the mobile robot.

[0044] From the above, we can conclude that: In this invention, the dynamic adjustment of response content through a narrative style fusion formula directly enhances the relevance of the interaction process. The weighted fusion of task type templates, environmental constraint factors, and user preference parameters in the formula allows the robot to flexibly adjust its expression focus according to specific scenarios. For example, it automatically simplifies instruction structures in complex environments and reduces redundant explanations when users are familiar with the operation, making the communication rhythm more in line with actual needs. The addition of random perturbation terms avoids mechanical repetition of response content, enhancing the naturalness and novelty of the interaction process, and improving the robot's response speed with long-term use.

[0045] In this invention, the effective communication of key information is ensured by calculating the product of risk consequences and spatial distance. This mechanism allows the robot to automatically filter out the most critical issues in multi-risk scenarios, prioritizing alerts to high-risk obstacles at close range rather than low-impact factors at a distance, thus reducing the information processing burden on users. The normalized priority values ​​ensure the simplicity of the prompts, avoiding distractions caused by information overload. In practical use, a noticeable improvement in the accuracy of potential risk prediction and a reduction in operational error rates are observed.

[0046] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0047] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A human-computer interaction method for a mobile robot, characterized in that: The method includes the following steps: S1: After the robot starts up, it automatically scans the surrounding space, builds a basic environmental map and identifies static obstacles, providing spatial context for subsequent interactions. The map data will be synchronized to S2 and S5 in real time. S2: Integrates voice commands, body movements and touch signals, and dynamically adjusts the input priority based on the environmental noise level in S1. The parsing results are transmitted to S3 in real time for intent judgment. S3: Based on the input signal parsed by S2 and combined with the user's historical interaction habits, generate a structured task description and send the task parameters to S4 for feedback confirmation; S4: Generate a humanized, scenario-based response based on the task description in S3, prompt relevant precautions for the task and confirm the user's willingness to perform, and simultaneously synchronize the user's confirmation result to S5 to start the execution process; S5: Utilizes the environmental map of S1 to plan the optimal movement path, receives obstacle update information from S6 in real time during execution, completes the physical task through wheel speed adjustment and steering control, and provides real-time feedback on the execution status to S6. S6: During the execution of S5, status indicators are provided by voice progress announcements, changes in body lights, and micro-movements of joints. The feedback content synchronously affects the timing of evaluation collection in S7. S7: After the task is completed, the results are presented proactively and user feedback is collected via voice. The evaluation data will be stored in an encrypted manner and used for model updates in S8. S8: Based on user reviews and historical interaction data collected in S7, dynamically adjust the input parsing weights of S2, the narrative style of S4, and the feedback frequency of S6.

2. The human-computer interaction method for a mobile robot as described in claim 1, characterized in that: In step S1, after the robot starts, it performs spatial scanning using a 3D LiDAR and a vision sensor array. The scanning range covers 360 degrees horizontally and -15 to +45 degrees vertically, constructing a rasterized environment map with a resolution of 0.05 meters per pixel. Simultaneously, an ultrasonic sensor is activated to extract the outlines of static obstacles, with the recognition confidence threshold set to 0.

85. The map data is sent to the input parsing module in S2 and the path planning module in S5 via the CAN bus at a frequency of 10 Hz. The initialization phase lasts no more than 15 seconds; if the timeout occurs, a low-power scanning mode is triggered.

3. The human-computer interaction method for a mobile robot as described in claim 1, characterized in that: In step S2, a dual-channel processing architecture is used to receive voice commands, body movements, and touch signals respectively. The sampling rate of voice commands is 16kHz with a dynamic range of 80dB. Body movements are based on skeletal keypoint detection with a frame rate of 25fps. Touch signals use capacitive sensing and support simultaneous recognition of 10 points. Voice features are extracted using Mel-frequency cepstral coefficients and combined with the ambient noise decibel value transmitted in S1. The sampling interval is 500ms, and the input weight matrix is ​​dynamically adjusted—when the noise value is higher than 65dB, the voice weight is reduced by 30% and the gesture recognition weight is increased. The parsing results are processed by a feature fusion network based on an attention mechanism and then pushed to the intent understanding module in S3 in real time in JSON format.

4. The human-computer interaction method for a mobile robot as described in claim 1, characterized in that: In step S3, a sequence labeling model combining bidirectional LSTM and CRF is used. The input layer receives the multimodal feature vector parsed in S2, with a dimension of 256. At the same time, it calls the user's historical interaction records in the local encrypted database, which include data from the last 90 days. The data is automatically anonymized daily. The task similarity score is generated by calculating cosine similarity, with a threshold set to 0.

7. The structured task description includes action type, divided into 6 major categories and 32 sub-items, target attributes, including shape, color, and material triples and spatial coordinates, and an absolute coordinate system based on the map in S1. The task parameters are encapsulated using the protobuf protocol and sent to the feedback generation unit in S4.

5. The human-computer interaction method for a mobile robot as described in claim 1, characterized in that: In step S4, the core actions, target objects, and spatial attributes are first extracted through the task semantic parsing module, while the environment dynamic database is called to generate context constraint factors. Subsequently, the narrative style decision layer combines the personalized model parameters of S8 to match the basic framework from the predefined narrative template library. The precaution extraction submodule scans the potential risk points in the task path through the rule engine and converts them into natural language warning elements.

6. The human-computer interaction method for a mobile robot as described in claim 1, characterized in that: In step S5, an improved A is adopted. The algorithm performs global path planning, with the heuristic function weight factor set to 1.2 to balance path length and smoothness. Local obstacle avoidance uses the DWA algorithm with a sampling period of 0.1 seconds. It receives dynamic obstacle velocity vectors transmitted from S6 in real time, with an update frequency of 20Hz. The wheel speed is adjusted by a PID controller, with an output range of -1.5 to +1.5 rad / s and a steering angle range of -30 to +30 degrees.

7. The human-computer interaction method for a mobile robot as described in claim 1, characterized in that: In step S6, the base layer generates a progress report every 5 seconds. The text-to-speech conversion uses the WaveNet model with a speech rate of 120 words / minute. The advanced layer controls the color gradient of the RGB ring light strip according to the task completion rate of 0% to 100%, with 0% corresponding to blue, 50% to cyan, and 90% to green.

8. The human-computer interaction method for a mobile robot as described in claim 1, characterized in that: In step S7, after the task completion signal is triggered, a dual-modal confirmation process is initiated after a 2-second delay: the visual end displays a summary of the task results on an OLED screen with a resolution of 128×64 pixels; the voice end plays evaluation guidance statements with the volume dynamically adjusted based on ambient noise, ranging from 50-85dB; user feedback is parsed in real time using keyword spotting technology, supporting 10 categories of evaluation terms; the evaluation data is encrypted using SHA-256 hash and stored in a local SQLite database, and is simultaneously pushed to the model optimization module in S8 via the MQTT protocol; if no feedback is received, it is automatically marked as a "neutral" evaluation after 30 seconds.

9. The human-computer interaction method for a mobile robot as described in claim 1, characterized in that: In step S8, an online gradient descent algorithm is deployed with a learning rate of 0.

01. The model is updated daily at 3:00 AM. The input layer receives the evaluation data from S7 and uses nearly 100 sliding windows and interaction logs, including the parsing accuracy of S2, the feedback acceptance rate of S4, and the number of interruptions in S6. The parameters are adjusted through a multi-objective optimization function.

10. A human-computer interaction system for a mobile robot, characterized in that: The system is applied to the human-computer interaction method for mobile robots as described in any one of claims 1 to 9.

Images