Multi-scene self-adaptive guide robot interaction method and system

Abstract

The invention provides a multi-scene self-adaptive guide robot interaction method and system in the technical field of guide robots. The method comprises the steps that S1, a guide robot obtains interaction voice and converts the interaction voice into an interaction text; s2, analyzing the interaction text to generate a reply text, converting the reply text into a reply voice, and playing the reply voice for interaction; s3, managing the dialogue content in the interaction process; s4, the guide robot moves based on the guide path in the interaction process, and automatically plays explanation voice when arriving at the guide point; step S5, when the interaction text carries the target point, planning an advancing path to the target point through a bidirectional progressive optimal search algorithm; and S6, displacement is carried out based on the advancing path, and operation information of the robot is displayed in real time. The method has the advantages that the efficiency and accuracy of path planning, the flexibility of voice interaction and the expansibility of the system are greatly improved, and then the user experience of guide robot interaction is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of tour guide robots, and in particular to a multi-scene adaptive tour guide robot interaction method and system. Background Art

[0002] Guide robots are intelligent devices that combine artificial intelligence, robotics, and sensor technology. They are primarily used in museums, exhibition halls, hospitals, airports, shopping malls, and other places to provide visitors with guidance, explanations, and consulting services. However, existing guide robots have the following drawbacks in practical applications:

[0003] 1. In terms of path planning, the algorithms of existing tour guide robots are unable to quickly and accurately plan the optimal path, and the smoothness and adaptability of the path need to be improved; 2. In terms of human-computer interaction, tour guide robots lack sufficient flexibility when facing individual tourists with different ages, educational levels, language habits, interests and hobbies, and find it difficult to meet the personalized interaction needs of users in different scenarios. The interactive experience is not intelligent and humane enough; 3. In terms of system integration and scalability, the lack of modular design leads to poor system maintainability and scalability, making it difficult to upgrade and optimize functions according to actual needs.

[0004] Therefore, how to provide a multi-scenario adaptive tour guide robot interaction method and system to improve the efficiency and accuracy of path planning, the flexibility of voice interaction, and the scalability of the system, thereby improving the user experience of the tour guide robot interaction, has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a multi-scenario adaptive guide robot interaction method and system, so as to improve the efficiency and accuracy of path planning, the flexibility of voice interaction, and the scalability of the system, thereby improving the user experience of the guide robot interaction.

[0006] In a first aspect, the present invention provides a multi-scenario adaptive navigation robot interaction method, comprising the following steps:

[0007] Step S1: The tour guide robot calls the audio pickup through the audio_capture node in ROS to obtain interactive voice, reduces noise on the interactive voice, and converts it into interactive text;

[0008] Step S2: The tour guide robot parses the interaction text using the Coqui TTS engine to generate a corresponding reply text, converts the reply text into a reply voice, and calls the speaker to play the reply voice through the soundPlay node in ROS to interact;

[0009] Step S3: During the interaction process, the navigation robot manages the conversation content including the interaction text and the reply text through a multi-round conversation engine based on a finite state machine;

[0010] Step S4: During the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path;

[0011] Step S5: When the interactive text carries a target point, a path to the target point is planned using a bidirectional progressive optimal search algorithm, an ellipse heuristic sampling algorithm, and a cubic B-spline curve interpolation algorithm;

[0012] Step S6: The navigation robot moves based on the travel path, displays the robot operation information in real time through the display screen, and records and stores the robot operation log in real time.

[0013] Furthermore, the step S1 is specifically as follows:

[0014] The tour guide robot calls the audio pickup through the audio_capture node in ROS to obtain interactive voice in real time. After reducing the noise of the interactive voice through the noise suppression algorithm, the interactive voice is converted into interactive text through the DNN recognition engine of iFlytek SDK.

[0015] Furthermore, in step S2, during the playback of the reply voice, the speech speed, voice rhythm and pitch are dynamically adjusted based on the interaction context.

[0016] Furthermore, the step S3 is specifically as follows:

[0017] During the interaction process, the tour guide robot uses a multi-round dialogue engine based on a finite state machine to call the smach package under ROS to manage the dialogue content including the interaction text and reply text, and tracks the dialogue status of the dialogue content including at least greetings, route queries and exhibit explanations.

[0018] Furthermore, the step S4 is specifically as follows:

[0019] During the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path. After the explanation voice is played, it determines whether the current guide point is the last one. If not, it continues to move to the next guide point.

[0020] The step S5 is specifically as follows:

[0021] When the interactive text carries a target point, an initial path to the target point is searched using a forward tree and a backward tree constructed using a bidirectional progressive optimal search algorithm. The initial path is converted to a sampling domain using an ellipse heuristic sampling algorithm. Collision checking is performed using a K-DTree data structure and Bresenham. The initial path is then smoothed using a cubic B-spline curve interpolation algorithm to obtain a travel path.

[0022] The step S6 is specifically as follows:

[0023] The navigation robot moves based on the travel path and displays the robot operation information including at least the robot posture, travel path, dynamic obstacle point cloud and point of interest marks on the display screen in real time through the three-dimensional monitoring interface of the RViz plug-in, and records and stores the robot operation log in real time.

[0024] In a second aspect, the present invention provides a multi-scenario adaptive navigation robot interaction system, comprising the following modules:

[0025] The interactive voice acquisition module is used by the tour guide robot to call the audio pickup through the audio_capture node in ROS to obtain interactive voice, reduce noise on the interactive voice, and convert it into interactive text;

[0026] The reply voice playback module is used for the tour guide robot to parse the interaction text through the Coqui TTS engine to generate the corresponding reply text, convert the reply text into reply voice, and call the speaker to play the reply voice through the soundPlay node in ROS for interaction;

[0027] A conversation content management module is used to manage the conversation content including the interaction text and reply text during the interaction process of the navigation robot through a multi-round conversation engine based on a finite state machine;

[0028] The explanation voice playback module is used for the guide robot to move based on the preset guide path during the interaction process, and automatically play the preset explanation voice when it reaches the guide point on the guide path;

[0029] A travel path planning module is used to plan a travel path to reach a target point by using a bidirectional progressive optimal search algorithm, an ellipse heuristic sampling algorithm, and a cubic B-spline curve interpolation algorithm when the interactive text carries a target point;

[0030] The robot operation information display module is used to guide the robot to move based on the travel path, display the robot operation information in real time through the display screen, and record and store the robot operation log in real time.

[0031] Furthermore, the interactive voice acquisition module is specifically used to:

[0032] The tour guide robot calls the audio pickup through the audio_capture node in ROS to obtain interactive voice in real time. After reducing the noise of the interactive voice through the noise suppression algorithm, the interactive voice is converted into interactive text through the DNN recognition engine of iFlytek SDK.

[0033] Furthermore, in the reply voice playing module, during the playing of the reply voice, the speaking speed, voice rhythm and pitch are dynamically adjusted based on the interaction context.

[0034] Furthermore, the conversation content management module is specifically used to:

[0035] During the interaction process, the tour guide robot uses a multi-round dialogue engine based on a finite state machine to call the smach package under ROS to manage the dialogue content including the interaction text and reply text, and tracks the dialogue status of the dialogue content including at least greetings, route queries and exhibit explanations.

[0036] Furthermore, the explanation voice playback module is specifically used to:

[0037] During the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path. After the explanation voice is played, it determines whether the current guide point is the last one. If not, it continues to move to the next guide point.

[0038] The travel path planning module is specifically used to:

[0039] When the interactive text carries a target point, an initial path to the target point is searched using a forward tree and a backward tree constructed using a bidirectional progressive optimal search algorithm. The initial path is converted to a sampling domain using an ellipse heuristic sampling algorithm. Collision checking is performed using a K-DTree data structure and Bresenham. The initial path is then smoothed using a cubic B-spline curve interpolation algorithm to obtain a travel path.

[0040] The robot operation information display module is specifically used for:

[0041] The navigation robot moves based on the travel path and displays the robot operation information including at least the robot posture, travel path, dynamic obstacle point cloud and point of interest marks on the display screen in real time through the three-dimensional monitoring interface of the RViz plug-in, and records and stores the robot operation log in real time.

[0042] The advantages of the present invention are:

[0043] 1. Call the audio pickup through the audio_capture node in ROS to obtain the interactive voice, reduce the noise of the interactive voice and convert it into interactive text; parse the interactive text through the Coqui TTS engine to generate the corresponding reply text, convert the reply text into reply voice, and call the speaker through the soundPlay node in ROS to play the reply voice for interaction; during the interaction process, the guide robot manages the conversation content including the interactive text and reply text through a multi-round dialogue engine based on a finite state machine; during the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path. When the interactive text carries the target point, the bidirectional progressive optimal search algorithm, the ellipse heuristic sampling algorithm and the cubic B-spline curve interpolation algorithm are used to plan the path to the target point, and the robot moves based on the path and plays the preset explanation voice through The display screen shows the robot's operation information in real time, and records and stores the robot's operation log in real time; that is, in the process of moving path planning, the bidirectional progressive optimal search algorithm, the ellipse heuristic sampling algorithm and the cubic B-spline curve interpolation algorithm are combined to improve the efficiency and accuracy of planning; in the process of converting interactive voice into interactive text, the DNN recognition engine of iFlytek SDK is combined to effectively improve the conversion accuracy and fully understand user needs; and the functional modules such as path planning and voice interaction are designed in blocks and integrated based on the ROS framework to facilitate later functional upgrades and expansions, ultimately greatly improving the efficiency and accuracy of path planning, the flexibility of voice interaction, and the scalability of the system, thereby greatly improving the user experience of the guide robot interaction.

[0044] 2. By combining the ROS audio_capture node with the iFlytek DNN recognition engine, full-link automation of voice acquisition, noise reduction, and recognition is achieved, improving the robustness of voice recognition in complex environments.

[0045] 3. By adopting dynamic TTS adjustment (speech rate, pitch, rhythm) technology, the Coqui TTS engine achieves context-adaptive speech output, enhancing the naturalness of interaction and outperforming traditional fixed-parameter speech synthesis solutions.

[0046] 4. Through the node-based design of ROS (such as soundPlay), the real-time performance of voice input and output is ensured and decoupled from the modules, supporting fast response in high-concurrency scenarios.

[0047] 5. By building a state machine through the ROS smach package, accurate state switching is performed for various types of dialogue intents such as greetings, route queries, and exhibit explanations, solving the logical confusion problem caused by context loss in traditional dialogue systems.

[0048] 6. Through a bidirectional progressive optimal search algorithm, that is, through parallel search of forward and backward trees, the path planning time in large-scale scenarios is significantly shortened; through elliptical heuristic sampling and collision verification, combined with K-DTree and Bresenham algorithms, the sampling space is dynamically compressed and obstacles are avoided to ensure path safety; through cubic B-spline curve interpolation, a smooth and executable robot motion trajectory is generated, reducing mechanical losses and surpassing the bump problem of traditional linear interpolation.

[0049] 7. Through the hybrid mode of preset paths and dynamic target point planning, it supports seamless switching between fixed navigation processes and user-defined paths to meet the needs of multiple scenarios.

[0050] 8. The 3D monitoring interface based on the RViz plug-in displays key information such as posture, obstacle point cloud, etc. in real time, improving the operation and maintenance personnel's global perception of the robot status.

[0051] 9. Through multi-layer adaptive design of dialogue management, path planning, and voice interaction, it can be expanded to complex indoor scenes such as museums, exhibition halls, and airports, and has universal applicability for technology migration.

[0052] 10. Through algorithm fusion (such as FSM+bidirectional search+elliptical sampling), modular design (ROS nodeization) and dynamic adjustment technology (TTS parameters, path smoothing), the pain points of traditional guide robots such as rigid interaction, low path planning efficiency, and poor adaptability to multiple scenarios are systematically solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] Figure 1 This is a flow chart of a multi-scene adaptive navigation robot interaction method of the present invention.

[0055] Figure 2 It is a structural schematic diagram of a multi-scene adaptive navigation robot interaction system of the present invention. DETAILED DESCRIPTION

[0056] The technical solution in the embodiments of the present application has the following overall idea: in the process of path planning, a bidirectional progressive optimal search algorithm, an ellipse heuristic sampling algorithm, and a cubic B-spline curve interpolation algorithm are combined to improve the efficiency and accuracy of planning; in the process of converting interactive voice into interactive text, the DNN recognition engine of iFLYTEK SDK is combined to effectively improve the conversion accuracy and fully understand user needs; and the functional modules such as path planning and voice interaction are designed in blocks and integrated based on the ROS framework to facilitate later functional upgrades and expansions, so as to improve the efficiency and accuracy of path planning, the flexibility of voice interaction, and the scalability of the system, thereby improving the user experience of the tour guide robot interaction.

[0057] Please refer to Figures 1 to 2 As shown, a preferred embodiment of the multi-scenario adaptive navigation robot interaction method of the present invention includes the following steps:

[0058] Step S1: The tour guide robot calls the audio pickup through the audio_capture node in ROS to obtain interactive voice, reduces noise on the interactive voice, and converts it into interactive text;

[0059] Step S2: The tour guide robot parses the interaction text using the Coqui TTS engine to generate a corresponding reply text, converts the reply text into a reply voice (e.g., synthesizing natural and fluent voice using the Tacotron2 and WaveGlow models), and calls the speaker through the soundPlay node in ROS to play the reply voice for interaction;

[0060] Through the node-based design of ROS (such as soundPlay), the real-time performance of voice input and output is ensured and decoupled from the modules, supporting fast response in high-concurrency scenarios.

[0061] Step S3: During the interaction process, the navigation robot manages the conversation content including the interaction text and the reply text through a multi-round conversation engine based on a finite state machine;

[0062] Step S4: During the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path;

[0063] Step S5: When the interactive text carries a target point, a path to the target point is planned using a bidirectional progressive optimal search algorithm, an ellipse heuristic sampling algorithm, and a cubic B-spline curve interpolation algorithm;

[0064] During specific implementation, a timeout mechanism with dual threshold detection is set. The first-level timeout reminds the user to input voice, and the second-level timeout enters the default navigation path to ensure the fault tolerance and efficiency of the system.

[0065] Step S6: The navigation robot moves based on the travel path, displays the robot operation information in real time through the display screen, and records and stores the robot operation log in real time.

[0066] This paper proposes a hybrid heuristic RRT-Connect algorithm to optimize the path planning performance of the tour guide robot, enabling it to quickly and accurately plan a smooth and safe path in complex dynamic environments. A voice interaction system based on iFlytek SDK and Coqui TTS API is constructed to improve speech recognition accuracy and the naturalness of speech synthesis, enhance multi-language and multi-timbre support, and meet the diverse interaction needs of users in different scenarios. At the same time, a modular system design architecture is adopted to design functional modules such as path planning and voice interaction in blocks. These modules are integrated based on the ROS (Robot Operating System) framework to construct a complete, scalable and maintainable tour guide robot system, facilitating future functional upgrades and expansions.

[0067] Through multi-layer adaptive design of dialogue management, path planning, and voice interaction, it can be expanded to complex indoor scenes such as museums, exhibition halls, and airports, and has universal applicability in technology migration.

[0068] Through algorithm fusion (such as FSM+bidirectional search+elliptical sampling), modular design (ROS nodeization) and dynamic adjustment technology (TTS parameters, path smoothing), the pain points of traditional guide robots such as rigid interaction, low path planning efficiency, and poor adaptability to multiple scenarios are systematically solved.

[0069] The step S1 is specifically as follows:

[0070] The tour guide robot calls the audio pickup through the audio_capture node in ROS to obtain interactive voice in real time. After reducing the noise of the interactive voice through a noise suppression algorithm (such as the WebRTC NS module), the interactive voice is converted into interactive text through the DNN recognition engine of the iFlytek SDK.

[0071] By combining the ROS audio_capture node with the iFlytek DNN recognition engine, full-link automation of voice acquisition, noise reduction, and recognition is achieved, improving the robustness of voice recognition in complex environments.

[0072] In step S2, during the playback of the reply voice, the speech speed, voice rhythm and pitch are dynamically adjusted based on the interactive context to enhance the authority and emotionality of the voice feedback.

[0073] By adopting dynamic TTS adjustment (speech rate, pitch, rhythm) technology and using the Coqui TTS engine to achieve context-adaptive speech output, the naturalness of interaction is enhanced, which is superior to traditional fixed-parameter speech synthesis solutions.

[0074] The step S3 is specifically as follows:

[0075] During the interaction process, the guide robot uses a multi-turn dialogue engine based on a finite state machine (FSM) to call the smach package under ROS to manage the conversation content, including the interaction text and reply text. It also tracks the conversation status of at least the greeting, route query, and exhibit explanation. It also sets a state switching delay and an "interrupt wake-up" function to ensure the continuity and fluency of the conversation.

[0076] By building a state machine through the ROS smach package, accurate state switching is performed for various types of dialogue intentions such as greetings, route queries, and exhibit explanations, solving the logical confusion problem caused by context loss in traditional dialogue systems.

[0077] The step S4 is specifically as follows:

[0078] During the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path. After the explanation voice is played, it determines whether the current guide point is the last one. If not, it continues to move to the next guide point.

[0079] The step S5 is specifically as follows:

[0080] When the interactive text carries a target point, a forward tree and a backward tree constructed by a bidirectional progressive optimal search algorithm are used to search for an initial path to the target point. A dynamic balancing strategy is used to exchange tree expansion priorities. An elliptical heuristic sampling algorithm is used to convert the sampling domain of the initial path. A rejection sampling method is used to ensure the validity of the sampling points. A K-DTree data structure and Bresenham are combined for collision verification to improve verification efficiency. The initial path is then smoothed using a cubic B-spline curve interpolation algorithm to obtain a travel path. That is, sharp turns or unnecessary jumps in the initial path are removed to improve the smoothness of the movement.

[0081] Through a bidirectional progressive optimal search algorithm, that is, parallel search of the forward tree and the backward tree, the path planning time in large-scale scenarios is significantly shortened; through elliptical heuristic sampling and collision verification, combined with the K-DTree and Bresenham algorithms, the sampling space is dynamically compressed and obstacles are avoided to ensure path safety; through cubic B-spline curve interpolation, a smooth and executable robot motion trajectory is generated, reducing mechanical losses and surpassing the bumpy problem of traditional linear interpolation.

[0082] Through a hybrid mode of preset paths and dynamic target point planning, it supports seamless switching between fixed navigation processes and user-defined paths to meet the needs of multiple scenarios.

[0083] Path planning is based on a decision-making layer, a control layer, and an execution layer. The decision-making layer integrates the Hybrid InformedRRT*-Connect algorithm, publishes the target pose through the MoveBase node of ROS, and receives environmental update data. The control layer uses an adaptive PID controller for precise trajectory tracking and designs a priority queue strategy to deal with dynamic obstacle interference. The execution layer uses ROS to drive onboard hardware devices and adopts a distributed computing architecture to ensure system stability.

[0084] The step S6 is specifically as follows:

[0085] The navigation robot moves based on the travel path and displays the robot operation information including at least the robot posture, travel path, dynamic obstacle point cloud and point of interest marks on the display screen in real time through the three-dimensional monitoring interface of the RViz plug-in, and records and stores the robot operation log in real time.

[0086] The robot operation log uses the ROS rosbag tool to record the complete interaction process, including the original waveform of the interactive voice, the mapping relationship between the interactive text and the reply voice, the path planning time series data, etc., to facilitate users to check and tune the system operation.

[0087] The RViz plug-in-based 3D monitoring interface displays key information such as position and obstacle point clouds in real time, enhancing operators' global awareness of the robot's status. In practice, a status diagram can be added to the RViz visualization interface, displaying abnormal states such as normal, partial path replanning, and navigation failure in different colors. Furthermore, voice alerts of varying severity can be played via text-to-speech (TTS) to enhance system transparency and operability.

[0088] During specific implementation, in a simulation environment, the Gazebo debugging interface is used to provide multiple perspectives for observing the robot's operation, support manual addition of sudden obstacle events, and test the system's real-time response and obstacle avoidance capabilities in complex environments.

[0089] A preferred embodiment of the multi-scenario adaptive navigation robot interactive system of the present invention includes the following modules:

[0090] The interactive voice acquisition module is used by the tour guide robot to call the audio pickup through the audio_capture node in ROS to obtain interactive voice, reduce noise on the interactive voice, and convert it into interactive text;

[0091] The reply voice playback module is used for the tour guide robot to parse the interaction text through the Coqui TTS engine to generate the corresponding reply text, convert the reply text into reply voice (for example, using the Tacotron2 and WaveGlow models to synthesize natural and fluent voice), and call the speaker to play the reply voice through the soundPlay node in ROS for interaction;

[0092] Through the node-based design of ROS (such as soundPlay), the real-time performance of voice input and output is ensured and decoupled from the modules, supporting fast response in high-concurrency scenarios.

[0093] A conversation content management module is used to manage the conversation content including the interaction text and reply text during the interaction process of the navigation robot through a multi-round conversation engine based on a finite state machine;

[0094] The explanation voice playback module is used for the guide robot to move based on the preset guide path during the interaction process, and automatically play the preset explanation voice when it reaches the guide point on the guide path;

[0095] A travel path planning module is used to plan a travel path to reach a target point by using a bidirectional progressive optimal search algorithm, an ellipse heuristic sampling algorithm, and a cubic B-spline curve interpolation algorithm when the interactive text carries a target point;

[0096] During specific implementation, a timeout mechanism with dual threshold detection is set. The first-level timeout reminds the user to input voice, and the second-level timeout enters the default navigation path to ensure the fault tolerance and efficiency of the system.

[0097] The robot operation information display module is used to guide the robot to move based on the travel path, display the robot operation information in real time through the display screen, and record and store the robot operation log in real time.

[0098] This paper proposes a hybrid heuristic RRT-Connect algorithm to optimize the path planning performance of the tour guide robot, enabling it to quickly and accurately plan a smooth and safe path in complex dynamic environments. A voice interaction system based on iFlytek SDK and Coqui TTS API is constructed to improve speech recognition accuracy and the naturalness of speech synthesis, enhance multi-language and multi-timbre support, and meet the diverse interaction needs of users in different scenarios. At the same time, a modular system design architecture is adopted to design functional modules such as path planning and voice interaction in blocks. These modules are integrated based on the ROS (Robot Operating System) framework to construct a complete, scalable and maintainable tour guide robot system, facilitating future functional upgrades and expansions.

[0099] Through multi-layer adaptive design of dialogue management, path planning, and voice interaction, it can be expanded to complex indoor scenes such as museums, exhibition halls, and airports, and has universal applicability in technology migration.

[0100] Through algorithm fusion (such as FSM+bidirectional search+elliptical sampling), modular design (ROS nodeization) and dynamic adjustment technology (TTS parameters, path smoothing), the pain points of traditional guide robots such as rigid interaction, low path planning efficiency, and poor adaptability to multiple scenarios are systematically solved.

[0101] The interactive voice acquisition module is specifically used for:

[0102] The tour guide robot calls the audio pickup through the audio_capture node in ROS to obtain interactive voice in real time. After reducing the noise of the interactive voice through a noise suppression algorithm (such as the WebRTC NS module), the interactive voice is converted into interactive text through the DNN recognition engine of the iFlytek SDK.

[0103] By combining the ROS audio_capture node with the iFlytek DNN recognition engine, full-link automation of voice acquisition, noise reduction, and recognition is achieved, improving the robustness of voice recognition in complex environments.

[0104] In the reply voice playback module, during the playback of the reply voice, the speech speed, voice rhythm and pitch are dynamically adjusted based on the interaction context to enhance the authority and emotionality of the voice feedback.

[0105] By adopting dynamic TTS adjustment (speech rate, pitch, rhythm) technology and using the Coqui TTS engine to achieve context-adaptive speech output, the naturalness of interaction is enhanced, which is superior to traditional fixed-parameter speech synthesis solutions.

[0106] The conversation content management module is specifically used to:

[0107] During the interaction process, the guide robot uses a multi-turn dialogue engine based on a finite state machine (FSM) to call the smach package under ROS to manage the conversation content, including the interaction text and reply text. It also tracks the conversation status of at least the greeting, route query, and exhibit explanation. It also sets a state switching delay and an "interrupt wake-up" function to ensure the continuity and fluency of the conversation.

[0108] By building a state machine through the ROS smach package, accurate state switching is performed for various types of dialogue intentions such as greetings, route queries, and exhibit explanations, solving the logical confusion problem caused by context loss in traditional dialogue systems.

[0109] The explanation voice playback module is specifically used for:

[0110] During the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path. After the explanation voice is played, it determines whether the current guide point is the last one. If not, it continues to move to the next guide point.

[0111] The travel path planning module is specifically used to:

[0112] When the interactive text carries a target point, a forward tree and a backward tree constructed by a bidirectional progressive optimal search algorithm are used to search for an initial path to the target point. A dynamic balancing strategy is used to exchange tree expansion priorities. An elliptical heuristic sampling algorithm is used to convert the sampling domain of the initial path. A rejection sampling method is used to ensure the validity of the sampling points. A K-DTree data structure and Bresenham are combined for collision verification to improve verification efficiency. The initial path is then smoothed using a cubic B-spline curve interpolation algorithm to obtain a travel path. That is, sharp turns or unnecessary jumps in the initial path are removed to improve the smoothness of the movement.

[0113] Through a bidirectional progressive optimal search algorithm, that is, parallel search of the forward tree and the backward tree, the path planning time in large-scale scenarios is significantly shortened; through elliptical heuristic sampling and collision verification, combined with the K-DTree and Bresenham algorithms, the sampling space is dynamically compressed and obstacles are avoided to ensure path safety; through cubic B-spline curve interpolation, a smooth and executable robot motion trajectory is generated, reducing mechanical losses and surpassing the bumpy problem of traditional linear interpolation.

[0114] Through a hybrid mode of preset paths and dynamic target point planning, it supports seamless switching between fixed navigation processes and user-defined paths to meet the needs of multiple scenarios.

[0115] Path planning is based on a decision-making layer, a control layer, and an execution layer. The decision-making layer integrates the Hybrid InformedRRT*-Connect algorithm, publishes the target pose through the MoveBase node of ROS, and receives environmental update data. The control layer uses an adaptive PID controller for precise trajectory tracking and designs a priority queue strategy to deal with dynamic obstacle interference. The execution layer uses ROS to drive onboard hardware devices and adopts a distributed computing architecture to ensure system stability.

[0116] The robot operation information display module is specifically used for:

[0117] The navigation robot moves based on the travel path and displays the robot operation information including at least the robot posture, travel path, dynamic obstacle point cloud and point of interest marks on the display screen in real time through the three-dimensional monitoring interface of the RViz plug-in, and records and stores the robot operation log in real time.

[0118] The robot operation log uses the ROS rosbag tool to record the complete interaction process, including the original waveform of the interactive voice, the mapping relationship between the interactive text and the reply voice, the path planning time series data, etc., to facilitate users to check and tune the system operation.

[0119] The RViz plug-in-based 3D monitoring interface displays key information such as position and obstacle point clouds in real time, enhancing operators' global awareness of the robot's status. In practice, a status diagram can be added to the RViz visualization interface, displaying abnormal states such as normal, partial path replanning, and navigation failure in different colors. Furthermore, voice alerts of varying severity can be played via text-to-speech (TTS) to enhance system transparency and operability.

[0120] During specific implementation, in a simulation environment, the Gazebo debugging interface is used to provide multiple perspectives for observing the robot's operation, support manual addition of sudden obstacle events, and test the system's real-time response and obstacle avoidance capabilities in complex environments.

[0121] In order to facilitate understanding of the present invention, the application scenarios of the present invention are illustrated as follows:

[0122] Application Scenario 1: Guide Robot Application in Museum Scenario

[0123] In museum settings, guide robots must navigate complex exhibition hall layouts and volatile visitor flows. Leveraging the hybrid heuristic RRT-Connect algorithm, the guide robot can quickly plan the optimal path, avoiding obstacles and guiding visitors through the exhibition halls. Through voice interaction, it can provide detailed introductions to exhibits, supporting multiple languages and voice tones to meet the needs of diverse visitors. The modular system design allows the robot's functionality to be expanded and optimized based on the theme and content of different exhibitions, such as adding explanation modules for specific exhibits.

[0124] Application scenario 2: Guide robot application in hospital scenarios

[0125] In hospital scenarios, navigation robots are needed to help patients quickly locate departments and provide information such as registration and treatment procedures. The hybrid heuristic RRT-Connect algorithm dynamically adjusts route planning based on real-time hospital traffic flow, ensuring patients reach their destinations on time. The voice interaction system can connect with the hospital's information system to obtain and update medical information in real time, providing patients with accurate navigation services. Furthermore, the system's fault tolerance and degradation mechanisms ensure stable operation in hospital environments, preventing system failures from impacting the patient experience.

[0126] Application Scenario 3: Guide Robot Application in Nursing Homes

[0127] In nursing homes, guide robots need to consider the mobility and psychological needs of the elderly. The hybrid heuristic RRT-Connect algorithm can plan routes that are suitable for the elderly's walking speed, avoiding sharp turns and steep slopes that are not suitable for elderly people. The voice interaction system uses a gentle and friendly tone to communicate with the elderly, providing information services such as daily care and medical care, and can also provide psychological care and support through simple conversational interactions. The modular system design facilitates functional customization and expansion based on the specific needs of the nursing home, such as adding guidance for elderly entertainment activities.

[0128] In summary, the advantages of the present invention are:

[0129] 1. Call the audio pickup through the audio_capture node in ROS to obtain the interactive voice, reduce the noise of the interactive voice and convert it into interactive text; parse the interactive text through the Coqui TTS engine to generate the corresponding reply text, convert the reply text into reply voice, and call the speaker through the soundPlay node in ROS to play the reply voice for interaction; during the interaction process, the guide robot manages the conversation content including the interactive text and reply text through a multi-round dialogue engine based on a finite state machine; during the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path. When the interactive text carries the target point, the bidirectional progressive optimal search algorithm, the ellipse heuristic sampling algorithm and the cubic B-spline curve interpolation algorithm are used to plan the path to the target point, and the robot moves based on the path and plays the preset explanation voice through The display screen shows the robot's operation information in real time, and records and stores the robot's operation log in real time; that is, in the process of moving path planning, the bidirectional progressive optimal search algorithm, the ellipse heuristic sampling algorithm and the cubic B-spline curve interpolation algorithm are combined to improve the efficiency and accuracy of planning; in the process of converting interactive voice into interactive text, the DNN recognition engine of iFlytek SDK is combined to effectively improve the conversion accuracy and fully understand user needs; and the functional modules such as path planning and voice interaction are designed in blocks and integrated based on the ROS framework to facilitate later functional upgrades and expansions, ultimately greatly improving the efficiency and accuracy of path planning, the flexibility of voice interaction, and the scalability of the system, thereby greatly improving the user experience of the guide robot interaction.

[0130] 2. By combining the ROS audio_capture node with the iFlytek DNN recognition engine, full-link automation of voice acquisition, noise reduction, and recognition is achieved, improving the robustness of voice recognition in complex environments.

[0131] 3. By adopting dynamic TTS adjustment (speech rate, pitch, rhythm) technology, the Coqui TTS engine achieves context-adaptive speech output, enhancing the naturalness of interaction and outperforming traditional fixed-parameter speech synthesis solutions.

[0132] 4. Through the node-based design of ROS (such as soundPlay), the real-time performance of voice input and output is ensured and decoupled from the modules, supporting fast response in high-concurrency scenarios.

[0133] 5. By building a state machine through the ROS smach package, accurate state switching is performed for various types of dialogue intents such as greetings, route queries, and exhibit explanations, solving the logical confusion problem caused by context loss in traditional dialogue systems.

[0134] 6. Through a bidirectional progressive optimal search algorithm, that is, through parallel search of forward and backward trees, the path planning time in large-scale scenarios is significantly shortened; through elliptical heuristic sampling and collision verification, combined with K-DTree and Bresenham algorithms, the sampling space is dynamically compressed and obstacles are avoided to ensure path safety; through cubic B-spline curve interpolation, a smooth and executable robot motion trajectory is generated, reducing mechanical losses and surpassing the bump problem of traditional linear interpolation.

[0135] 7. Through the hybrid mode of preset paths and dynamic target point planning, it supports seamless switching between fixed navigation processes and user-defined paths to meet the needs of multiple scenarios.

[0136] 8. The 3D monitoring interface based on the RViz plug-in displays key information such as posture, obstacle point cloud, etc. in real time, improving the operation and maintenance personnel's global perception of the robot status.

[0137] 9. Through multi-layer adaptive design of dialogue management, path planning, and voice interaction, it can be expanded to complex indoor scenes such as museums, exhibition halls, and airports, and has universal applicability for technology migration.

[0138] 10. Through algorithm fusion (such as FSM+bidirectional search+elliptical sampling), modular design (ROS nodeization) and dynamic adjustment technology (TTS parameters, path smoothing), the pain points of traditional guide robots such as rigid interaction, low path planning efficiency, and poor adaptability to multiple scenarios are systematically solved.

[0139] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A multi-scenario adaptive navigation robot interaction method, characterized by: The steps include: Step S1: The tour guide robot calls the audio pickup through the audio_capture node in ROS to obtain interactive voice, reduces noise on the interactive voice, and converts it into interactive text; Step S2: The tour guide robot parses the interaction text using the Coqui TTS engine to generate a corresponding reply text, converts the reply text into a reply voice, and calls the speaker to play the reply voice through the soundPlay node in ROS to interact; Step S3: During the interaction process, the navigation robot manages the conversation content including the interaction text and the reply text through a multi-round conversation engine based on a finite state machine; Step S4: During the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path; Step S5: When the interactive text carries a target point, a path to the target point is planned using a bidirectional progressive optimal search algorithm, an ellipse heuristic sampling algorithm, and a cubic B-spline curve interpolation algorithm; Step S6: The navigation robot moves based on the travel path, displays the robot operation information in real time through the display screen, and records and stores the robot operation log in real time.

2. The multi-scenario adaptive navigation robot interaction method according to claim 1, characterized in that: The step S1 is specifically as follows: The tour guide robot calls the audio pickup through the audio_capture node in ROS to obtain interactive voice in real time. After reducing the noise of the interactive voice through the noise suppression algorithm, the interactive voice is converted into interactive text through the DNN recognition engine of iFlytek SDK.

3. The multi-scenario adaptive navigation robot interaction method according to claim 1, characterized in that: In step S2, during the playback of the reply voice, the speaking speed, voice rhythm and pitch are dynamically adjusted based on the interaction context.

4. The multi-scenario adaptive navigation robot interaction method according to claim 1, characterized in that: The step S3 is specifically as follows: During the interaction process, the tour guide robot uses a multi-round dialogue engine based on a finite state machine to call the smach package under ROS to manage the dialogue content including the interaction text and reply text, and tracks the dialogue status of the dialogue content including at least greetings, route queries and exhibit explanations.

5. The multi-scenario adaptive navigation robot interaction method according to claim 1, characterized in that: The step S4 is specifically as follows: During the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path. After the explanation voice is played, it determines whether the current guide point is the last one. If not, it continues to move to the next guide point. The step S5 is specifically as follows: When the interactive text carries a target point, an initial path to the target point is searched using a forward tree and a backward tree constructed using a bidirectional progressive optimal search algorithm. The initial path is converted to a sampling domain using an ellipse heuristic sampling algorithm. Collision checking is performed using a K-DTree data structure and Bresenham. The initial path is then smoothed using a cubic B-spline curve interpolation algorithm to obtain a travel path. The step S6 is specifically as follows: The navigation robot moves based on the travel path and displays the robot operation information including at least the robot posture, travel path, dynamic obstacle point cloud and point of interest marks on the display screen in real time through the three-dimensional monitoring interface of the RViz plug-in, and records and stores the robot operation log in real time.

6. A multi-scenario adaptive navigation robot interactive system, characterized by: Includes the following modules: The interactive voice acquisition module is used by the tour guide robot to call the audio pickup through the audio_capture node in ROS to obtain interactive voice, reduce noise on the interactive voice, and convert it into interactive text; The reply voice playback module is used for the tour guide robot to parse the interaction text through the Coqui TTS engine to generate the corresponding reply text, convert the reply text into reply voice, and call the speaker to play the reply voice through the soundPlay node in ROS for interaction; A conversation content management module is used to manage the conversation content including the interaction text and reply text during the interaction process of the navigation robot through a multi-round conversation engine based on a finite state machine; The explanation voice playback module is used for the guide robot to move based on the preset guide path during the interaction process, and automatically play the preset explanation voice when it reaches the guide point on the guide path; A travel path planning module is used to plan a travel path to reach a target point by using a bidirectional progressive optimal search algorithm, an ellipse heuristic sampling algorithm, and a cubic B-spline curve interpolation algorithm when the interactive text carries a target point; The robot operation information display module is used to guide the robot to move based on the travel path, display the robot operation information in real time through the display screen, and record and store the robot operation log in real time.

7. The multi-scenario adaptive navigation robot interactive system according to claim 6, characterized in that: The interactive voice acquisition module is specifically used for: The tour guide robot calls the audio pickup through the audio_capture node in ROS to obtain interactive voice in real time. After reducing the noise of the interactive voice through the noise suppression algorithm, the interactive voice is converted into interactive text through the DNN recognition engine of iFlytek SDK.

8. The multi-scenario adaptive navigation robot interactive system according to claim 6, characterized in that: In the reply voice playing module, during the playing of the reply voice, the speaking speed, voice rhythm and pitch are dynamically adjusted based on the interaction context.

9. The multi-scenario adaptive navigation robot interactive system according to claim 6, characterized in that: The conversation content management module is specifically used to: During the interaction process, the tour guide robot uses a multi-round dialogue engine based on a finite state machine to call the smach package under ROS to manage the dialogue content including the interaction text and reply text, and tracks the dialogue status of the dialogue content including at least greetings, route queries and exhibit explanations.

10. The multi-scenario adaptive navigation robot interactive system according to claim 6, characterized in that: The explanation voice playback module is specifically used for: During the interaction process, the guide robot moves based on the preset guide path, and automatically plays the preset explanation voice when it reaches the guide point on the guide path. After the explanation voice is played, it determines whether the current guide point is the last one. If not, it continues to move to the next guide point. The travel path planning module is specifically used to: When the interactive text carries a target point, an initial path to the target point is searched using a forward tree and a backward tree constructed using a bidirectional progressive optimal search algorithm. The initial path is converted to a sampling domain using an ellipse heuristic sampling algorithm. Collision checking is performed using a K-DTree data structure and Bresenham. The initial path is then smoothed using a cubic B-spline curve interpolation algorithm to obtain a travel path. The robot operation information display module is specifically used for: The navigation robot moves based on the travel path and displays the robot operation information including at least the robot posture, travel path, dynamic obstacle point cloud and point of interest marks on the display screen in real time through the three-dimensional monitoring interface of the RViz plug-in, and records and stores the robot operation log in real time.

Images