Multi-module collaborative navigation system for intelligent robot

Abstract

The invention discloses a multi-module collaborative navigation system for an intelligent robot, which relates to the field of robot navigation and comprises a multi-mode perception decision module, a semantic visual navigation module, a path memory fusion module and a rolling predictive control module which are electrically connected in sequence and work cooperatively. The multi-modal perception decision module performs weighted fusion on heterogeneous information, outputs scene representation and decision, and provides data support; the semantic visual navigation module analyzes the natural language instruction and combines visual data to complete path planning; the path memory fusion module carries out backtracking correction in an abnormal environment of semantic ambiguity and perception mismatch; and the rolling predictive control module realizes trajectory tracking and dynamic obstacle avoidance. According to the system, dependence on a prior map is reduced, instruction performability and dynamic obstacle avoidance capability are improved, collision risks are reduced, navigation real-time performance, stability and robustness are enhanced, and the system is compatible with multiple chassis and is suitable for complex dynamic scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot navigation, and in particular to a multi-module collaborative navigation system for intelligent robots. BACKGROUND

[0002] With the rapid development of artificial intelligence, robot technology and automation systems, intelligent robots have been widely used in warehousing, logistics, medical treatment, agriculture and unmanned driving fields. In particular, in the aspect of autonomous navigation, with the improvement of sensor performance and computing power, robots gradually have the ability to perform autonomous perception, decision-making and path planning in complex environments, laying a foundation for high-reliability and human-robot collaborative applications.

[0003] Currently, the mainstream navigation follows the technical pipeline of "mapping - positioning - planning - control". However, the above method still has obvious shortcomings in complex dynamic scenes: first, it relies on prior maps, has weak real-time adaptability, and is difficult to cope with rapid environmental changes, especially when dynamic obstacles such as pedestrians and vehicles appear, the navigation accuracy decreases; second, it has insufficient ability to analyze ambiguous or complex natural language instructions, and is difficult to accurately grasp spatial relationships and environmental semantics; third, the dynamic obstacle avoidance performance is limited, and in high-speed interaction scenarios, it is easy to have slow response and insufficient accuracy, increasing the risk of collision; fourth, the computing load is high, and the real-time decision-making ability is insufficient, resulting in control lag when moving quickly or performing intensive tasks, affecting navigation efficiency and task completion.

[0004] In order to overcome the above limitations, there is an urgent need for an intelligent navigation algorithm that deeply couples visual perception, language understanding and control optimization: under weak / no prior map conditions, relying on online visual and semantic alignment to generate task targets and local paths in real time, and performing unified optimization and closed-loop control under dynamic obstacles and safety constraints, thereby improving the navigation accuracy, obstacle avoidance ability and real-time performance of robots in complex dynamic environments, meeting the application requirements of urban and complex indoor scenes. SUMMARY

[0005] The purpose of the present application is to provide a multi-module collaborative navigation system for intelligent robots to solve the problems in the prior art.

[0006] To achieve the above purpose, the present application provides the following technical scheme: a multi-module collaborative navigation system for intelligent robots, the multi-module collaborative navigation system comprising a multi-modal perception decision module, a semantic visual navigation module, a path memory fusion module and a rolling prediction control module; The multi-modal perception decision module is used for confidence weighted fusion of heterogeneous information, outputs stable scene representation and executable decision, and provides data input for the semantic visual navigation module and the rolling prediction control module; The semantic visual navigation module parses the natural language commands input by the user through a large language model and combines them with visual perception data to confirm spatial targets and path planning. The path memory fusion module is used to perform backtracking and correction in abnormal environments, including ambiguous semantic instructions, temporary visual perception mismatch, and sudden environmental changes. The rolling prediction control module is used to track the trajectory of the intelligent robot and perform dynamic obstacle avoidance under the spatial target and path planning given by the semantic visual navigation module.

[0007] The multimodal perception and decision-making module includes an information representation unit and a risk assessment unit; The information representation unit is used by the system to normalize and weight the confidence of each sensor source in the current period, and calculate and output the fusion representation. The calculation formula for the fusion representation is: ; in, For confidence level, For sensor source indexing, For the current period; It is the confidence score of the i-th sensor source in the current period. Temperature coefficient; For Softmax weights, The feature embedding of the i-th sensor source at time k; The risk assessment unit is used to construct individual risks and aggregate individual risks into scenario risks. The calculation formula for scenario risks is as follows: ; in, The robot at each moment The position and velocity, where i is the obstacle index. The estimated position and velocity of the i-th obstacle are given. This is the equivalent safety radius of the obstacle; Denotes the Euclidean norm. To prevent small constants with a denominator of zero; The collision time index is the time to collide with the i-th obstacle. This is the risk attenuation coefficient. Aggregate risks for specific scenarios.

[0008] The multimodal perception and decision-making module also includes an intent decision-making unit; The intent decision unit uses lightweight behavior scoring to select the current execution intent and outputs control preferences such as speed limit and safety margin; the calculation formula for the behavior score is: ; in, For the set of candidate behaviors, Indicates at time Select behavior The progress towards the target; In order to act The predicted risks below; For the cost of execution; For the corresponding weights, The behavior that yields the highest overall score.

[0009] Through the closed-loop process of "confidence-weighted fusion, time-distance risk assessment, and behavior scoring decision-making" described above, the multimodal perception and decision-making module enables the robot to maintain a stable scene understanding when the environment changes and the sensor degrades, reducing its reliance on static maps. In high-speed or crowded scenarios with mixed pedestrian and vehicle traffic, it provides forward-looking safety constraints and executable speed / avoidance preferences, thereby reducing the solution pressure and collision risk of the rolling prediction control module from the source and improving the robot's overall real-time performance, adaptability, and task completion efficiency.

[0010] The semantic visual navigation module includes a semantic parsing unit; The semantic parsing unit receives the user's natural language instructions, parses and understands the spatial description in the instructions using a large language model, and the parsing process is as follows: Let the natural language instructions input by the user be... ,in Let T be the total number of words in the command, and let T be the number of words in the command. The spatial target and navigation commands are generated using a built-in formula from a large language model. The built-in formula is: ; in, Indicates natural language instructions Convert into executable navigation commands .

[0011] The semantic visual navigation module also includes an image processing unit; The image processing unit uses a visual sensor to acquire environmental images, which are then denoted as... ,in For time Image frames at any given time, Given the number of image frames, an image processing model is used to extract features from the environmental image, generating a set of visual features. The feature extraction formula of the image processing model is: ; in, From the original image Extracting visual features The function, wherein the visual features include landmarks and object location information in the environment.

[0012] The semantic visual navigation module also includes a comprehensive processing unit; The integrated processing unit is used to perform fusion calculations on the outputs of the semantic parsing unit and the image processing unit. The fusion calculation formula is as follows: ; in It is a fusion function. It is a joint representation; According to the joint statement The robot extracts landmark information from the environment from visual and linguistic information and generates target locations. Set the current robot position as Target location The path planning between the current position and the target location is solved using a formula, which is: ; in, For the discrete path points to be optimized, For navigation destination, Let j be the geometric center of the j-th obstacle; The equivalent radius of the obstacle. For the safety margin related to body size / positioning error, Weighting of obstacle avoidance penalties; For the number of path points, Number of obstacles, index range ; represents the Euclidean norm, As a positive part operator, it only incurs a penalty at that time. Represents a set of variables Find the minimum value.

[0013] By comprehensively processing visual and linguistic information, the semantic visual navigation module improves the robot's ability to execute complex or ambiguous commands. It resolves semantics into sub-targets and aligns them with visual landmarks in real time. Relying on online perception rather than static maps, it generates paths consistent with the scene, ensuring accurate command understanding and feasible routes.

[0014] The path memory fusion module includes a trajectory recording unit and a trajectory scoring unit; The trajectory recording unit is used to form a memory set with controlled capacity and updated over time, the memory set being... ,in, For indexing memory entries; For historical pose anchor points; for corresponding visual feature embeddings; for instruction semantic summaries; For timestamps; The trajectory scoring unit is used to calculate a score for each trajectory. The scoring calculation formula is as follows: ; Where is the weighting coefficient. For cosine similarity, It is the Euclidean norm. As for the current visual features, This is the current instruction vector. This is the current pose.

[0015] The path memory fusion module also includes a constraint construction unit; The constraint construction unit implements hierarchical control decisions based on the confidence level of the monotonic mapping, predicts the generation of fusion references and the construction of security constraints in the time domain, and dynamically adjusts security and task continuity. The expression for the confidence level is: ; in It is a monotonic mapping; The formula for generating the fusion reference is: ; in, For the semantic visual navigation module in step The current reference trajectory point, For As an anchor point, a historical reference aligned with time. For follow Monotonically increasing fusion weights; The formula for constructing the security constraints is as follows: ; in, This is a collection of indexes that have been running safely for the most recent period. Indicates Center, radius The Euclidean sphere provides a safety margin.

[0016] The path memory fusion module provides robust backtracking correction and reference fusion for the robot in the event of semantic ambiguity or perception mismatch through a closed-loop mechanism of retrieval-evaluation-adaptive fusion-safety constraints, reducing near-obstacle jitter and invalid replanning, and improving the stability and robustness of long-term navigation.

[0017] The specific implementation steps of the rolling prediction control module are as follows: The optimal control input is generated by calculating the current state and environmental images collected at fixed intervals. The formula for calculating the optimal control input is: ; In the formula, Indexed by the current time step, prediction step The predicted time domain length is The predicted position and velocity are denoted as and the control input is . Input change amount Reference sequence Generated by the semantic visual navigation module and aligned with time, weights For symmetric positive semidefinite, weighted norm , It is the Euclidean norm. For obstacle avoidance, The number of obstacles is given, and their center and radius are respectively... and This is a safety margin related to the dimensional error of the body. For positive part operators, For obstacle avoidance weights.

[0018] The aforementioned structure achieves multi-objective collaborative optimization with a single objective function, avoiding inconsistencies and chattering caused by sequential strategies. To ensure feasibility and safety, optimization is performed under engineering constraints such as kinematic / simplified dynamics recursion of the robot chassis, actuator amplitude and rate of change limits, and velocity and attitude boundaries. The predicted position and scale of dynamic obstacles are updated in real time with perception and automatically reflected in the soft safety term of the objective function in the next rolling cycle. A fast numerical method suitable for embedded systems is used to solve the problem, ensuring that the implementable optimal control quantity is obtained within millisecond cycles, thereby maintaining stable tracking and interpretable safe avoidance in unknown and variable scenarios.

[0019] To meet the robot's real-time adaptation and obstacle avoidance needs in dynamic scenarios, the rolling predictive control module adopts rolling prediction and online optimization. Without relying on prior maps, it uniformly constrains "trajectory tracking - control smoothness - safe distance", anticipates pedestrians or vehicles, and quickly closes the loop of the first control variable, reducing latency and collision risk.

[0020] The connection method between the modules of the multi-module cooperative navigation system is as follows: The output of the multimodal perception decision module is electrically connected to the input of the semantic visual navigation module; the output of the semantic visual navigation module is electrically connected to the input of the path memory fusion module; and the output of the path memory fusion module is electrically connected to the input of the rolling prediction control module.

[0021] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention constructs and updates scene representations in real time through multimodal perception and semantic alignment, reducing reliance on prior maps; it can maintain navigation accuracy even when the environment changes, improving the robot's adaptability to dynamic scenes.

[0022] 2. This invention automatically decomposes complex or ambiguous natural language into executable sub-goals and aligns them with visual landmarks in real time; it combines path memory for reference fusion, thereby reducing spatial relationship understanding errors and improving the robot's executability and stability of human commands.

[0023] 3. This invention unifies "trajectory tracking - control smoothing - safe distance" within a single optimization framework through the rolling predictive control module, enabling proactive avoidance of pedestrians / vehicles and reducing collision risks; it also maintains the continuity and stability of robot movement in crowded or high-speed interactive scenarios.

[0024] 4. This invention reduces computational waiting and instruction lag through rolling time-domain optimization and "first control quantity execution" closed-loop design; risk-driven speed and safety margin constraints enable the robot to still have rapid response capability and high task completion efficiency under rapid movement and task-intensive conditions.

[0025] 5. This invention maintains route stability under interferences such as occlusion, lighting changes, and crowds by backtracking historical path anchor points, using safe corridors and adaptive weights, reducing invalid replanning and near-obstacle jitter, and improving the robustness and completion rate of robot long-term navigation.

[0026] 6. This invention allows for flexible parameter adjustment based on preferences for "faster / more stable / safer" through weights and constraints, is compatible with multiple chassis such as differential / Ackerman, has customizable computing power, is easy to deploy at the edge, and can be quickly integrated into existing robot systems. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the working stage structure of a specific embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of the learning stage structure in a specific embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Example 1: As Figure 1 As shown, the present invention provides a technical solution: a multi-module collaborative navigation system for intelligent robots. The multi-module collaborative navigation system includes a multimodal perception and decision-making module, a semantic visual navigation module, a path memory fusion module, and a rolling prediction and control module. The multimodal perception and decision-making module is used to perform confidence-weighted fusion of heterogeneous information, output stable scene representation and executable decisions, and provide data input for the semantic visual navigation module and the scrolling prediction control module; The multimodal perception and decision-making module includes an information representation unit and a risk assessment unit; The information representation unit is used by the system to normalize and weight the confidence of each sensor source in the current period, and calculate and output the fusion representation. The calculation formula for the fusion representation is: ; in, For confidence level, For sensor source indexing, For the current period; It is the confidence score of the i-th sensor source in the current period. Temperature coefficient; For Softmax weights, The feature embedding of the i-th sensor source at time k; The risk assessment unit is used to construct individual risks and aggregate individual risks into scenario risks. The calculation formula for scenario risks is as follows: ; in, The robot at each moment The position and velocity, where i is the obstacle index. The estimated position and velocity of the i-th obstacle are given. This is the equivalent safety radius of the obstacle; Denotes the Euclidean norm. To prevent small constants with a denominator of zero; The collision time index is the time to collide with the i-th obstacle. This is the risk attenuation coefficient. Aggregate risks for specific scenarios; The multimodal perception and decision-making module also includes an intent decision-making unit; The intent decision unit uses lightweight behavior scoring to select the current execution intent and outputs control preferences such as speed limit and safety margin; the calculation formula for the behavior score is: ; in, For the set of candidate behaviors, Indicates at time Select behavior The progress towards the target; In order to act The predicted risks below; For the cost of execution; For the corresponding weights, The behavior that yields the highest overall score; The semantic visual navigation module parses the natural language commands input by the user through a large language model and combines them with visual perception data to confirm spatial targets and path planning. The semantic visual navigation module includes a semantic parsing unit; The semantic parsing unit receives the user's natural language instructions, parses and understands the spatial description in the instructions using a large language model, and the parsing process is as follows: Let the natural language instructions input by the user be... ,in Let T be the total number of words in the command, and let T be the number of words in the command. The spatial target and navigation commands are generated using a built-in formula from a large language model. The built-in formula is: ; in, Indicates natural language instructions Convert into executable navigation commands ; The semantic visual navigation module also includes an image processing unit; The image processing unit uses a visual sensor to acquire environmental images, which are then denoted as... ,in For time Image frames at any given time, Given the number of image frames, an image processing model is used to extract features from the environmental image, generating a set of visual features. The feature extraction formula of the image processing model is: ; in, From the original image Extracting visual features The function, wherein the visual features include landmarks and object location information in the environment; The semantic visual navigation module also includes a comprehensive processing unit; The integrated processing unit is used to perform fusion calculations on the outputs of the semantic parsing unit and the image processing unit. The fusion calculation formula is as follows: ; in It is a fusion function. It is a joint representation; According to the joint statement The robot extracts landmark information from the environment from visual and linguistic information and generates target locations. Set the current robot position as Target location The path planning between the current position and the target location is solved using a formula, which is: ; in, For the discrete path points to be optimized, For navigation destination, Let j be the geometric center of the j-th obstacle; The equivalent radius of the obstacle. For the safety margin related to body size / positioning error, Weighting of obstacle avoidance penalties; For the number of path points, Number of obstacles, index range ; Denotes the Euclidean norm. For positive part operators, only when Punishment will be imposed at that time. Represents a set of variables Minimize; The path memory fusion module is used to perform backtracking and correction in abnormal environments, including ambiguous semantic instructions, temporary visual perception mismatch, and sudden environmental changes. The path memory fusion module includes a trajectory recording unit and a trajectory scoring unit; The trajectory recording unit is used to form a memory set with controlled capacity and updated over time, the memory set being... ,in, For indexing memory entries; As historical pose anchor points; Embed the corresponding visual features; A semantic summary of the instruction; For timestamps; The trajectory scoring unit is used to calculate a score for each trajectory. The scoring calculation formula is as follows: ; in, These are weighting coefficients. For cosine similarity, It is the Euclidean norm. As for the current visual features, This is the current instruction vector. Current pose; The path memory fusion module also includes a constraint construction unit; The constraint construction unit implements hierarchical control decisions based on the confidence level of the monotonic mapping, predicts the generation of fusion references and the construction of security constraints in the time domain, and dynamically adjusts security and task continuity. The expression for the confidence level is: ; in It is a monotonic mapping; The formula for generating the fusion reference is: ; in, For the semantic visual navigation module in step The current reference trajectory point, For As an anchor point, a historical reference aligned with time. For follow Monotonically increasing fusion weights; The formula for constructing the security constraints is as follows: ; in, This is a collection of indexes that have been running safely for the most recent period. Indicates Center, radius Euclidean ball, For safety margin; The rolling prediction control module is used to track the trajectory of the intelligent robot and perform dynamic obstacle avoidance under the spatial target and path planning given by the semantic visual navigation module; The specific implementation steps of the rolling prediction control module are as follows: The optimal control input is generated by calculating the current state and environmental images collected at fixed intervals. The formula for calculating the optimal control input is: ; In the formula, Indexed by the current time step, prediction step The predicted time domain length is The predicted positions and velocities are denoted as follows: The control input is Input change amount Reference sequence Generated by the semantic visual navigation module and aligned with time, weights For symmetric positive semidefinite, weighted norm , It is the Euclidean norm. For obstacle avoidance, The number of obstacles is given, and their center and radius are respectively... and This is a safety margin related to the dimensional error of the body. For positive part operators, For obstacle avoidance weights; The connection method between the modules of the multi-module cooperative navigation system is as follows: The output of the multimodal perception decision module is electrically connected to the input of the semantic visual navigation module; the output of the semantic visual navigation module is electrically connected to the input of the path memory fusion module; and the output of the path memory fusion module is electrically connected to the input of the rolling prediction control module.

[0031] For example: The intelligent robot platform in this embodiment consists of a differential chassis, a forward-facing RGB-D camera, an optional 2D LiDAR, an IMU / odometer, and an on-board computing unit. The system uses ROS2 or equivalent middleware for timing alignment and data dissemination to meet the real-time navigation needs of indoor corridors and open areas.

[0032] The software side includes four core modules: multimodal perception decision-making, semantic visual navigation, path memory fusion, and rolling predictive control, forming a closed loop of "perception-understanding-planning-control": the multimodal perception output scene and dynamic obstacle state natural language are parsed into sub-target chains and aligned with visual landmarks; historical memory provides backtracking and reference fusion at low confidence; and the rolling predictive control module generates control quantities and issues them for execution in the rolling time domain. Multimodal perception decision module This module performs confidence-weighted fusion of heterogeneous information from cameras, LiDAR, and odometry under conditions of no or weak prior maps, outputting a stable scene representation. The executable decisions directly provide robust input to the semantic visual navigation module and the scrolling prediction control module. Confidence levels of each sensor source. Visibility, texture / point cloud density, and time are given synchronously, and the temperature coefficient is applied. The normalization yields the weights: ; In the embodiments, The fusion update frequency is 20Hz. When strong indoor light affects camera confidence... During descent, the radar corresponding Automatically increase, thereby Maintain continuity and stability and avoid dependence on static maps.

[0033] For dynamic obstacles, this module calculates individual risks based on time interval indicators and aggregates them into a scenario risk metric, using the defined... ; Calculate collision time index This allows for the aggregation of scenario risk metrics. A two-level threshold is used: when When entering the "Deceleration and Boundary" level, the upper limit of linear velocity is limited to 0.5 m / s, and the safety margin in rolling predictive control is adjusted from... m is increased to 0.40m, and obstacle avoidance weight is increased simultaneously. (Increased from 5 to 8); when A temporary stop suggestion is triggered at time s, and the rolling predictive control module performs emergency braking and rapid replanning. This tiered strategy takes effect within a closed loop within a 50ms control cycle, ensuring low-latency risk response.

[0034] Regarding the selection of behavioral intent, the module considers the candidate set. (Going straight / Slowing down / Detouring / Temporary stop) are scored, and decision-making criteria are used. ; In the embodiment, weight setting , , When the overall score indicates "go straight" or "decelerate", the module outputs the next reference speed. With safety margin As the hard / soft constraint input for upper-level semantic visual navigation and scrolling predictive control, it works in conjunction with the penalty term in the objective function; after the risk is eliminated, the parameters automatically return to the normal range in the next control cycle, restoring traffic efficiency.

[0035] Through a closed loop of "confidence-weighted fusion - TTC grading - behavior scoring", it continuously provides scenarios consistent with the actual situation, such as data degradation and mixed pedestrian and vehicle traffic. With executable constraints, the upper-level path and control modules possess both forward-looking safety and low latency characteristics, thereby reducing collision risks and improving task completion rates; Semantic visual navigation module This module runs continuously at 10Hz under weak / no prior map conditions, instantly converting natural language into executable paths consistent with the scene. User commands Mapped into a chain of executable commands and sub-targets by a large language model ; The end-to-end resolution and path generation latency does not exceed 120ms. Camera frame sequence. Extracting visual features ; And merge it with the language results into a joint representation in the same coordinate system. ; in accordance with Determine the current target location using online-sensed landmark constraints. and the robot's current position Spatial consistency verification was performed. During implementation, the landmark matching radius was set to 0.50m; the mean square error of the aligned positioning was 0.25m in the corridor and 0.35m in the open area (with continuous correction at 10Hz). Subsequently, the module generates a local reference path within a fixed 10m field of view for the control layer to track. To balance accessibility and obstacle avoidance, the aforementioned soft-constraint optimization model is adopted: ; In the embodiments, The reference path is refreshed at 10Hz; when a temporary blockage occurs in the channel, the system recalculates and sends out feasible alternative routes within a 100ms control cycle.

[0036] Through semantic parsing Feature acquisition Fusion representation Through a closed-loop process of "near-field optimization," the robot can reliably translate fuzzy commands such as "turn right" and "pass the library" into walkable paths consistent with the scene, while maintaining defined numerical indicators: a matching radius of 0.50m, a resolution latency of no more than 120ms, an alignment error of 0.25 / 0.35m, and a field of view of 10m. This provides quantitative support for implementation and results; Path memory fusion module This module provides steady-state support for the robot through a "backtracking-disambiguation-correction" process in case of ambiguity in language commands, temporary visual mismatch, or sudden environmental changes. The system maintains its memory set at 0.5-second intervals. ; in For historical pose anchor points, For corresponding visual features, For instruction semantic summary, This is a timestamp; outdated or low-value entries are automatically discarded to maintain search efficiency.

[0037] When location or semantic confidence decreases, memory retrieval is triggered: based on current visual features. With instruction vector Calculate the matching score for each memory. ; And by Obtain the usable confidence level. Set the confidence threshold during implementation. :when When the robot retreats to the nearest "safe anchor point," the retreat distance is no more than 2 meters, and it usually recovers stability within 1-2 control cycles; when When, select Enter reference calibration.

[0038] To suppress instantaneous errors and reduce ineffective replanning, the current reference and historical segments are adaptively fused to form a corrected reference (refreshed at 10Hz) for the rolling predictive control module to track: ; In this embodiment, the fusion strategy can reduce the amplitude of control input jitter by about 20% in near-obstacle scenarios, while reducing repeated replanning caused by semantic ambiguity.

[0039] To ensure safety during the retraction and re-advancement processes, a "safe corridor" is constructed based on a set of anchor points that have not experienced recent collisions. ; And the safety margin of the rolling predictive control module is used. As the expansion radius. Path preference remains. Internally, even if it is briefly disturbed and deviates, it can quickly return under the guidance of the fused reference, thereby maintaining the robot's route continuity and navigation robustness in dynamic environments. Rolling Predictive Control Module This module uses the reference sequence provided by the semantic visual navigation module. As input, a unified trade-off is made between "trajectory tracking, control smoothing, energy consumption, and safety distance" within the rolling time domain, and only the first control variable at the current moment is executed to reduce decision lag. Its core optimization uses the objective function: ; In this embodiment, the control period is set to 50ms, and the prediction step size is... (Approximately 1 second window); safety margin Obstacle avoidance weighting When the environmental perception determines that the risk is high (e.g., approaching a pedestrian from the opposite direction), the system automatically reduces the upper speed limit given by the upper layer to 0.4-0.6 m / s, and then uses soft constraints in the subsequent rolling optimization. Enhancement and The distance is maintained; once the risk is eliminated, the weight is restored to the normal level to ensure traffic efficiency.

[0040] With the above parameter configuration, This system effectively suppresses abrupt changes in control commands and, in conjunction with a closed-loop strategy of "first control input execution," maintains low-latency output in fast-moving and task-intensive scenarios. Typical tests on mixed corridor and open area sections show that the mean square error of path tracking remains stable at 0.10-0.15m, with a minimum passing distance of no less than 0.35m. When temporary obstructions (cart traversing) or crowds crossing occur, the system can complete online re-optimization of deceleration / detour within 1-2 control cycles, while maintaining smooth changes in control input. It achieves forward-looking avoidance of dynamic obstacles and high-precision tracking of reference paths without relying on static maps.

[0041] In summary, this specific embodiment, in scenarios with no or weak prior maps and dynamic mixed traffic, sequentially obtains a stable scene representation through multimodal confidence fusion, achieves low-latency response through hierarchical risk perception based on collision time, advances based on behavior scoring, generates executable constraints by balancing risks and costs, and triggers memory backtracking and safe corridor constraints when semantic ambiguity or perception degradation occurs. Combined with rolling predictive control, it completes smooth backtracking and re-advancement, reduces collision and near-obstacle jitter, and improves navigation stability and efficiency.

[0042] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A multi-module cooperative navigation system for intelligent robots, characterized in that: The multi-module collaborative navigation system includes a multimodal perception and decision-making module, a semantic visual navigation module, a path memory fusion module, and a rolling prediction and control module; The multimodal perception and decision-making module is used to perform confidence-weighted fusion of heterogeneous information, output stable scene representation and executable decisions, and provide data input for the semantic visual navigation module and the scrolling prediction control module; The semantic visual navigation module parses the natural language commands input by the user through a large language model and combines them with visual perception data to confirm spatial targets and path planning. The path memory fusion module is used to perform backtracking and correction in abnormal environments, including ambiguous semantic instructions, temporary visual perception mismatch, and sudden environmental changes. The rolling prediction control module is used to track the trajectory of the intelligent robot and perform dynamic obstacle avoidance under the spatial target and path planning given by the semantic visual navigation module.

2. The multi-module cooperative navigation system for intelligent robots according to claim 1, characterized in that: The multimodal perception and decision-making module includes an information representation unit and a risk assessment unit; The information representation unit is used by the system to normalize and weight the confidence of each sensor source in the current period, and calculate and output the fusion representation. The calculation formula for the fusion representation is: ； in, For confidence level, For sensor source indexing, For the current period; It is the confidence score of the i-th sensor source in the current period. Temperature coefficient; For Softmax weights, For the feature embedding of the i-th sensor source at time k; The risk assessment unit is used to construct individual risks and aggregate individual risks into scenario risks. The calculation formula for scenario risks is as follows: ； in, The robot at each moment The position and velocity, where i is the obstacle index. The estimated position and velocity of the i-th obstacle are given. This is the equivalent safety radius of the obstacle; Denotes the Euclidean norm. To prevent small constants with a denominator of zero; The collision time index is the time to collide with the i-th obstacle. This is the risk attenuation coefficient. Aggregate risks for specific scenarios.

3. A multi-module cooperative navigation system for intelligent robots according to claim 2, characterized in that: The multimodal perception and decision-making module also includes an intent decision-making unit; The intent decision unit uses lightweight behavior scoring to select the current execution intent and outputs control preferences such as speed limit and safety margin; the calculation formula for the behavior score is: ； in, For the set of candidate behaviors, Indicates the action to be chosen at any given time. The progress towards the target; In order to act The predicted risks below; For the cost of execution; For the corresponding weights, The behavior that yields the highest overall score.

4. A multi-module cooperative navigation system for intelligent robots according to claim 3, characterized in that: The semantic visual navigation module includes a semantic parsing unit; The semantic parsing unit receives the user's natural language instructions, parses and understands the spatial description in the instructions using a large language model, and the parsing process is as follows: Let the natural language instructions input by the user be... ,in Let T be the total number of words in the command, and let T be the number of words in the command. The spatial target and navigation commands are generated using a built-in formula from a large language model. The built-in formula is: ； in, Indicates natural language instructions Convert into executable navigation commands .

5. A multi-module cooperative navigation system for intelligent robots according to claim 4, characterized in that: The semantic visual navigation module also includes an image processing unit; The image processing unit uses a visual sensor to acquire environmental images, which are then denoted as... ,in For time Image frames at any given time, Given the number of image frames, an image processing model is used to extract features from the environmental image, generating a set of visual features. The feature extraction formula of the image processing model is: ； in, From the original image Extracting visual features The function, wherein the visual features include landmarks and object location information in the environment.

6. A multi-module cooperative navigation system for intelligent robots according to claim 5, characterized in that: The semantic visual navigation module also includes a comprehensive processing unit; The integrated processing unit is used to perform fusion calculations on the outputs of the semantic parsing unit and the image processing unit. The fusion calculation formula is as follows: ； in It is a fusion function. It is a joint representation; According to the joint statement The robot extracts landmark information from the environment from visual and linguistic information and generates target locations. Set the current robot position as Target location The path planning between the current position and the target location is solved using a formula, which is: ； in, For the discrete path points to be optimized, For navigation destination, Let j be the geometric center of the j-th obstacle; The equivalent radius of the obstacle. For the safety margin related to body size / positioning error, Weighting of obstacle avoidance penalties; For the number of path points, Number of obstacles, index range ; Denotes the Euclidean norm. For positive part operators, only when Punishment will be imposed at that time. Represents a set of variables Find the minimum value.

7. A multi-module cooperative navigation system for intelligent robots according to claim 6, characterized in that: The path memory fusion module includes a trajectory recording unit and a trajectory scoring unit; The trajectory recording unit is used to form a memory set with controlled capacity and updated over time, the memory set being... ,in, For indexing memory entries; As historical pose anchor points; Embed the corresponding visual features; A semantic summary of the instruction; For timestamps; The trajectory scoring unit is used to calculate a score for each trajectory. The scoring calculation formula is as follows: ； in, These are weighting coefficients. For cosine similarity, It is the Euclidean norm. As for the current visual features, This is the current instruction vector. This is the current pose.

8. A multi-module cooperative navigation system for intelligent robots according to claim 7, characterized in that: The path memory fusion module also includes a constraint construction unit; The constraint construction unit implements hierarchical control decisions based on the confidence level of the monotonic mapping, predicts the generation of fusion references and the construction of security constraints in the time domain, and dynamically adjusts security and task continuity. The expression for the confidence level is: ； in It is a monotonic mapping; The formula for generating the fusion reference is: ； in, For the semantic visual navigation module in step The current reference trajectory point, For As an anchor point, a historical reference aligned with time. For follow Monotonically increasing fusion weights; The formula for constructing the security constraints is as follows: ； in, This is a collection of indexes that have been running safely for the most recent period. Indicates Center, radius Euclidean ball, This is for a safety margin.

9. A multi-module cooperative navigation system for intelligent robots according to claim 8, characterized in that: The specific implementation steps of the rolling prediction control module are as follows: The optimal control input is generated by calculating the current state and environmental images collected at fixed intervals. The formula for calculating the optimal control input is: ； In the formula, Indexed by the current time step, prediction step The predicted time domain length is ; The predicted values ​​of position and velocity are denoted as follows: The control input is Input change amount Reference sequence Generated by the semantic visual navigation module and aligned with time, weights For symmetric positive semidefinite, weighted norm , It is the Euclidean norm. For obstacle avoidance, The number of obstacles is given, and their center and radius are respectively... and This is a safety margin related to the dimensional error of the body. For positive part operators, For obstacle avoidance weights.

10. A multi-module cooperative navigation system for intelligent robots according to claim 9, characterized in that: The connection method between the modules of the multi-module cooperative navigation system is as follows: The output of the multimodal perception decision module is electrically connected to the input of the semantic visual navigation module; the output of the semantic visual navigation module is electrically connected to the input of the path memory fusion module; and the output of the path memory fusion module is electrically connected to the input of the rolling prediction control module.

Images