Robot intelligent obstacle avoidance navigation method based on semantic segmentation and path prediction

Abstract

The invention relates to the field of robot technology and artificial intelligence, and discloses a robot intelligent obstacle avoidance navigation method based on semantic segmentation and path prediction, which comprises the following steps: an environment perception and modeling stage: identifying social semantic attributes of a dynamic agent and predicting an autonomous intention baseline of the dynamic agent; in the dynamic game topology mapping stage, a master-slave, collaborative or non-cooperative game topology model is selected according to social semantic attributes; an anti-fact reaction generation stage: deducing a reaction track of the dynamic intelligent agent to candidate actions of the robot according to the selected game topology; and a variable structure planning and solving stage: dynamically constructing a joint interaction cost function according to the game topology, and evaluating the candidate actions to solve the optimal action. The problems that a traditional robot navigation strategy is single and cannot adapt to complex social situations are solved, and the navigation safety, efficiency and social adaptability of the robot in a dynamic man-machine coexistence environment are improved.

Description

Technical Field

[0001] This invention relates to the fields of robotics and artificial intelligence, specifically to a robot intelligent obstacle avoidance and navigation method based on semantic segmentation and path prediction. Background Technology

[0002] Currently, mobile robots are gradually moving from structured industrial environments to dynamic and complex scenarios where humans and robots coexist, such as shopping malls, hospitals, and office buildings. In these environments, robots need to possess autonomous navigation and intelligent obstacle avoidance capabilities. This is a core technological prerequisite for ensuring their safe and efficient completion of tasks.

[0003] For the aforementioned navigation tasks, existing technologies typically treat pedestrians or other moving entities in the environment as dynamic obstacles. These solutions acquire the position and velocity information of dynamic obstacles using LiDAR or visual sensors. They then predict the trajectory of the obstacles in the short term based on kinematic or dynamic models. The machine planning module then calculates a local path that avoids collisions and points towards the target based on these predictions. The entire decision-making process relies on a fixed optimization objective function.

[0004] However, existing technologies have limitations in handling complex human-computer interactions. These methods typically abstract all dynamic entities into undifferentiated kinematic points, neglecting their interactive intentions as social intelligent agents. As a result, robots cannot distinguish between a distracted pedestrian and a cooperative partner with clear intentions. Their obstacle avoidance behavior appears rigid and lacks social adaptability. Their decision-making models are also often fixed. Using a uniform cost function or avoidance rule makes it difficult to make the most appropriate trade-offs between different interaction scenarios such as courtesy, cooperation, and competition. This lack of interaction ability ultimately leads to robot behavior that sometimes does not conform to human social intuition. This not only reduces navigation efficiency but also affects human trust and acceptance of robots.

[0005] Therefore, this invention provides a robot intelligent obstacle avoidance and navigation method based on semantic segmentation and path prediction to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a robot intelligent obstacle avoidance and navigation method based on semantic segmentation and path prediction. This method solves the problem that existing robots, in dynamic environments rich in social cues, lack the ability to understand and adapt to the interactive intentions of other intelligent agents, resulting in rigid navigation behavior, low efficiency, or safety hazards.

[0007] To achieve the above objectives, the present invention provides a robot intelligent obstacle avoidance and navigation method based on semantic segmentation and path prediction, comprising: The environmental perception and modeling steps involve acquiring sensor data, identifying dynamic intelligent agents in the environment, and generating structured state data for the dynamic intelligent agents, including their autonomous intention baseline and socio-semantic attributes. The dynamic game topology mapping step involves selecting a corresponding game topology identifier for the dynamic agent from a game topology library containing multiple game topology identifiers, based on the socio-semantic attributes. The counterfactual reaction generation step generates a set of candidate robot actions and, for each candidate robot action in the set, calculates the counterfactual reaction trajectory of the dynamic agent based on the reaction inference function constrained by the game topology identifier. The variable structure planning and solution steps involve dynamically constructing a joint interaction cost function based on the game topology identifier, and using the joint interaction cost function to evaluate the robot's candidate action set in order to solve for the optimal action. The control execution and closed-loop steps convert the optimal action into physical control commands to drive the robot's movement, and then return to execute the environmental perception and modeling steps.

[0008] By employing the above technical solutions, this invention enables robots to not only physically locate dynamic intelligent agents through environmental perception and modeling steps, but also classify their interaction intentions based on socio-semantic attributes and predict their original paths using autonomous intention baselines. Through dynamic game-theoretic topology mapping, abstract socio-semantic attributes are mapped onto specific interaction models, allowing the robot to predict the nature of interactions. Through counterfactual response generation, multi-party responses under different candidate actions are deduced. Finally, through variable structure planning and solving steps, different decision criteria are adopted based on different interaction models. This method endows robots with the ability to dynamically adjust their interaction strategies according to social contexts, upgrading from a simple obstacle avoidance logic to a socialized game-theoretic decision-making logic, thereby achieving safer, more efficient, and intuitive navigation in complex dynamic environments.

[0009] Preferably, in the environmental perception and modeling step, the socio-semantic attribute is selected from a set including attention-deficient features, normal passage features, and competitive features. In the dynamic game topology mapping step, the game topology library at least includes master-slave game topology identifiers, cooperative game topology identifiers, and non-cooperative game topology identifiers, and the rule for selecting the corresponding game topology identifier is as follows: when the socio-semantic attribute is an attention-deficient feature, a master-slave game topology identifier is selected; when the socio-semantic attribute is a normal passage feature, a cooperative game topology identifier is selected; when the socio-semantic attribute is a competitive feature, a non-cooperative game topology identifier is selected.

[0010] By employing the aforementioned technical solution, the socio-semantic attributes extracted from visual features are deterministically correlated with a clear game-theoretic interaction model, providing a clear and reliable basis for subsequent strategy switching. This classification and mapping mechanism is fundamental to enabling robots to move from passive obstacle avoidance to proactive, adaptive interactive decision-making, allowing the system to determine the most suitable interaction framework for different types of interaction objects before planning begins.

[0011] Preferably, when the game topology identifier is a master-slave game topology identifier, in the counterfactual reaction generation step, the reaction inference function sets the counterfactual reaction trajectory of the dynamic agent to be approximately equal to its autonomous intention baseline; in the variable structure planning and solving step, the dynamic construction of the joint interaction cost function specifically involves: constructing a master-slave game cost function with the goal of minimizing robot efficiency cost and safety cost, and with human efficiency cost as a hard constraint.

[0012] By employing the above technical solution, for dynamic agents lacking attention, the system assumes their behavior is predictable and non-interactive. The robot proactively assumes full responsibility for avoidance, prioritizing its own navigation efficiency and absolute safety in its decision-making. Simultaneously, through hard constraints, it ensures its actions do not interfere with the dynamic agent's path. This strategy simulates the behavior of yielding to unconscious participants in human society, effectively guaranteeing the safety of interaction.

[0013] Preferably, when the game topology identifier is a cooperative game topology identifier, in the counterfactual reaction generation step, the reaction inference function calls a cooperative interaction model to generate a coupled trajectory in which both the robot and the dynamic agent make avoidance adjustments as the counterfactual reaction trajectory; in the variable structure planning and solving step, the dynamic construction of the joint interaction cost function specifically involves constructing a cooperative game cost function that minimizes the total social cost by weighted summing of robot efficiency cost, safety cost, and human efficiency cost.

[0014] By adopting the above technical solution, the system anticipates cooperation between dynamic intelligent agents in normal passage. The decision-making objective is to minimize the total social cost, including costs for all parties, and to seek a solution that is beneficial to both. This strategy transforms the robot's behavior from purely self-interested to demonstrating a willingness to cooperate, thereby achieving a smoother and more harmonious human-machine interaction process.

[0015] Preferably, when the game topology identifier is a non-cooperative game topology identifier, in the counterfactual reaction generation step, the reaction inference function calls a competitive interaction model to infer the trajectory of the dynamic agent under competitive conditions with the goal of maximizing its own interests as the counterfactual reaction trajectory; in the variable structure planning and solving step, the dynamic construction of the joint interaction cost function specifically involves: constructing a non-cooperative game cost function that removes human efficiency costs from the cost items and introduces robot benefit items.

[0016] By adopting the above technical solution, the system switches to a more assertive decision-making mode for dynamic agents with competitive intentions. The planning no longer considers the efficiency of the other party, but focuses on its own efficiency and safety, and may include a robot benefit measure for seizing advantageous positions or priority passage. This enables the robot to make more decisive and assertive decisions in competitive scenarios, avoiding unnecessary waiting or stalemates.

[0017] Preferably, the robot efficiency cost is used to measure the deviation of the robot's action from its navigation target, and its calculation combines the deviation of the action trajectory from the reference path and the deviation of the action speed from the target speed; the safety cost is used to measure the minimum safe distance between the robot and the dynamic agent, and its calculation is the positive part of the difference between the preset minimum safe distance threshold and the actual minimum distance; the human efficiency cost is used to measure the path deviation of the dynamic agent caused by interaction, and its calculation is the deviation between the counterfactual reaction trajectory of the dynamic agent and its autonomous intention baseline.

[0018] By adopting the above technical solution, complex interaction effects are decomposed into three quantifiable basic cost items: robot efficiency, safety, and human efficiency. This provides a foundational module for dynamically constructing different joint interaction cost functions based on different game topologies, enabling the variable structure characteristics of the cost function and ensuring that every decision has a clear and quantifiable evaluation basis.

[0019] This invention provides an intelligent obstacle avoidance and navigation method for robots based on semantic segmentation and path prediction. It has the following beneficial effects: 1. This invention identifies the socio-semantic attributes of dynamic intelligent agents during the environmental perception and modeling steps, and maps them to different game topology models such as master-slave, cooperative, or non-cooperative models. This enables the robot to flexibly adjust its decision-making framework according to the different intentions of the interacting objects. Compared with methods using fixed obstacle avoidance rules, the robot of this invention behaves more reasonably in complex human-computer interaction scenarios.

[0020] 2. This invention utilizes a variable-structure joint interaction cost function dynamically constructed based on game topology, employing different optimization objectives for different interaction scenarios. For example, in master-slave games, human efficiency cost is used as a hard constraint to maximize safety; in cooperative games, weighted summation is used to minimize total social cost to improve overall efficiency. This enables the robot to achieve a dynamic and refined balance between safety and traffic efficiency, overcoming the limitations of fixed cost functions in volatile environments.

[0021] 3. This invention combines the socio-semantic attributes of a dynamic intelligent agent with a clear interaction model, and generates corresponding actions accordingly. This makes the robot's navigation behavior logic clear and easier for humans to understand and predict. When the robot exhibits socially intuitive behavioral patterns such as courtesy, cooperation, or competition when facing different interactive objects, it can effectively reduce uncertainty in human-robot coexistence environments, thereby increasing human acceptance and trust in robots and promoting the deployment and application of robots in real social environments. Attached Figure Description

[0022] Figure 1 This is a system block diagram of the present invention; Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] See attached document Figure 1 This invention provides a robot intelligent obstacle avoidance and navigation method based on semantic segmentation and path prediction. The method is implemented through a system that provides an environment perception and state modeling module, which is configured to perform data acquisition, environment understanding and state parameter extraction during the robot navigation process.

[0025] The environmental perception and state modeling module includes a sensor data processing unit. This unit is connected to the robot's LiDAR and vision camera. It receives point cloud data from the LiDAR and image data from the vision camera. The unit performs time synchronization and spatial joint calibration on the point cloud and image data. Through a multi-sensor fusion algorithm, the unit outputs the semantic segmentation result of the current environment. The semantic segmentation result divides the objects in the environment into a set of static obstacles. and dynamic intelligent agent set Static obstacle set Includes fixed structures in the environment, such as walls, shelves, and static furniture. Dynamic intelligent agent set. This includes moving objects in the environment, including pedestrians, other mobile robots, and non-rigidly moving objects. This unit is for dynamic intelligent agents. Each object in the tracking is assigned a unique tracking identifier, and its position information within consecutive time frames is recorded.

[0026] The environmental perception and state modeling module includes an autonomous intent baseline prediction unit. This unit is configured to receive a dynamic set of agents. Historical motion data for each agent in a dynamic agent set. Any intelligent agent in This unit extracts its state before the current time. Frame position sequence as observed historical trajectory The autonomous intent baseline prediction unit utilizes a trajectory prediction model. Processing historical observation trajectories To output intelligent agents In the future preset time period Intra-autonomous intention baseline Autonomous Intent Baseline Characterizing intelligent agents The predicted path is generated based on the robot's own motion inertia and destination orientation, assuming no interference from the robot's current or future actions.

[0027] The environmental perception and state modeling module includes a semantic attribute classification unit. This unit is configured to receive and process a set of dynamic agents. Middle intelligence agent Corresponding visual observation data Visual observation data Based on the agent from the raw image data The bounding box captures the image region or extracts skeletal keypoint features. The semantic attribute classification unit utilizes a semantic attribute classifier. Processing visual observation data To output intelligent agents Social semantic attributes Socio-semantic attributes Used to characterize intelligent agents The interaction category features are selected from a preset set of attributes, which includes attention-deficient features, normal passage features, and competitive features.

[0028] Through the above processing, the environmental perception and state modeling module converts the original sensor signals into a set containing static obstacles. Dynamic intelligent agent set and for each intelligent agent Autonomous Intent Baseline Social semantic attributes The structured state data is then transmitted to the subsequent dynamic game topology mapping module.

[0029] See attached document Figure 1 The system includes a Dynamic Game Topology Mapping (GTM) module. The input of this module is connected to the output of the environment perception and state modeling module, and it receives the generated structured state data, particularly for each dynamic agent. Social semantic attributes .

[0030] The dynamic game topology mapping module includes a library for storing game topologies. The storage unit of this game topology library. Several game topology identifiers are predefined to characterize different interaction model structures between the robot and the dynamic agent. In one specific embodiment, this game topology library... It must include at least: a master-slave game topology identifier A cooperative game topology identifier A non-cooperative game topology identifier .

[0031] The dynamic game topology mapping module further includes a topology mapping rule unit. This unit is configured to implement a topology mapping function. This function is used to determine the socio-semantic attributes of the input. From the game topology library Choose a corresponding game topology identifier. The mapping rules within this topology mapping rule unit are specifically configured as follows: when the received social semantic attributes... When the feature is attention-deficient, the output is... When the normal passage characteristic is displayed, the output is: When the feature is competitive, the output is... The output of the dynamic game topology mapping module is connected to the topology-dependent counterfactual response generation module and the variable structure intent negotiation and planning module. This module will select the game topology identifier. Output to the subsequent modules mentioned above.

[0032] See attached document Figure 1 The system includes a topology-dependent counterfactual response generation module (CRG). The inputs of this module are connected to the environment perception and state modeling module and the dynamic game topology mapping module. The counterfactual response generation module receives the autonomous intent baseline. Sum game topology identifiers .

[0033] The topology-dependent counterfactual response generation module includes a robot candidate action set generation unit. This unit generates a discrete set of robot candidate actions based on the robot's current kinematic state and navigation target. The topology-dependent counterfactual reaction generation module further includes a topology-dependent reaction inference unit. This unit is configured to implement a reaction inference function. This unit traverses the robot's candidate action set. Each candidate action in And for each Call the reaction deduction function To generate intelligent agents Counterfactual reaction trajectory Reaction deduction function The computational behavior is determined by the game topology identifier. As a condition, it serves as a constraint. At that time, the unit is configured to track the agent's reaction trajectory. Set as its autonomous intention baseline :when At that time, the unit is configured to invoke the collaborative interaction model. To calculate a coupling avoidance response trajectory ;when At that time, the unit is configured to invoke the competitive interaction model. To calculate a reaction trajectory aimed at maximizing one's own interests. The output of the topology-dependent counterfactual response generation module is connected to the variable structure intent negotiation and planning module. This module will generate a set of candidate actions for the robot. Each candidate action The generated set of trajectories containing the responses of all agents. It is then transmitted to subsequent modules.

[0034] See attached document Figure 1 The system includes a Variable Structure Intent Negotiation and Planning (SIN) module. The inputs of this module are connected to a dynamic game topology mapping module and a topology-dependent counterfactual response generation module. The Variable Structure Intent Negotiation and Planning module receives game topology identifiers. Robot candidate action set The corresponding set of reaction trajectories Reference path and target speed .

[0035] The variable structure intent negotiation and planning module includes a basic cost item definition unit. This unit is configured to... Each candidate action in Calculate a set of basic cost items, including: robot efficiency cost Used to measure the deviation between actions and navigation targets; safety costs. This is used to measure the minimum safe distance from an intelligent agent; and the cost of human efficiency. It is used to measure the path deviation caused by the interaction of the agent.

[0036] The variable structure intent negotiation and planning module further includes a variable structure JIC cost function construction unit. This unit is based on the received game topology identifier. Dynamically construct the Joint Interaction Cost (JIC) function. When At that time, the unit is configured to construct a master-slave game cost function, which aims to minimize the robot's efficiency cost and safety cost, while taking the human efficiency cost as a hard constraint; when When this unit is configured to construct a cooperative game cost function, the function minimizes the total social cost by weighted summation of all basic cost items; when In this case, the unit is configured to construct a non-cooperative game cost function that removes the human efficiency cost term from the cost and introduces a robot benefit term.

[0037] The variable structure intent negotiation and planning module further includes an optimal action solution unit. This unit is configured to use the cost function determined by the variable structure JIC cost function construction unit to solve the robot's candidate action set. Every action in The evaluation is performed, and the candidate action that minimizes the JIC value is identified as the optimal action. .

[0038] The output of the variable structure intent negotiation and planning module is connected to the control execution module, and the calculated optimal action is then passed on. Transmitted to the control execution module.

[0039] See attached document Figure 1 The system includes a control execution module.

[0040] The input of this module is connected to the output of the variable structure intent negotiation and planning module, and is configured to receive the optimal action. .

[0041] The control execution module includes an instruction translation unit. This unit is responsible for translating abstracted optimal actions. This translates into specific physical control commands that the robot's underlying motion controller can recognize and execute, such as the target linear velocity. and target angular velocity .

[0042] The output of the control execution module is connected to the robot's underlying motion controller. The instruction conversion unit converts the calculated target linear velocity... and target angular velocity The command is sent to the underlying motion controller, which then drives the robot to perform physical movements.

[0043] The execution of the control execution module constitutes the final link in the closed-loop control of the entire navigation system. After the current control cycle is completed, the system returns to the environmental perception and state modeling module to begin the next cycle.

[0044] See attached document Figure 2 The present invention provides a robot intelligent obstacle avoidance and navigation method based on semantic segmentation and path prediction, the method including step S100: environmental perception and modeling.

[0045] Step S100 includes: acquiring point cloud data from the lidar mounted on the robot and acquiring image data from the vision camera. Performing time synchronization and spatial joint calibration processing on the point cloud data and image data.

[0046] Step S100 further includes: processing the fused data through a semantic segmentation model to divide the environment into a set of static obstacles. and dynamic intelligent agent set Static obstacle set Includes fixed structures in the environment and a dynamic set of intelligent agents. Includes moving objects in the environment. Each agent in the system is assigned a unique tracking identifier and its continuous location information is recorded.

[0047] Step S100 further includes: targeting the dynamic agent set Each intelligent agent in Extract its time before the current moment. Frame position sequence, as its observation history trajectory .

[0048] Step S100 further includes: observing historical trajectories Input into a pre-trained trajectory prediction model Trajectory prediction model Calculate and output the intelligent agent In the future preset time period Intra-autonomous intention baseline This autonomous intent baseline Characterizing intelligent agents Predicted path when there is no robot interference. This calculation process is expressed by the following formula: ; in, Represents intelligent agents The baseline path of autonomous intent; Represents a trajectory prediction model; Represents intelligent agents The historical trajectory of observation.

[0049] Step S100 further includes: from the intelligent agent Extract visual observation data from the corresponding image data. .Should It is the bounding box image of the agent or its skeletal key point features.

[0050] Step S100 further includes: transferring visual observation data Input to a semantic attribute classifier Semantic attribute classifier Calculate and output the intelligent agent Social semantic attributes Socio-semantic attributes Features are selected from a predefined set, which includes attention-deficient features, normal passage features, and competitive features. The calculation process is represented by the following formula: ; in, Represents intelligent agents The socio-semantic attributes of; This represents a semantic attribute classifier; Represents intelligent agents Visual observation data.

[0051] After step S100 is completed, a set containing , ,as well as Each intelligent agent of and The method continues with step S200, using the structured state data as input to step S200.

[0052] See attached document Figure 2 After executing step S100, the method continues to execute step S200: dynamic game topology mapping.

[0053] Step S200 receives the output from step S100 for the set of dynamic agents. Each intelligent agent Social semantic attributes This step accesses a predefined game topology library. This game topology library It stores multiple game topology identifiers used to represent different interaction model structures.

[0054] Game topology library It must contain at least: a master-slave game topology identifier Used to characterize asymmetric master-slave interactions; a topological identifier for cooperative games. A topological identifier for characterizing symmetric cooperative avoidance interactions; and a topological identifier for non-cooperative games. , used to characterize competitive non-cooperative interactions.

[0055] Step S200 applies a topology mapping function. The topology mapping function Configured to be based on the input socio-semantic attributes From the game topology library Choose a corresponding game topology identifier. This mapping process is implemented through a set of deterministic rules or a pre-defined lookup table. The mapping process is represented by the following formula: ; in, Represented as an intelligent agent Selected game topology identifier; Represents the topology mapping function; Represents intelligent agents The socio-semantic attributes of .

[0056] In a specific implementation, the topology mapping function The following rules apply: when When the feature is attention deficit, this step will Set as ; when When the characteristics are normal passage, this step will Set as ; when When it is a competing feature, this step will Set as .

[0057] After step S200 is completed, the selected game topology identifier is output. The method continues to execute step S300, and will As input to steps S300 and S400.

[0058] See attached document Figure 2After executing step S200, the method continues to execute step S300: generating counterfactual reactions.

[0059] Step S300 receives the output from step S100 for each agent. Autonomous Intent Baseline and receive the output from step S200 for the intelligent agent. Selected game topology identifier .

[0060] Step S300 includes: First, based on the robot's current kinematic state and navigation target, generating a discrete set of candidate robot actions. This set Each candidate action This represents a sequence of control commands or a candidate trajectory for the robot in the short time domain in the future.

[0061] Step S300 further includes: traversing the robot's candidate action set. Each candidate action in For each This step applies a reaction deduction function. To compute intelligent agents Counterfactual reaction trajectory .

[0062] The reaction derivation function The computational behavior is determined by the game topology identifier from step S200. This serves as a constraint. The derivation process is represented by the following formula: ; in, Represents intelligent agents Actions of the robot The counterfactual reaction trajectory; Represents the reaction deduction function; This indicates a candidate action for the robot. This indicates the intelligent agent received from step S100. The baseline of autonomous intent; This indicates the game topology identifier received from step S200.

[0063] This step specifically performs the deduction of topological dependencies in the following manner: when At that time, the game topology identifier is the master-slave game topology, and this step will determine the reaction deduction function. Configured for rigid prediction mode. In this mode, the agent makes decisions. Will not affect the robot's actions Generate interactive responses and trace their counterfactual response trajectory. Set to approximately equal to its autonomous intent baseline : ; when When the game topology identifier is a cooperative game topology, this step will determine the reaction inference function. Switch to collaborative simulation mode. In this mode, a preset collaborative interaction model is invoked. To calculate .Should The model processes robot actions simultaneously and the autonomous intent of intelligent agents This generates a coupled trajectory in which both sides make avoidance adjustments.

[0064] ; in, This represents a collaborative interaction model.

[0065] when When the game topology identifier is a non-cooperative game topology, this step will determine the reaction inference function. Switch to competitive simulation mode. In this mode, a preset competitive interaction model is invoked. To calculate .Should The model is used to infer the intelligent agent. The response trajectory under competitive conditions, aimed at maximizing its own interests.

[0066] ; in, This represents a competitive interaction model.

[0067] After step S300 is completed, a set of candidate actions for the robot is generated. Each candidate action in Each generated a corresponding trajectory set containing the responses of all agents. The method continues to execute step S400, and will , as well as As input to step S400 See attached document Figure 2 After executing step S300, the method continues to execute step S400: variable structure programming and solution.

[0068] Step S400 receives the robot candidate action set from step S300. and corresponding to, including all intelligent agents The set of counterfactual reaction trajectories This step also receives information from step S200 for each agent. Selected game topology identifier and the autonomous intent baseline from step S100 This step also receives the robot's reference path. and target speed Step S400 includes: First, traversing the robot's candidate action set. Each candidate action in .

[0069] For each candidate action and its corresponding counterfactual reaction trajectory This step calculates a set of basic cost items. Calculate the robot efficiency cost. This cost measures the actions performed by the robot. The deviation between it and its preset navigation target.

[0070] ; in, Indicates the efficiency cost of the robot; Indicates the path deviation weight; Indicates action Corresponding trajectory and reference path The deviation between; Indicates the weight of speed deviation; Indicates action The corresponding speed; This indicates the robot's target speed.

[0071] Calculate security costs This cost measures the actions performed by the robot. Time and the reaction trajectory of all intelligent agents The minimum safety margin between.

[0072] ; in, Indicates safety costs; This indicates the preset minimum safe distance threshold; Indicates that the robot is Perform actions at all times Location; Represents intelligent agents exist Execute response at all times Location. Calculating human efficiency costs. This cost measures all intelligent agents. Counterfactual reaction trajectory and their respective autonomous intention baselines The deviation between them.

[0073] ; in, Indicates human efficiency costs; Represents intelligent agents Individual interaction weights; This represents a function used to calculate the deviation between two trajectories.

[0074] Step S400 further includes: based on the game topology identifier received from step S200 (with a single main interactive agent) For example, dynamically construct the Joint Interaction Cost (JIC) function.

[0075] when At this time, this step constructs the master-slave game cost function. In this structure, human efficiency costs It is set as a hard constraint.

[0076] ; The constraints are: ; in, This represents the cost function in a master-slave game. A high-weighting coefficient indicating safety costs; This represents a preset threshold for tolerating minimal interference.

[0077] when At this time, this step constructs the cost function of the collaborative game. In this structure, all basic cost items are weighted and summed.

[0078] ; in, Represents the cost function of cooperative game; The weighting coefficient represents the cost of human efficiency.

[0079] when At this time, this step constructs the cost function of the non-cooperative game. In this structure, human efficiency costs Removed and a robot revenue item introduced. .

[0080] ; in, Represents the cost function of a non-cooperative game; Indicates the weight of robot revenue; Indicates the execution of an action The potential benefits that will result.

[0081] Step S400 further includes: using the constructed joint interaction cost function ( , ,or (One) The robot's candidate action set Every movement The action that minimizes the cost is evaluated and determined as the optimal action. .

[0082] ; in, Indicates the selected optimal action; This represents the robot's candidate action set; Indicates according to The selected variable structure cost function. After step S400 is completed, the optimal action is output. The method continues to execute step S500, and will As input to step S500.

[0083] See attached document Figure 2 After executing step S400, the method continues to execute step S500: control execution and loop closure.

[0084] Step S500 receives the optimal action solved and output in step S400. .

[0085] Step S500 includes: selecting the optimal action This is translated into concrete physical control instructions executable by the robot's underlying motion controller. In a specific implementation, this translation process includes converting the abstract optimal motion... The analysis identifies a set of target physical quantities, including the target linear velocity. and target angular velocity .

[0086] Step S500 further includes: calculating the target linear velocity and target angular velocity The command is sent to the underlying motion controller.

[0087] The underlying motion controller, based on the received data and The instructions calculate the control signals required to drive the robot's actuators (such as motors) to drive the robot body to perform optimal actions. The corresponding physical motion.

[0088] After step S500 is completed, the method returns to step S100 (environmental perception and modeling) to obtain the robot's updated state and environmental sensor data, and begins the next navigation calculation cycle. In this way, the method executes cyclically at a preset frequency (e.g., 10Hz) to form closed-loop control.

[0089] In summary, by enabling robots to understand the intentions and interaction attributes of others in dynamic and complex social environments, and flexibly switch their own interaction strategies accordingly, more intelligent, safe, and efficient navigation can be achieved. The entire system forms a rapid closed loop of perception, understanding, decision-making, and execution.

[0090] First, in the environmental perception and modeling phase, the focus is not merely on identifying physical obstacles, but also on a deep understanding of dynamic agents. By fusing data from LiDAR and visual sensors, the system can not only locate static objects and dynamic individuals in the environment, but also perform two key predictions using advanced models. The first is predicting the autonomous intention baseline path of the dynamic agent under undisturbed conditions, which is equivalent to anticipating the original goal and movement of others. The second is analyzing its visual characteristics, such as posture and gaze, to determine its current socio-semantic attributes, i.e., whether it is in a state of inattention, normal passage, or competition. This step transforms raw, cluttered sensor signals into structured decision-making data containing high-level semantics.

[0091] Secondly, in the dynamic game topology mapping and counterfactual response generation stage, the system dynamically selects the most suitable interaction model from a pre-set game topology library based on the socio-semantic attributes identified in the previous stage. For example, when faced with a distracted pedestrian, the system switches to a master-slave game model, proactively assuming full responsibility for avoidance; when faced with a pedestrian crossing normally, it switches to a cooperative game model, anticipating mutual cooperation; and when faced with someone cutting in front, it adopts a non-cooperative game model, responding with a stronger self-priority strategy. After selecting a model, the system extrapolates different reactions that other agents might make under the current game rules for its own series of candidate actions, thereby generating multiple sets of counterfactual interaction trajectories, providing rich extrapolation data for subsequent decision-making.

[0092] Secondly, in the variable structure planning and solution phase, the system selects the optimal action based on the interactive deduction results generated in the previous steps. The essence of this phase lies in its variable structure cost function. This cost function dynamically adjusts its composition and weights according to the selected game model. In a master-slave game, the system strictly constrains its own behavior to avoid interfering with others; in a cooperative game, the system strives to minimize the total cost of the entire interactive system and seek the socially optimal solution; while in a non-cooperative game, it ignores the efficiency costs of others and instead maximizes its own passage benefits. By evaluating each candidate action under this dynamically changing cost function, the system can always find the optimal action in the current specific social context.

[0093] Finally, in the control execution and closed-loop phase, the calculated optimal action is transformed into physical instructions that the robot's underlying controller can understand and execute, such as specific target linear and angular velocities, driving the robot to complete the physical movement made in this decision. This complete process of perception, decision-making, and execution cycles continuously at a high frequency (e.g., ten times per second), enabling the robot to respond in real time to dynamic changes in the environment and others, exhibiting high adaptability and social intelligence, and ultimately achieving smooth and safe navigation.

Claims

1. A robot intelligent obstacle avoidance navigation method based on semantic segmentation and path prediction, characterized in that, The method comprises the following steps: S100, environment perception and modeling: acquiring sensor data, identifying dynamic agents in the environment, and generating structured state data containing autonomous intention baseline and social semantic attributes for the dynamic agents; S200, dynamic game topology mapping: according to the social semantic attributes, selecting a corresponding game topology identifier for the dynamic agent from a game topology library containing a plurality of game topology identifiers; S300, counterfactual reaction generation: generating a set of robot candidate actions, and for each robot candidate action in the set of robot candidate actions, calculating a counterfactual reaction trajectory of the dynamic agent according to a reaction deduction function constrained by the game topology identifier; S400, variable structure planning and solving: dynamically constructing a joint interaction cost function according to the game topology identifier, and evaluating the set of robot candidate actions using the joint interaction cost function to solve an optimal action; S500, control execution and closed loop: converting the optimal action into physical control instructions to drive robot movement, and returning to the environment perception and modeling step.

2. The robot intelligent obstacle avoidance navigation method based on semantic segmentation and path prediction according to claim 1, characterized in that, In step S100, the social semantic attributes are selected from a set including attention deficit features, normal traffic features, and competitive features.

3. The robot intelligent obstacle avoidance navigation method based on semantic segmentation and path prediction according to claim 2, characterized in that, In step S200, the game topology library at least contains master-slave game topology identifiers, cooperative game topology identifiers, and non-cooperative game topology identifiers, and the rule for selecting a corresponding game topology identifier is: When the social semantic attributes are attention deficit features, the master-slave game topology identifier is selected; When the social semantic attributes are normal traffic features, the cooperative game topology identifier is selected; When the social semantic attributes are competitive features, the non-cooperative game topology identifier is selected.

4. The robot intelligent obstacle avoidance navigation method based on semantic segmentation and path prediction according to claim 3, characterized in that, In step S300, when the game topology identifier is a master-slave game topology identifier, the reaction deduction function sets the counterfactual reaction trajectory of the dynamic agent to be equal to the autonomous intention baseline.

5. The method of claim 4, wherein, In step S400, when the game topology identifier is a master-slave game topology identifier, the dynamic construction of the joint interaction cost function is specifically: Constructing a master-slave game cost function that minimizes robot efficiency cost and safety cost as the target, and taking human efficiency cost as a hard constraint condition.

6. The robot intelligent obstacle avoidance navigation method based on semantic segmentation and path prediction according to claim 3, characterized in that, In step S300, when the game topology identifier is a cooperative game topology identifier, the reaction deduction function calls a cooperative interaction model to generate a coupled trajectory in which both the robot and the dynamic agent make avoidance adjustments as the counterfactual reaction trajectory.

7. The method of claim 6, wherein, In step S400, when the game topology identifier is a cooperative game topology identifier, the dynamic construction of the joint interaction cost function is specifically: Constructing a cooperative game cost function that minimizes the total social cost by weighted sum of robot efficiency cost, safety cost, and human efficiency cost.

8. The robot intelligent obstacle avoidance navigation method based on semantic segmentation and path prediction according to claim 3, characterized in that, In step S300, when the game topology identifier is a non-cooperative game topology identifier, the reaction deduction function calls a competitive interaction model to deduce a trajectory of the dynamic agent under a competitive condition with a maximum self-interest as an objective as the counterfactual reaction trajectory.

9. The method of claim 8, wherein, In step S400, when the game topology identifier is a non-cooperative game topology identifier, the dynamic construction joint interaction cost function is specifically: A non-cooperative game cost function is constructed by removing the human efficiency cost in the cost term and introducing a robot benefit term.

10. The method of claim 5, wherein, The robot efficiency cost is used to measure the deviation of the robot from the navigation target; the safety cost is used to measure the minimum safety distance from the dynamic agent; and the human efficiency cost is used to measure the path deviation of the dynamic agent due to interaction.

Images