Robot movement control method and robot

Abstract

This application provides a robot mobility control method and a robot. The method includes: acquiring the robot's current task information, the current panoramic image corresponding to the current waypoint, and the historical navigation information corresponding to the current waypoint; inputting the current task information, the current panoramic image, the historical navigation information, and environmental memory topology information into a pre-trained visual language model to predict node recommendation information, preferred movement direction, and target state information; determining the next waypoint based on the node recommendation information, preferred movement direction, and target state information, and controlling the robot to move to the next waypoint. This application achieves a key leap from intelligent understanding to reliable action by jointly inferring causal factors from language intent, panoramic observation, historical experience, and structural knowledge within a unified representation space, thereby improving the robustness and efficiency of navigation decisions.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and more specifically, to a robot movement control method and a robot. Background Technology

[0002] Object navigation is a key task in embodied artificial intelligence and robot interaction, requiring the seamless integration of visual perception, natural language understanding, and spatial reasoning to enable robots to navigate in unknown environments to find designated target objects.

[0003] Existing target navigation methods are typically based on learning (reinforcement learning, imitation learning) or zero-shot methods (open vocabulary scene understanding, map-guided exploration).

[0004] However, these two types of target navigation methods in the existing technology generally rely on the concatenation of temporary visual observations or simple map representations, which easily leads to target confusion, path redundancy or even complete failure in long-term navigation tasks. At the same time, they lack real-time adaptability to unknown environments and lack effective support from high-level semantic context such as room type and object distribution, resulting in poor navigation performance during target navigation and affecting the success rate of robot interaction tasks. Summary of the Invention

[0005] The purpose of this application is to provide a robot movement control method and a robot to address the shortcomings of the prior art, thereby solving the problem of poor navigation performance in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, one embodiment of this application provides a robot movement control method, the method comprising: At the current time step, acquire the robot's current task information, the current panoramic image corresponding to the current waypoint, and the historical navigation information corresponding to the current waypoint; The environmental memory topology information of the current time step is determined, and the current task information, the current panoramic image, the historical navigation information and the environmental memory topology information are input into a pre-trained visual language model. The visual language model predicts node recommendation information, preferred motion direction and target state information. The target state information is used to indicate the completion state of the current task information in the current panoramic image. The environmental memory topology information is used to indicate the environmental semantics of the robot at the current time step. Based on the node recommendation information, the preferred movement direction, and the target state information, the next waypoint of the current waypoint is determined, and the robot is controlled to move to the next waypoint.

[0007] Secondly, another embodiment of this application provides a robot, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the robot is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of any of the methods described in the first aspect above.

[0008] The beneficial effects of this application are as follows: By acquiring the robot's current task information, the current panoramic image corresponding to the current waypoint, and the historical navigation information corresponding to the current waypoint, and determining the environmental memory topology information of the current time step, discrete waypoints can be upgraded to structured long-term memories with semantic attributes and cross-time step associations. Furthermore, the current task information, current panoramic image, historical navigation information, and environmental memory topology information are input into a pre-trained visual language model. Through this model, language intent, panoramic observation, historical experience, and structural knowledge are jointly causally inferred within a unified representation space, thereby predicting node recommendation letters. The system uses information such as node recommendations, preferred movement direction, and target state to determine the next waypoint from the current waypoint. It can transform the semantic suggestions output by the visual language model into actionable movement commands that are both goal-oriented, environmentally adaptable, and physically safe. This achieves a key leap from intelligent understanding to reliable action and controls the robot to move to the next waypoint. It can reduce path drift caused by single-frame false detection or occlusion and avoid getting lost due to local observation failures, thereby improving the robustness and efficiency of navigation decisions. At the same time, it increases the success rate in long-distance tasks.

[0009] Furthermore, in practical applications, there is no need for offline training for specific scenarios, significantly reducing the adaptation costs of engineering deployment. In addition, it reduces computing and storage overhead, adapts to robot edge deployment, and enhances human-machine collaboration friendliness. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic flowchart of a robot movement control method provided in an embodiment of this application; Figure 2 This is a schematic flowchart illustrating the process of determining the next waypoint in the robot mobility control method provided in this application embodiment; Figure 3 This is an alternative flowchart illustrating the process of determining the next waypoint in the robot mobility control method provided in this application embodiment; Figure 4 This is an alternative flowchart illustrating the process of determining the next waypoint in the robot mobility control method provided in this application embodiment; Figure 5 This is a schematic flowchart illustrating the process of determining the hazard score of each candidate waypoint in the robot mobility control method provided in this application embodiment; Figure 6 This is a flowchart illustrating the process of calculating the target actionable parameters for each candidate waypoint in the robot mobility control method provided in this application embodiment. Figure 7 This is a flowchart illustrating the process of calculating the target actionable parameters for each candidate waypoint in the robot mobility control method provided in this application embodiment. Figure 8 This is a flowchart illustrating the process of determining the environmental memory topology information at the current time step in the robot mobility control method provided in this application embodiment. Figure 9 A flowchart illustrating the process of determining the current semantic point cloud in the robot movement control method provided in this application embodiment; Figure 10 This is a schematic diagram of the robot provided in an embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0013] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0014] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0015] Existing target navigation methods are typically based on learning (reinforcement learning, imitation learning) or zero-shot methods (open vocabulary scene understanding, map-guided exploration).

[0016] However, these two types of target navigation methods in the existing technology have the following drawbacks: First, spatial memory is fragmented and lacks a mechanism for accumulating long-term environmental structural knowledge, which can easily lead to target confusion, path redundancy, or even complete failure in long-term navigation tasks. Second, the application of topological structures is limited. Traditional topological maps are mostly statically predefined or generated based on pre-calculated paths, lacking real-time adaptability to unknown environments and not deeply integrated with language-driven spatial reasoning. Third, semantic information is separated from geometric structure, and navigation decisions lack effective support from high-level semantic contexts such as room type and object distribution. Finally, some methods still require task-specific training, have insufficient zero-shot generalization ability, and are difficult to adapt to open-vocabulary object categories and diverse indoor scenes, limiting deployment costs and adaptability.

[0017] Therefore, these two types of target navigation methods in the existing technology generally rely on the concatenation of temporary visual observations or simple map representations, which easily leads to target confusion, path redundancy or even complete failure in long-term navigation tasks. At the same time, they lack real-time adaptability to unknown environments and lack effective support from high-level semantic contexts such as room type and object distribution, resulting in poor navigation performance during target navigation and affecting the success rate of robot interaction tasks.

[0018] Based on the aforementioned problems, this application proposes a robot mobility control method. This method acquires the robot's current task information, the current panoramic image corresponding to the current waypoint, and the historical navigation information corresponding to the current waypoint. It also determines the environmental memory topology information for the current time step. The current task information, current panoramic image, historical navigation information, and environmental memory topology information are input into a pre-trained visual language model. The visual language model predicts node recommendation information, preferred movement direction, and target state information. Based on these information, the next waypoint is determined, and the robot is controlled to move to the next waypoint. This method enables joint alignment of natural language tasks, panoramic images, and topological node semantics within the same embedding space, achieving cross-modal causal reasoning. It reduces path drift caused by single-frame false detections or occlusion, avoids getting lost due to local observation failures, and thus improves the robustness and efficiency of navigation decisions. Furthermore, it increases the success rate in long-distance tasks.

[0019] First, the relevant background of the robot movement control method provided in the embodiments of this application will be explained.

[0020] It should be understood that the background related to the robot movement control method provided in the embodiments of this application can be deployed in any device that needs to perform dynamic environmental perception and movement, such as a robot, specifically a humanoid robot, a humanoid robot, or a robot dog.

[0021] By example, by implementing the robot mobility control method provided in the embodiments of this application, the robot can more intelligently understand the environment and perform mobility tasks or other complex tasks.

[0022] For example, a robot can be a robot dog. Specifically, a robot dog is a robot that mimics the appearance of a canine and typically walks on four legs.

[0023] Specifically, a robot dog may include a mechanical structure, a drive system, a sensing system, a control system, and a power supply system.

[0024] For example, the mechanical structure includes a fuselage body and a leg structure, the leg structure including multiple legs, each leg including a hip joint and a knee joint, and may also include an ankle joint, the mechanical structure being used to perform specific actions, such as movement.

[0025] The drive system includes joint motors, reducers, and transmission structures. The joint motors control the rotational or linear motion of the joints in the leg structure, providing precise power output. The reducers lower the rotational speed and increase the torque, enabling the robot dog to stand and carry heavy loads. The transmission structure, including timing belts and connecting rods, converts the rotational motion of the motors into leg swinging motion.

[0026] The sensing system includes a vision sensor and a lidar sensor. Optionally, the sensing system may also include a torque sensor, an inertial measurement unit, and a tactile sensor. The sensing system is used to detect environmental information in real time to obtain real-time environmental information for controlling the robot dog. Optionally, the vision sensor may include a binocular camera and a depth camera.

[0027] Optionally, the vision sensor can be located on the head, front end, or top of the robot dog.

[0028] Optionally, the lidar can be fixed to the head, front end, or top of the robot dog.

[0029] The control system includes a processor, a memory, and a communication module. The memory stores machine-readable instructions that the processor can execute. When the robot dog's control system is running, the processor executes the machine-readable instructions to process and store the sensor data obtained by the sensing system, and executes the steps of the robot movement control method provided in this application embodiment to control the robot dog's movement.

[0030] Optionally, the robot dog may also include a human-computer interaction interface to enable human-computer interaction.

[0031] The robot movement control method provided in this application will be described in detail below with reference to several embodiments.

[0032] Figure 1 This is a flowchart illustrating a robot movement control method provided in an embodiment of this application, with reference to... Figure 1 As shown, the executing entity of this method is any electronic device with processing capabilities, such as the aforementioned robot dog, and the method includes: S101. At the current time step, obtain the robot's current task information, the current panoramic image corresponding to the current waypoint, and the historical navigation information corresponding to the current waypoint.

[0033] It should be noted that during the robot's movement, waypoint generation and movement are performed periodically. The current time step is a relative temporal concept, dynamically advancing with the robot's navigation progress. For example, when the robot is at time step t, its current waypoint is the one at time step t. The pose anchor point determined by the decision and reached through motion control is used as the spatiotemporal reference in steps S101-S104 executed at time step t. That is, in this embodiment, the current time step is taken as time step t, and the starting point of the next time step (i.e., the next waypoint) is determined from the current waypoint at the current time step t.

[0034] Here, the current waypoint corresponding to the current time step refers to the discrete cognitive anchor point formed by the robot during continuous navigation. The current waypoint corresponds to a spatial point in real space. The current waypoint corresponding to the current time step is the next waypoint of the previous time step.

[0035] Optionally, at the current time step, the robot's current task information, the current panoramic image corresponding to the current waypoint, and the historical navigation information corresponding to the current waypoint are obtained.

[0036] The current task information refers to the task the robot needs to perform. This information indicates the target object. The current panoramic image corresponding to the current waypoint is a panoramic image obtained by fusing images acquired from the current waypoint and all historical waypoints preceding it.

[0037] The historical navigation information corresponding to the current waypoint refers to the navigation information corresponding to all historical waypoints before the current waypoint. The historical navigation information corresponding to the current waypoint includes at least the historical movement sequence, which includes multiple historical waypoints. The historical movement sequence is used to characterize the robot's movement trajectory.

[0038] Optionally, the historical navigation information corresponding to the current waypoint also includes: a sequence of historical waypoint semantic labels and a sequence of historical task completion statuses. The sequence of historical waypoint semantic labels includes the room type associated with each historical waypoint, and the sequence of historical task completion statuses includes the completion status of similar or related tasks in the past at each historical waypoint.

[0039] S102. Determine the environmental memory topology information of the current time step, and input the current task information, current panoramic image, historical navigation information and environmental memory topology information into the pre-trained visual language model. The visual language model then predicts the node recommendation information, preferred motion direction and target state information.

[0040] Optionally, the environmental memory topology information for the current time step is determined. This environmental memory topology information is used to indicate the robot's environmental semantics at the current time step, and it is used to capture scene connectivity, adjacency structure, and semantic information within the environment.

[0041] The environmental memory topology information includes multiple nodes and their attribute information. Each node represents a spatial point in the robot's environment, and the attribute information of each node is used to indicate the semantic features of the spatial point. Specifically, the essence of environmental memory topology information is spatial memory. The attribute information of each node refers to the topological description data representing the semantic state of the surrounding environment of that spatial location, obtained by the robot after performing spatial neighborhood retrieval and statistical analysis with the spatial location corresponding to that node as the query origin.

[0042] Optionally, the environmental memory topology information may also include the edges corresponding to each node, wherein the edges corresponding to each node are used to represent the robot's explored paths.

[0043] The attribute information of each node includes at least: identifier, location information, neighboring object category information, room type, and number of frontier points. The attribute information of each node may also include: confidence level of the neighboring point cloud.

[0044] Specifically, location information refers to the node's three-dimensional spatial coordinates and orientation in the global coordinate system. Neighborhood object category information refers to the statistical distribution of the semantic categories to which all points in the current object point cloud belong within a preset radius *r* centered on the node. Room type refers to the high-level semantic discrimination result of the functional area type of the physical space to which the current waypoint belongs. Number of leading-edge points refers to the total number of leading-edge points in the current leading-edge point cloud contained within the current node's neighborhood, and is a scalar integer. Neighborhood point cloud confidence is a quantitative assessment value of the quality and consistency of point cloud data within the neighborhood centered on the current node, reflecting the reliability of the memory in that area.

[0045] In one example, the environmental memory topology information for the current time step can be determined based on the robot's position, posture, and panoramic image at the current time step.

[0046] In another example, a pre-trained implicit neural network can be used to fit a local spatial implicit function with the current waypoint as the origin, and multiple geometric clusters can be obtained by sampling in the local space. Then, semantic mean aggregation is performed on each geometric cluster to obtain semantic labels, and the centroid of each geometric cluster is calculated as the coordinates of the node to obtain the environmental memory topology information of the current time step.

[0047] Optionally, the current task information, current panoramic image, historical navigation information, environmental memory topology information, and preset prompt words are input into a pre-trained visual language model, which then performs reasoning to obtain node recommendation information, preferred movement direction, and target state information.

[0048] Among them, the current task information is used to provide high-level semantic target constraints, the current panoramic image is used to provide omnidirectional, unobstructed instantaneous environmental observation, the historical navigation information is used to provide spatiotemporal experience memory and task execution constraints, and the environmental memory topology information is used to provide structured, reasonable long-term environmental knowledge.

[0049] The node recommendation information includes the identifier k' of the nodes prioritized for exploration by the visual language model, and the preferred motion direction d. VLM This refers to the direction that the visual language model determines may contain the target object.

[0050] The target status information indicates the completion status of the current task information in the current panoramic image. The target status information includes: completed and incomplete.

[0051] Specifically, the target state information is "complete," meaning the target object corresponding to the current task information exists in the current panoramic image. The target state information is "incomplete," meaning the target object corresponding to the current task information does not exist in the current panoramic image. For example, the target state information g... found ∈{0,1}.

[0052] Optionally, while outputting target state information, the visual language model can also output target hit information corresponding to the target state information. Target hit information includes: hit and miss. A hit means that the category of the target object is determined to exist by the visual language model, that is, the target object belongs to the preset object category directory. A miss means that the category of the target object is determined not to exist by the visual language model, that is, the target object does not belong to the preset object category directory.

[0053] S103. Based on the node recommendation information, preferred movement direction, and target status information, determine the next waypoint of the current waypoint and control the robot to move to the next waypoint.

[0054] In one example, based on the target state information, the corresponding preset decision-making strategy can be invoked, and the next waypoint of the current waypoint can be determined by the decision-making strategy, node recommendation information, and preferred movement direction, and the robot can be controlled to move to the next waypoint.

[0055] In another example, after obtaining node recommendation information, preferred direction of movement, and target state information, the node recommendation information, preferred direction of movement, and target state information can be input into a pre-trained waypoint decision model, which will then use the waypoint decision model to infer the next waypoint.

[0056] It should be noted that after determining the next waypoint from the current waypoint, path planning can be performed between the current waypoint and the next waypoint, resulting in multiple intermediate path points. In this embodiment, each intermediate path point only guides the robot's movement during its movement and does not participate in the determination of the next waypoint. Furthermore, for ease of explanation, this embodiment uses a one-to-one correspondence between the current time step and the current waypoint; however, this can be adjusted according to actual circumstances in practical applications.

[0057] In this embodiment, by acquiring the robot's current task information, the current panoramic image corresponding to the current waypoint, and the historical navigation information corresponding to the current waypoint, and determining the environmental memory topology information of the current time step, discrete waypoints can be upgraded to structured long-term memories with semantic attributes and cross-time step associations. The current task information, current panoramic image, historical navigation information, and environmental memory topology information are input into a pre-trained visual language model. The visual language model performs joint causal inference on language intent, panoramic observation, historical experience, and structural knowledge in a unified representation space, thereby predicting node recommendation information, preferred movement direction, and target state information. Based on the node recommendation information, preferred movement direction, and target state information, the next waypoint of the current waypoint is determined. The semantic suggestions output by the visual language model can be transformed into motion commands that are immediately executable, have goal orientation, environmental adaptability, and physical safety, realizing a key leap from intelligent understanding to reliable action and controlling the robot to move to the next waypoint. This can reduce path drift caused by single-frame false detection or occlusion, avoid getting lost due to local observation failure, thereby improving the robustness and efficiency of navigation decisions, and increasing the success rate in long-distance tasks.

[0058] Furthermore, in practical applications, there is no need for offline training for specific scenarios, significantly reducing the adaptation costs of engineering deployment. In addition, it reduces computing and storage overhead, adapts to robot edge deployment, and enhances human-machine collaboration friendliness.

[0059] In one possible implementation, Figure 2 This is a flowchart illustrating the process of determining the next waypoint in the robot mobility control method provided in this application embodiment, with reference to... Figure 2 As shown, in S103 above, the next waypoint is determined based on node recommendation information, preferred direction of movement, and target status information, including: S201. Determine the waypoint determination mode for the next waypoint based on the target status information.

[0060] Optionally, the waypoint determination mode for the next waypoint can be determined based on the target status information.

[0061] The waypoint determination modes include exploration mode and non-exploration mode.

[0062] In one example, when the target status information is "completed", the waypoint determination mode is determined to be non-exploration mode; when the target status information is "incomplete", the waypoint determination mode is determined to be exploration mode.

[0063] In another example, when the target status information is "completed" and the target hit information is "hit", the waypoint determination mode is determined to be non-exploration mode; when the target status information is "incomplete" and the target hit information is "missed", the waypoint determination mode is determined to be exploration mode.

[0064] S202. Based on the node recommendation information, waypoint determination mode, and preferred direction of movement, determine the next waypoint for the current waypoint.

[0065] Optionally, after obtaining the waypoint determination pattern, the next waypoint is determined based on the waypoint determination pattern, node recommendation information, and preferred direction of movement.

[0066] In one example, the corresponding waypoint determination strategy can be obtained based on the waypoint determination pattern, and the next waypoint can be determined based on the waypoint determination strategy, node recommendation information, and preferred direction of movement.

[0067] In another example, waypoint determination patterns, node recommendation information, and preferred directions of motion can be input into a pre-trained waypoint decision model, which will then infer the next waypoint.

[0068] By using target state information, the waypoint determination pattern for the next waypoint is determined, and based on the waypoint determination pattern, the next waypoint is determined. This upgrades the starting conditions for navigation decisions from physical quantities such as time, number of steps, and spatial distance to target state information, a high-level cognitive state that represents the progress of the semantic task. This makes the navigation process strongly bound to the semantics of the task, and enables the navigation decision-making process to adaptively switch between efficiently expanding cognitive boundaries and achieving high-precision arrival, thereby improving the adaptive capability of the navigation process.

[0069] In one possible implementation, Figure 3 This is another flowchart illustrating the process of determining the next waypoint in the robot mobility control method provided in this application embodiment, referring to... Figure 3 As shown, in S202 above, the next waypoint is determined based on node recommendation information, waypoint determination mode, and preferred direction of movement, including: S301. Determine at least one candidate waypoint corresponding to the current waypoint.

[0070] Optionally, at least one candidate waypoint can be determined corresponding to the current waypoint. The candidate waypoints include all movable points in the robot's environment.

[0071] In one example, the current scene base point cloud corresponding to the current waypoint can be determined based on the robot's image and attitude at the current waypoint. Then, from the current scene base point cloud, the current navigable point cloud is obtained by filtering according to a preset navigable height range and passable structures. Each navigable point cloud in the current navigable point cloud is then considered as a candidate waypoint. The process of determining the current scene base point cloud and the navigable point cloud can be referred to S901-S905.

[0072] In another example, within the current panoramic image, a search can be performed with the current waypoint as the center, according to preset candidate waypoint search criteria, to obtain at least one candidate waypoint corresponding to the current waypoint. The candidate waypoint search criteria include at least: geometric feasibility constraints, semantic rationality constraints, and task-oriented constraints.

[0073] For example, geometric feasibility constraints are used to ensure the physical reachability of each candidate waypoint, including orientation constraints, altitude constraints, and accessibility constraints. Semantic rationality constraints are used to ensure that each candidate waypoint has navigational value, including connectivity constraints, functional constraints, and unexplored area boundary constraints. Semantic rationality constraints are used to ensure that each candidate waypoint is strongly relevant to the mission, including visual saliency constraints and historical mission relevance constraints.

[0074] S302. Based on the node recommendation information, waypoint determination mode, and preferred direction of movement, calculate the target action parameters for each candidate waypoint.

[0075] It is understandable that after obtaining each candidate waypoint, the action score of each candidate waypoint under the corresponding waypoint determination mode can be determined as the target action parameter, so that the candidate waypoint with the maximum benefit can be selected as the next waypoint according to the action score.

[0076] Optionally, after obtaining each candidate waypoint, the target action parameters of each candidate waypoint are calculated based on the node recommendation information, waypoint determination mode, and preferred direction of movement.

[0077] For example, the target action parameters of the candidate waypoints are calculated based on the positions of the candidate waypoints, the current waypoint, node recommendation information, waypoint determination mode, and preferred direction of movement.

[0078] S303. Based on the target actionable parameters of each candidate waypoint, determine the next waypoint from among the candidate waypoints.

[0079] Optionally, after obtaining the target actionable parameters for each candidate waypoint, the candidate waypoint corresponding to the maximum value of the target actionable parameters can be selected as the next waypoint from among the candidate waypoints.

[0080] By identifying candidate waypoints and calculating their target actionable parameters based on node recommendation information, waypoint determination patterns, and preferred movement directions, the next waypoint is determined from among these candidate waypoints according to these parameters. This method enables the calculation of target actionable parameters through multi-source heterogeneous parameter coupling, ensuring that even if a single signal fails, the remaining dimensions still provide strong constraints. This avoids waypoint decision collapse caused by distortion in single-frame visual observations or missing local geometric information, significantly improving navigation robustness in long-term, highly dynamic indoor environments. Furthermore, it achieves spatial alignment and causal weighting of four types of information: natural language task intent, panoramic visual observation, structured environmental topology, and spatiotemporal navigation experience, ensuring that navigation decisions possess both high-level semantic rationality and low-level motion feasibility.

[0081] In one possible implementation, Figure 4 This is another flowchart illustrating the process of determining the next waypoint in the robot mobility control method provided in this application embodiment, referring to... Figure 4 As shown, in S303 above, based on the target actionable parameters of each candidate waypoint, the next waypoint of the current waypoint is determined from each candidate waypoint, including: S401. Determine the hazard score of each candidate waypoint, and based on the hazard score of each candidate waypoint, select at least one optional waypoint from among the candidate waypoints.

[0082] Optionally, a hazard score can be calculated for each candidate waypoint, and at least one optional waypoint can be selected from the candidate waypoints based on the hazard score of each candidate waypoint.

[0083] For example, the distance between each candidate waypoint and the nearest obstacle corresponding to each candidate waypoint can be calculated, and the hazard score of each candidate waypoint can be calculated based on the distance between each candidate waypoint and the nearest obstacle corresponding to each candidate waypoint.

[0084] By determining the hazard score of each candidate waypoint and screening them based on the hazard score, it can be ensured that any selectable waypoint entering the decision loop meets the physical feasibility baseline, thus preventing the misselection of high-scoring but high-risk waypoints from the outset and improving safety during the navigation process.

[0085] S402. Obtain the maximum value of the target action parameters among the available waypoints, and obtain the target waypoint corresponding to the maximum value, and use the target waypoint as the next waypoint of the current waypoint.

[0086] Optionally, the maximum value of the target actionable parameter among the optional waypoints is obtained, and the target waypoint corresponding to the maximum value is obtained, and the target waypoint is used as the next waypoint of the current waypoint.

[0087] In one possible implementation, Figure 5This is a schematic flowchart illustrating the process of determining the hazard score of each candidate waypoint in the robot mobility control method provided in this application embodiment, with reference to... Figure 5 As shown, S401 above determines the hazard score for each candidate waypoint, including: S501. Identify all obstacle points in the current panoramic image.

[0088] In one example, all obstacle points in the current panoramic image can be obtained based on the pre-constructed current obstacle point cloud (Mobs). The process of determining the current scene base point cloud and the current obstacle point cloud can be referred to in S904.

[0089] In another example, obstacle identification can be performed on the current panoramic image to obtain all obstacle points in the current panoramic image.

[0090] S502. Based on each candidate waypoint and all obstacle points, calculate the shortest distance between each candidate waypoint and all obstacle points.

[0091] Alternatively, it can be based on candidate waypoints p i And all obstacle points, calculate the shortest distance of each candidate waypoint relative to all obstacle points. Specifically, it can be achieved by referring to the following formula (2).

[0092] By finding the shortest distance between each candidate waypoint and all obstacle points, it is possible to capture immediate collision risks.

[0093] S503. Based on each candidate waypoint and all obstacle points, calculate the maximum and minimum distances from each candidate point to the obstacle.

[0094] Optionally, the maximum and minimum distances from all candidate waypoints to obstacles can be calculated based on all candidate waypoints and all obstacle points. Specifically, this can be achieved by referring to the following formulas (3)-(4).

[0095] The maximum distance from all candidate points to obstacles can characterize the local spatial redundancy, while the minimum distance from all candidate points to obstacles can reveal the spatial constraint strength.

[0096] S504. Calculate the hazard score of each candidate waypoint based on the shortest, maximum, and minimum distances relative to all obstacle points.

[0097] Optionally, the hazard score of each candidate waypoint can be calculated based on the shortest, maximum, and minimum distances relative to all obstacle points. Specifically, this can be achieved using the following formula (1).

[0098] By calculating the shortest, maximum, and minimum distances of each candidate waypoint relative to all obstacle points, the hazard score of each candidate waypoint is obtained. Through the fusion of three-dimensional distance features, a quantitative description of the local risk field topology of the candidate waypoints can be achieved.

[0099] The following is an illustrative explanation of the process for determining the target actionable parameters for each candidate waypoint. Before calculating the target actionable parameters for each candidate waypoint, it should be understood that all candidate waypoints constitute a candidate point cloud set. In the candidate point cloud set, for any candidate waypoint, the target point cloud in multiple dimensions can be determined. When calculating the target actionable parameters for that candidate waypoint, the sub-target actionable parameters of each dimension can be calculated based on the target point cloud in each dimension, thereby calculating the target actionable parameters for the candidate waypoint.

[0100] For any candidate waypoint p i And the target point cloud S in any dimension, and the actionable parameters of the candidate waypoints as sub-targets in that dimension. This can be achieved by referring to the following formulas (1)-(4): (1) (2) (3) (4) in, q is a preset error coefficient, and q is any point in the target point cloud.

[0101] Based on this, the following explains the target action parameters for each candidate waypoint calculated.

[0102] In one possible implementation, Figure 6 This is a flowchart illustrating the calculation of target actionable parameters for each candidate waypoint in the robot mobility control method provided in this application embodiment, with reference to... Figure 6 As shown, in S302 above, based on node recommendation information, waypoint determination mode, and preferred direction of movement, the target action parameters for each candidate waypoint are calculated, including: S601. If the waypoint determination mode is exploration mode, determine the directional guidance parameters of the candidate waypoints based on the preferred direction of movement.

[0103] Optionally, if the waypoint determination mode is exploration mode, the directional guidance parameters of the candidate waypoints are calculated based on the preferred direction of movement. These directional guidance parameters are the actionable parameters of the sub-targets in the directional dimension.

[0104] For example, according to the preferred direction of motion d VLMDetermine the candidate point cloud set along the preferred motion direction d. VLM The point set is used as the target point cloud S in the direction dimension. dir And the candidate waypoint p is calculated by referring to the aforementioned formulas (1)-(4). i Directional guidance parameters .

[0105] By using directional guidance parameters, the visual reasoning ability of a visual language model can be used to identify possible directions to the target during navigation.

[0106] S602. Determine the node guidance parameters for candidate waypoints based on the node recommendation information.

[0107] Optionally, if the waypoint determination mode is exploration mode, the node guidance parameters of the candidate waypoints are calculated based on the node identifiers in the node recommendation information. These node guidance parameters are the actionable parameters of the sub-targets at the node level.

[0108] For example, based on the node identifier k' in the node recommendation information, the node corresponding to node N is found from the pre-constructed current frontier point cloud. k The set of associated frontier points serves as the target point cloud S in the node dimension. node And the candidate waypoint p is calculated by referring to the aforementioned formulas (1)-(4). i Node boot parameters .

[0109] By using node-guided parameters, high-potential areas can be prioritized for exploration during navigation by leveraging the topological reasoning capabilities of the visual language model.

[0110] S603. Based on the current frontier point cloud in the current semantic point cloud, determine the frontier exploration parameters of the candidate waypoints.

[0111] Optionally, if the waypoint determination mode is exploration mode, the front exploration parameters of the candidate waypoints are calculated based on the current front edge point cloud in the pre-constructed current semantic point cloud. These front edge exploration parameters are the actionable parameters of the sub-targets in the front edge dimension.

[0112] For example, the current leading edge point cloud Mfro is obtained from the pre-constructed point cloud as the target point cloud in the leading edge dimension, and the candidate waypoint p is calculated according to the aforementioned formulas (1)-(4). i Frontier exploration parameters Among them, the current cutting-edge point cloud Mfro can be implemented with reference to S905.

[0113] By exploring cutting-edge parameters, a wide range of exploration pressures can be provided during navigation, supplementing the targeted guidance provided by the visual language model.

[0114] S604. Based on historical navigation information, determine the historical avoidance parameters of candidate waypoints.

[0115] Optionally, if the waypoint determination mode is exploration mode, historical avoidance parameters of the candidate waypoints are calculated based on historical navigation information. These historical avoidance parameters are the actionable parameters of the sub-targets in the historical trajectory dimension.

[0116] For example, based on historical navigation information, a set of points consisting of historical trajectory points is determined as the target point cloud S in the historical trajectory dimension. hist And the candidate waypoint p is calculated according to the aforementioned formulas (1)-(4). i Historical avoidance parameters .

[0117] By using historical avoidance parameters, it is possible to avoid repeatedly visiting already explored areas during navigation.

[0118] S605, calculate the sum of the direction guidance parameters, node guidance parameters, frontier exploration parameters, and historical avoidance parameters, as the target action parameters for candidate waypoints.

[0119] For example, the sum of the direction guidance parameters, node guidance parameters, frontier exploration parameters, and historical avoidance parameters of the candidate waypoint is calculated as the target actionable parameters of the candidate waypoint.

[0120] In exploration mode, by using directional guidance parameters, node guidance parameters, frontier exploration parameters, and historical avoidance parameters, the target action parameters of candidate waypoints are calculated. This allows for the expansion of cognitive boundaries while achieving a balance between high-level semantics and low-level geometry, avoiding over-reliance on a single information source, and guiding the robot to conduct multi-dimensional exploration. This balances exploration efficiency, target proximity, and safety performance. Furthermore, it enhances the robustness of the exploration process.

[0121] The above describes the process of determining the target action parameters of candidate waypoints when the waypoint determination mode is exploration mode. The following describes the process of determining the target action parameters of candidate waypoints when the waypoint determination mode is non-exploration mode.

[0122] In one possible implementation, Figure 7 This is a flowchart illustrating the calculation of target actionable parameters for each candidate waypoint in the robot mobility control method provided in this application embodiment, with reference to... Figure 7 As shown, in S302 above, based on node recommendation information, waypoint determination mode, and preferred direction of movement, the target action parameters for each candidate waypoint are calculated, including: S701. If the waypoint determination mode is non-exploration mode, determine the direction guidance parameters of the candidate waypoints based on the preferred direction of movement.

[0123] Optionally, if the waypoint determination mode is non-exploration mode, the directional guidance parameters of the candidate waypoints are calculated based on the preferred direction of movement. These directional guidance parameters are the actionable parameters of the sub-targets in the directional dimension.

[0124] For example, according to the preferred direction of motion d VLM Determine the candidate point cloud set along the preferred motion direction d. VLM The point set is used as the target point cloud S in the direction dimension. dir And the candidate waypoint p is calculated by referring to the aforementioned formulas (1)-(4). i Directional guidance parameters .

[0125] By using directional guidance parameters, the visual reasoning ability of a visual language model can be used to identify possible directions to the target during navigation.

[0126] S702. Determine the proximity guidance parameters for candidate waypoints.

[0127] Optionally, proximity guidance parameters can be determined based on the candidate waypoints. These proximity guidance parameters are the actionable parameters of the sub-targets in the semantic proximity dimension.

[0128] In one example, based on the current object point cloud in the pre-constructed current semantic point cloud, a set of point clouds with the same category as the target object is determined, and this set is used as the target point cloud S. sem And the candidate waypoint p is calculated by referring to the aforementioned formulas (1)-(4). i Proximity guidance parameters for candidate waypoints The determination of the current object's point cloud can be implemented with reference to S902.

[0129] In another example, a set of point clouds of the same category as the target object can be searched using the candidate waypoints as centers and within a preset search radius, and this set can be used as the target point cloud S. sem And the candidate waypoint p is calculated by referring to the aforementioned formulas (1)-(4). i Proximity guidance parameters for candidate waypoints .

[0130] By using proximity guidance parameters, the robot can be guided to precisely approach the identified target instance during navigation.

[0131] S703. Calculate the sum of the directional guidance parameters and the neighboring guidance parameters, and use them as the target action parameters for the candidate waypoints.

[0132] Optionally, the sum of the directional guidance parameters and the neighboring guidance parameters is calculated as the target actionable parameters for the candidate waypoints.

[0133] In non-exploration mode, actionable parameters for candidate waypoints are calculated using directional and proximity guidance parameters. This prevents the robot from exploring irrelevant areas during navigation once the target has been visually confirmed. Simultaneously, it guides the agent closer to the confirmed target instance, achieving precise localization. It also reduces computational complexity and accelerates waypoint selection. Furthermore, retaining directional guidance parameters ensures a smooth transition from exploration to target acquisition, maintaining navigation continuity and preventing abrupt directional changes.

[0134] In one possible implementation, Figure 8 This is a flowchart illustrating the process of determining the environmental memory topology information at the current time step in the robot mobility control method provided in this application embodiment, with reference to... Figure 8 As shown, determining the environmental memory topology information for the current time step in S102 includes: S801. Determine the current semantic point cloud based on the robot's image information and attitude information at the current waypoint.

[0135] Optionally, responding to the robot reaching the current waypoint corresponding to the current time step includes: when the robot reaches the position corresponding to the current waypoint, determining whether the robot meets the preset pose convergence conditions according to the preset pose convergence conditions; if so, determining that the robot has reached the current waypoint corresponding to the current time step. The preset pose convergence conditions may include at least one of the following: position error < 5 cm, orientation error < 3°, angular velocity / linear velocity < threshold, and completion of sensor data acquisition synchronization, etc.

[0136] Optionally, a sensing system can be used to acquire image and attitude information of the robot at the current waypoint. The image information includes RGB images and depth images. The attitude information includes: position p t =(x, y, z) and orientation q t Among them, the direction towards P t These can be represented using Euler angles (roll, pitch, yaw) or quaternions. Roll is the roll angle, pitch is the pitch angle, and yaw is the yaw angle. The quaternion consists of (w, x', y', z'), where w is a scalar, x' is the x-axis vector in 3D space, y' is the y-axis vector in 3D space, and z' is the z-axis vector in 3D space.

[0137] Optionally, after obtaining the image information and attitude information, the current semantic point cloud can be constructed based on the robot's image information and attitude information at the current waypoint, using the current waypoint as the spatiotemporal anchor point.

[0138] The current semantic point cloud includes multiple spatially registered sub-point clouds. Specifically, each point cloud is an independent topological entity, and the sub-point clouds share physical addressing, geometric origin, and semantic heterogeneity. Each sub-point cloud originates from the same observation but carries functions under different task-specific semantic dimensions.

[0139] In one example, after obtaining image information and pose information, each sub-point cloud can be constructed based on the image information and pose information according to a pre-configured construction strategy for each sub-point cloud, and used as the current semantic point cloud.

[0140] In one example, after obtaining image information and pose information, the sub-point clouds of the previous time step can be incrementally updated online according to the image information and pose information and the pre-configured generation strategy of each sub-point cloud to obtain the sub-point clouds of the current time step, which are used as the current semantic point cloud.

[0141] The current semantic point cloud is a collection of multiple sub-point clouds that are spatially registered in a unified global coordinate system and have clear semantic divisions. For example, the multiple sub-point clouds may include one or more of the following: scene base point cloud, object point cloud, navigable point cloud, front edge point cloud, interactive intent point cloud, illumination-sensitive point cloud, and memory confidence point cloud.

[0142] Specifically, the scene base point cloud represents the main structure of the robot's environment, serving as a geometric reference skeleton during navigation. The object point cloud represents identifiable or interactive dynamic and semi-static objects in the robot's environment, providing category information about objects. The navigable point cloud represents the actually safe traversable areas in the robot's environment, serving as feasibility input for path planning. The leading edge point cloud represents the boundary lines or points between explored and unexplored areas in the robot's environment, providing exploration targets for the robot. The interaction intent point cloud represents the surface area of ​​the object most likely to point to or request interaction in the current field of view. The illumination-sensitive point cloud represents areas in the robot's environment where visual features are easily lost due to changes in illumination. The memory confidence point cloud represents the stability of each spatial point in the robot's environment in long-term memory.

[0143] In one example, a pre-trained open vocabulary detection model, combined with task instructions, can be used to locate at least one pixel region of the noun phrase in the task instructions within an RGB image. Each pixel region is then back-projected into 3D space, registered with the current scene's base point cloud, and interactive weights are assigned to overlapping points, thereby generating an interactive intent point cloud. The open vocabulary detection model could, for example, include the GroundingDINO model.

[0144] In another example, the local contrast variance of each pixel in the RGB image can be calculated and combined with a preset illuminance map to identify low-illuminance or high-light overflow areas, thereby obtaining at least one target area. The pixels in each target area are then back-projected into 3D points to obtain a light-sensitive point cloud.

[0145] S802. Determine the environmental memory topology information for the current time step based on the current waypoint and the current semantic point cloud.

[0146] Optionally, after obtaining the current semantic point cloud, it can be dynamically constructed and updated using the current waypoint and the current semantic point cloud to obtain the environmental memory topology information of the current time step, thereby capturing the scene connection relationship, adjacency structure and semantic information in the environment through the environmental memory topology information of the current time step.

[0147] The environmental memory topology information includes multiple nodes and their attribute information. Each node represents a spatial point in the robot's environment, and its attribute information indicates the semantic features of the spatial point. Specifically, the essence of environmental memory topology information is spatial memory. The attribute information of each node refers to the topological description data representing the semantic state of the surrounding environment of that spatial location, obtained by the robot using the spatial location corresponding to that node as the query origin and performing spatial neighborhood retrieval and statistical analysis on the constructed current semantic point cloud.

[0148] Optionally, the environmental memory topology information may also include the edges corresponding to each node and the robot's historical movement node sequence, wherein the edges corresponding to each node are used to represent the robot's explored paths, and the historical movement node sequence is used to represent the robot's movement trajectory.

[0149] For example, taking the first time step as an example, when the robot arrives at the first time step, the first node can be created, and the attribute information of the first node can be generated based on the current semantic point cloud of the first waypoint.

[0150] For example, the attribute information of each node includes at least: identifier, location information, neighboring object category information, room type, and number of frontier points. The attribute information of each node may also include: neighboring point cloud confidence.

[0151] Specifically, location information refers to the node's three-dimensional spatial coordinates and orientation in the global coordinate system. Neighborhood object category information refers to the statistical distribution of the semantic categories to which all points in the current object point cloud belong within a preset radius *r* centered on the node. Room type refers to the high-level semantic discrimination result of the functional area type of the physical space to which the current waypoint belongs. Number of leading-edge points refers to the total number of leading-edge points in the current leading-edge point cloud contained within the current node's neighborhood, and is a scalar integer. Neighborhood point cloud confidence is a quantitative assessment value of the quality and consistency of point cloud data within the neighborhood centered on the current node, reflecting the reliability of the memory in that area.

[0152] In one example, the category information of neighboring objects can be determined by the current object point cloud.

[0153] In another example, the voxel coverage density in the neighborhood of a node can be determined by the current scene base point cloud; the multi-view observation consistency in the neighborhood of a node can be determined by the current object point cloud; the proportion of interpolation points in the neighborhood of a node can be determined by the current navigable point cloud; and a weighted fusion is performed based on the voxel coverage density in the neighborhood of a node, the multi-view observation consistency in the neighborhood of a node, and the proportion of interpolation points in the neighborhood of a node to obtain the confidence of the neighborhood point cloud of a node. Thus, by embedding the memory quality assessment mechanism into the topological node, the robot has the metacognitive ability to "know where it is uncertain," enabling the decision-making process to dynamically reduce the weight of low-confidence nodes.

[0154] By utilizing image and pose information, the current semantic point cloud is determined. Based on the current waypoint and the current semantic point cloud, the environmental memory topology information for the current time step is determined. By binding waypoints, semantic point clouds, and environmental memory topology information, the robot's instantaneous observations at the current time step can be transformed into structured environmental memory information, forming an evolvable environmental cognition model. This enables the continuous accumulation of long-term spatial structure knowledge, online mapping of unknown environments, and solves the problems of fragmented spatial memory and lack of long-term structural knowledge accumulation in existing technologies. Furthermore, through this globally structured memory of environmental memory topology information, the next waypoint of the current waypoint can be determined, and the robot can be controlled to move to the next waypoint. This reduces path drift caused by single-frame false detections or occlusions and avoids getting lost due to local observation failures, thereby improving the robustness and efficiency of navigation decisions. At the same time, it increases the success rate in long-distance tasks.

[0155] Furthermore, in practical applications, there is no need for offline training for specific scenarios or for predefined prior knowledge related to the environment. The robot can start navigation directly when it arrives in an unknown environment, which greatly reduces the adaptation cost of engineering deployment and can achieve perfect adaptation in zero-sample navigation scenarios.

[0156] In one possible implementation, Figure 9 This is a flowchart illustrating the process of determining the current semantic point cloud in the robot movement control method provided in this application embodiment, with reference to... Figure 9 As shown, the current semantic point cloud includes: the current scene base point cloud, the current object point cloud, the current navigable point cloud, the current obstacle point cloud, and the current leading edge point cloud; in S801 above, the current semantic point cloud is determined based on the robot's image information and attitude information at the current waypoint, including: S901. Based on the image information and posture information, incrementally update the historical scene base point cloud of the robot in the previous time step to obtain the current scene base point cloud.

[0157] Optionally, the scene point cloud to be added can be determined based on the image information and pose information, and the scene point cloud to be added can be added to the historical scene base point cloud of the previous time step to obtain the initial scene base point cloud of the current time step. The initial scene base point cloud of the current time step can be optimized to obtain the current scene base point cloud.

[0158] Optionally, the incremental scene point cloud at the current time step can be determined based on image information, posture information, and the historical scene base point cloud of the previous time step. Based on the incremental scene point cloud, the robot's historical base point cloud at the previous time step can be incrementally updated to obtain the current scene base point cloud.

[0159] Among them, the current scene base point cloud refers to the scene base point cloud at the current time step.

[0160] S902. Based on the image information and posture information, incrementally update the historical object point cloud of the robot in the previous time step to obtain the current object point cloud.

[0161] Optionally, at least one object at the current time step can be identified based on image information and pose information, and the historical object point cloud of the previous time step can be incrementally updated based on each object to obtain the current object point cloud.

[0162] Optionally, the current object point cloud can be obtained by filtering from the current scene base point cloud according to a preset object category directory.

[0163] For example, a pre-trained semantic segmentation model can perform semantic segmentation on RGB images in image information based on a preset object category directory, and identify multiple categories of objects in the RGB image and the regions where each category of objects is located.

[0164] For example, the objects of each category are traversed. For the object of the current category, based on the region where the object of the current category is located, multiple corresponding pixels of the object of the current category in the depth image are determined. Then, each corresponding pixel is transformed into the camera coordinate system according to the robot's camera intrinsic parameter matrix to obtain multiple second spatial points corresponding to the depth image. Specifically, the process of transforming each corresponding pixel into the camera coordinate system according to the robot's camera intrinsic parameter matrix to obtain multiple second spatial points corresponding to the depth image can be implemented with reference to the following formula (9).

[0165] For example, after obtaining multiple second spatial points corresponding to the depth image, color data is added to each second spatial point according to the RGB image. Specifically, it can be implemented with reference to the aforementioned S901.

[0166] For example, after obtaining each second spatial point with color attributes, the second spatial points are subjected to rigid body transformation according to the posture information, and the second spatial points are transformed to the global coordinate system to obtain multiple object spatial points. Specifically, the object spatial points can be calculated with reference to the following formula (10).

[0167] For example, after obtaining the spatial points of each object, the second incremental point cloud of each object spatial point and the historical object point cloud is determined, and the second incremental point cloud is updated to the historical object point cloud according to the pre-trained voxel mesh model to obtain the current object point cloud.

[0168] Specifically, the point cloud of objects in the c-th category The following formula (5) can be used as a reference: (5) in, For the point cloud of the c-th category, VoxelDownsample is a voxel mesh model. For the j-th object point of the c-th category, r pcd This represents the resolution of the voxel mesh model.

[0169] S903. Based on the current scene base point cloud and the current object point cloud, update the robot's historical navigable point cloud at the previous time step to obtain the current navigable point cloud.

[0170] Optionally, after obtaining the current scene base point cloud, semantic construction is performed based on the current scene base point cloud and the current object point cloud, and the historical navigable point cloud of the robot in the previous time step is updated to obtain the current navigable point cloud.

[0171] For example, geometric filtering is performed on the current scene base point cloud to obtain a first navigable point cloud set, and semantic enhancement is performed on the current object point cloud to obtain a second navigable point cloud set. Based on the first and second navigable point cloud sets, the robot's historical navigable point cloud at the previous time step is updated to obtain the current navigable point cloud. This ensures that no possible ground areas are missed while ensuring that special structures are correctly understood and processed, thereby enabling the robot to navigate safely and intelligently in the real world.

[0172] S904. Determine the current obstacle point cloud based on the current scene base point cloud.

[0173] Optionally, at least one candidate point cloud is extracted from the current scene base point cloud according to a preset first height threshold. The first height threshold can be Z... floor +δ。 Z floor δ is the preset ground plane elevation, and δ is the preset height error threshold.

[0174] Optionally, spatial clustering is performed on each candidate point cloud to obtain multiple spatially connected point clusters, thereby classifying spatially connected points into the same obstacle species. Spatial clustering includes density clustering based on Euclidean distance.

[0175] Optionally, the obstacle type of each spatial connected point cluster is determined by a preset obstacle type identification condition, and the obstacle type determination result of each spatial connected point cluster is obtained. Based on the obstacle type determination result, at least one obstacle sub-point cloud is extracted from each spatial connected point cluster.

[0176] For example, the preset obstacle type recognition conditions may include: discrete obstacle recognition conditions and vertical obstacle recognition conditions.

[0177] The obstacle recognition conditions for discrete types are: minimum bounding box height H ≥ 0.1m and base area S ≤ 0.5m². Specifically, the minimum bounding box height and base area of ​​the spatially connected point clusters are calculated and judged. If the discrete obstacle recognition conditions are met, all point clouds in the corresponding spatially connected point cluster are treated as obstacle sub-point clouds, thereby ensuring that the robot avoids collisions when facing discrete obstacles.

[0178] The condition for recognizing vertical obstacles is that the normal vector is perpendicular to gravity. Specifically, the normal vector of the spatial connected point cluster is obtained through PCA analysis, and the angle between the normal vector and the direction of gravity is calculated. If the angle between the normal vector and the direction of gravity is less than a preset angle range, it is determined to be a vertical obstacle. The upper edge points of the corresponding spatial connected point cluster are extracted and used as obstacle sub-point clouds, thereby ensuring that the robot defines a passable boundary when facing vertical obstacles.

[0179] Optionally, the union of each obstacle sub-point cloud is calculated, and voxel downsampling and denoising are performed using a pre-trained voxel mesh model to generate the current obstacle point cloud Mobs.

[0180] For example, the current obstacle point cloud Mobs can be generated by voxel downsampling and denoising using a pre-trained voxel mesh model, as shown in the following formula (11).

[0181] By using a first height threshold, at least one candidate point cloud is extracted from the current scene base point cloud. By determining the obstacle type, geometric feature-driven intelligent classification can be achieved, enabling differentiated representation of different types of obstacles. This can significantly reduce the amount of point cloud data while ensuring navigation safety, and at the same time provide clearer obstacle boundary information for path planning.

[0182] S905. Determine the current leading edge point cloud based on the current scene base point cloud, the current navigable point cloud, and the current obstacle point cloud.

[0183] Optionally, based on the current scene base point cloud, a discrete continuous grid space corresponding to the current scene base point cloud is generated. In the discrete continuous grid space corresponding to the current scene base point cloud, the grid state of each grid in the discrete continuous grid space is determined by the current navigable point cloud and the current obstacle point cloud. Based on the grid state of each grid, the neighborhood relationship of each grid is determined. Based on the neighborhood relationship of each grid, the current frontier point cloud is generated. This achieves efficient spatial reasoning while maintaining semantic information, completes boundary detection with linear time complexity, and naturally obtains noise resistance and smoothness through the discretization characteristics of the grid. Ultimately, it supports real-time, incremental exploration planning.

[0184] For example, the continuous space can be divided into Nx segments based on the spatial boundaries of the current scene's base point cloud. There are Ny grids, each with a size of rg. rg. Specifically, it can be determined by referring to the following formula (6): (6) Where, x min The minimum x-coordinate of the spatial boundary of the current scene's base point cloud.max The maximum x-coordinate of the spatial boundary of the current scene's base point cloud, y min The minimum y-coordinate of the spatial boundary of the current scene's base point cloud. max This represents the maximum y-coordinate of the spatial boundary of the current scene's base point cloud.

[0185] For example, after obtaining each grid, the point clouds in the current navigable point cloud and the current obstacle point cloud are sequentially projected into each grid to obtain the grid state of each grid.

[0186] Specifically, iterate through all point clouds in the current navigable point cloud and the current obstacle point cloud. For the current point cloud p that has been traversed, determine the corresponding grid of the current point cloud p based on the horizontal and vertical coordinates of the current point cloud p, and determine the grid state of the corresponding grid based on the type of the current point cloud p.

[0187] For example, for the current point cloud p, the mesh φ(p) corresponding to the current point cloud p can be obtained by referring to the following formula (7): (7) Where px is the x-coordinate of the current point cloud and py is the y-coordinate of the current point cloud.

[0188] Specifically, if the current point cloud p is a navigable point cloud, the grid state of the corresponding grid is determined to be navigable; if the current point cloud p is an obstacle point cloud, the grid state of the corresponding grid is determined to be obstacle.

[0189] Specifically, after traversing all point clouds in the current navigable point cloud and the current obstacle point cloud, all grids in the non-navigable state and the non-obstacle state are determined to be in the unknown state.

[0190] For example, for grids whose grid state is unknown, a preset boundary grid judgment condition is used to determine whether the grid state of each grid is a boundary state, thereby obtaining each grid in a boundary state. The boundary grid judgment condition includes: the grid is not in a navigable state, is not in an obstacle state, is adjacent to at least one grid in a navigable state, and is not directly adjacent to a grid in an obstacle state.

[0191] For example, after obtaining each mesh of the boundary state, the boundary points can be generated by projecting each mesh of the boundary state, thereby obtaining the current frontier point cloud Mfro. Specifically, it can be obtained by referring to the following formula (8): (8) in, The first in the current cutting-edge point cloud Mfro The frontier point cloud, (i k i k ) is the first The coordinates of the grid at each boundary state, Z floor x is the preset ground plane elevation. min The minimum x-coordinate of the spatial boundary of the current scene's base point cloud, y min The minimum y-coordinate of the spatial boundary of the current scene's base point cloud.

[0192] By using image information, pose information, and the current scene base point cloud, the current object point cloud, current navigable point cloud, current obstacle point cloud, and current frontier point cloud are determined separately. Each object point cloud is responsible for a core concern: the current object point cloud focuses on the semantic target, the current navigable point cloud focuses on the feasible region, the current obstacle point cloud focuses on the safety boundary, and the current frontier point cloud focuses on the exploration direction. This separation of concerns at the architectural level avoids redundant and ineffective computations, reducing complexity from O(N^4) to O(4N). Furthermore, at the semantic level, it achieves intelligence through hierarchical understanding, security through multi-layered protection, real-time performance through incremental updates, interpretability through layered visualization, performance through space-for-time tradeoffs, and scalability through plug-and-play functionality.

[0193] In one possible implementation, the image information includes: an RGB image and a depth image. In step S901 above, based on the image information and pose information, the robot incrementally updates the historical scene base point cloud from the previous time step to obtain the current scene base point cloud, including: Based on the depth image and the robot's camera intrinsic parameter matrix, multiple first spatial points corresponding to the depth image are determined; based on the RGB image, color attributes are added to each first spatial point; based on the pose information, each first spatial point is transformed to the global coordinate system to obtain multiple current spatial points; the first incremental point cloud of each current spatial point and the historical scene base point cloud is determined; based on the pre-trained voxel mesh model, the first incremental point cloud is updated to the historical scene base point cloud to obtain the current scene base point cloud.

[0194] Optionally, each pixel in the depth image can be transformed to the camera coordinate system based on the robot's camera intrinsic parameter matrix to obtain multiple first spatial points corresponding to the depth image.

[0195] For example, the first spatial point X c The following formula (9) can be used to determine this: (9) Where d=D t (u,v), D t Let u be the depth image, v be the x-coordinate of the depth image, and K be the camera intrinsic parameter matrix.

[0196] Optionally, color attributes can be added to each first spatial point based on the pixel values ​​of each pixel in the RGB image.

[0197] The color attribute of the first spatial point can be its color value. For example, the first spatial point X... c Color value c = I t (u,v). Among them, I t It is an RGB image.

[0198] Optionally, after obtaining each first spatial point with color attributes, the first spatial points are subjected to rigid body transformation according to the attitude information, and the first spatial points are transformed to the global coordinate system to obtain multiple current spatial points.

[0199] For example, the current spatial point X w The following formula (10) can be used to determine this: (10) Where, q t For orientation, R(q) t ) represents the rotation matrix corresponding to the orientation, p t For position, X c This is the first spatial point.

[0200] Optionally, after obtaining each current spatial point, it can be determined whether each current spatial point falls within the covered voxels of the historical scene base point cloud. If not, the current spatial point is determined as the first incremental point cloud, thereby retaining only the newly added part of the environment, eliminating duplicate observations, and significantly compressing the amount of data.

[0201] Optionally, after obtaining the first incremental point cloud, the first incremental point cloud is updated to the historical scene base point cloud according to the pre-trained voxel mesh model to obtain the current scene base point cloud, so as to reduce the point cloud density, improve the calculation effect, eliminate measurement noise, and at the same time maintain the integrity of geometric features.

[0202] For example, the scene base point cloud Mscene can be obtained by referring to the following formula (11): (11) Where Mscene is the scene base point cloud, VoxelDownsample is the voxel mesh model, and X... w For the current spatial point, r pcd This represents the resolution of the voxel mesh model.

[0203] By determining the first incremental point cloud of each current spatial point and the historical scene base point cloud, the first incremental point cloud is updated to the historical scene base point cloud through a pre-trained voxel mesh model to obtain the current scene base point cloud. This allows only the incremental point cloud to be injected in the current time step, naturally suppressing repeated mapping and redundant storage. At the same time, updating through the voxel mesh model can establish a long-term scene base memory with temporal continuity and spatial consistency while ensuring geometric accuracy.

[0204] In one possible implementation, S902 updates the robot's historical navigable point cloud at the previous time step based on the current scene base point cloud and the current object point cloud to obtain the current navigable point cloud, including: According to the preset navigable height range, at least one first base point is extracted from the current scene base point cloud; at least one second base point is selected from the current object point cloud; each first base point and each second base point is used as a base increment point, and each base increment point is traversed. For the current base increment point, it is determined whether to generate an interpolation point corresponding to the base increment point based on the position of the current base increment point. If so, the base increment point is interpolated to generate at least one interpolation point corresponding to the base increment point; according to the pre-trained voxel mesh model, each base increment point and the interpolation point corresponding to each base increment point are updated to the historical navigable point cloud to obtain the current navigable point cloud.

[0205] Optionally, at least one first base point can be extracted from the current scene base point cloud according to a preset navigable height range.

[0206] For example, the navigable altitude range can be determined by a preset ground elevation Z. floor The navigable altitude range is obtained by comparing the altitude with a preset altitude error threshold δ. Specifically, the navigable altitude range can be [Z]. floor -δ,Z floor +δ].

[0207] Optionally, the current object point cloud is filtered to select the point cloud corresponding to special passable structures in the current object point cloud to obtain at least one second base point.

[0208] In one example, the robot also stores navigable labels for objects of various categories. These navigable labels indicate whether an object of a particular category is a navigable, accessible structure. For example, the navigable label for an object of the "stairs" category is "navigable".

[0209] Specifically, from the current object point cloud, according to the navigable labels of various object categories, the corresponding point cloud is searched to obtain at least one second base point.

[0210] In another example, the current object point cloud can be identified by a pre-trained recognition model to determine whether each point cloud is a navigable point. If so, it is used as a second base point.

[0211] Optionally, each first base point and each second base point are treated as a base increment point, and each base increment point is traversed. For the current base increment point, a distance judgment is made based on the position corresponding to the current base increment point to determine whether to generate an interpolation point corresponding to the base increment point. If so, the base increment point is interpolated to generate at least one interpolation point corresponding to the base increment point.

[0212] For example, the current distance is calculated based on the position of the current basic increment point and the robot's current position. It is then determined whether the current distance is greater than a preset step size threshold. If so, the interpolation point corresponding to the basic increment point is determined, thereby solving the problem of limited sensor field of view and creating a continuous passable surface. In addition, under this processing method, even if a wall blocks part of the area, a passable path can be inferred, improving the success rate of navigation.

[0213] Specifically, regarding the current basic incremental point X i You can refer to the following formula (12) to determine whether the current distance is greater than the preset step size threshold: (12) Among them, X i As the current base increment point, P stand Let Δ be the robot's current location. step This is the preset step size threshold.

[0214] For example, the interpolation point P k It can be generated by referring to the following formula (13): (13) Among them, P k X is the interpolation point. i As the current base increment point, P stand Let Δ be the robot's current location. step This is the preset step size threshold.

[0215] Optionally, after obtaining all interpolation points, all interpolation points can be filtered to retain only those whose height is still within the navigable height range, thus obtaining the interpolation points corresponding to each basic incremental point.

[0216] Optionally, based on the pre-trained voxel mesh model, each basic increment point and the interpolation point corresponding to each basic increment point are updated to the historical navigable point cloud to obtain the current navigable point cloud, thereby reducing the point cloud density, improving the computational effect, eliminating measurement noise, and maintaining the integrity of geometric features.

[0217] For example, each basic increment point and its corresponding interpolation point are added to the historical navigable point cloud, and the historical navigable point cloud is uniformly downsampled using a pre-trained voxel grid model to obtain the current navigable point cloud. Specifically, the navigable point cloud Mnav can be obtained by referring to the following formula (14): (14) Where Mnav is a navigable point cloud, VoxelDownsample is a voxel mesh model, and r pcd This represents the resolution of the voxel mesh model.

[0218] By geometrically filtering the current scene's base point cloud, multiple first base points are obtained. Height constraints ensure complete coverage of ordinary ground areas. By semantically enhancing the current object point cloud, multiple second base points are obtained. Semantic information allows special structures such as stairs and ramps to be incorporated into the navigable space, solving the problem that a single method cannot cope with the complexity of real-world environments. Interpolation fills the gaps in sensor observations, constructing a continuous, navigable surface. Voxel downsampling reduces data density and noise while maintaining geometric features. The resulting navigable point cloud not only provides a continuous and uniform spatial foundation for path planning but also supports adaptive navigation strategies through the semantic information of point origins. This enables safe and efficient navigation decisions in complex dynamic environments.

[0219] In one possible implementation, in S802 above, the environmental memory topology information of the current time step is determined based on the current waypoint and the current semantic point cloud.

[0220] Create a new node corresponding to the current waypoint, and determine the attribute information of the new node based on the current waypoint and the current semantic point cloud; add the new node and its attribute information to the robot's environmental memory topology information in the previous time step to obtain the environmental memory topology information of the current time step.

[0221] Optionally, a new node corresponding to the current waypoint is created, and attribute information of the new node is generated based on the current waypoint and the current semantic point cloud.

[0222] In one example, the first node can be created when the robot reaches the first waypoint corresponding to the first time step.

[0223] In another example, when the robot reaches a waypoint that is not the first time step, a new node can be created based on the previous node. Specifically, the identifier of the new node is obtained by incrementing the identifier of the previous node.

[0224] For example, after creating a new node, the location information of the new node is determined based on the location of the current waypoint, the neighborhood object category information of the new node is determined based on the location of the current waypoint and the current object point cloud in the current semantic point cloud, the number of leading points of the new node is determined based on the current leading point cloud, and the room type of the new node is determined based on the current object point cloud.

[0225] Optionally, the newly added node and its attribute information can be added to the robot's environmental memory topology information in the previous time step to obtain the environmental memory topology information in the current time step.

[0226] For example, when the robot reaches the first waypoint corresponding to the first time step, the newly added node and its attribute information can be used as the environmental memory topology information of the current time step; when the robot reaches a non-first waypoint corresponding to a non-first time step, the newly added node and its attribute information are added to the environmental memory topology information of the robot in the previous time step to obtain the environmental memory topology information of the current time step.

[0227] By using the current waypoint and the current semantic point cloud, the attribute information of the newly added node is determined, and the newly added node and its attribute information are added to the environmental memory topology information of the robot in the previous time step to obtain the environmental memory topology information of the current time step. This enables the robot to learn in exploration and reason in memory, solving the problems of target confusion and path redundancy caused by memory fragmentation in long-term navigation.

[0228] In one possible implementation, the attribute information of the newly added node includes: location information, neighbor object category information, room type, and number of frontier points. The above steps determine the attribute information of the newly added node based on the current waypoint and the current semantic point cloud, including: Based on the current waypoint, determine the location information; based on the location information and the current object point cloud in the current semantic point cloud, determine the neighboring object category information; acquire the current panoramic image corresponding to the current waypoint, and determine the room type based on the current panoramic image; based on the location information and the current front point cloud in the current semantic point cloud, determine the number of front points.

[0229] Optionally, the position information of the new node can be determined based on the attitude information of the current waypoint.

[0230] For example, the position p in the attitude information of the current waypoint tTransform (x, y, z) to the global coordinate system and omit the z-axis to obtain n. k =(x k , y k ), which serves as the location information for the newly added node k.

[0231] Optionally, with the position of the newly added node k as the center and the preset first perception radius r as the radius, the objects around the newly added node k are identified from the current object point cloud in the current semantic point cloud to obtain the neighborhood object category information.

[0232] For example, neighborhood object category information may include {chair, table}.

[0233] By using location information and the current object point cloud in the current semantic point cloud, the category information of neighboring objects can be determined, which can ensure that the semantic information is strongly correlated with the robot's current decision pose, avoid semantic ambiguity caused by global statistics, and realize atomic-level binding between spatial topological nodes and local semantic fields, thereby improving navigation accuracy.

[0234] It is understandable that image information can be obtained at each waypoint. Therefore, the image information of multiple historical waypoints and the current waypoint can be fused to obtain the current panoramic image corresponding to the current waypoint.

[0235] Optionally, the current panoramic image is input into a pre-trained visual language model, which identifies the room type of the newly added node. Only the current panoramic image and the text prompt need to be input to identify the most matching semantic label, thereby improving the zero-shot open vocabulary generalization ability in the navigation process and also improving robustness.

[0236] Optionally, the number of leading points can be calculated based on location information, a preset second perception radius, and the current leading point cloud in the current semantic point cloud.

[0237] For example, the number of front points f k The following formula (15) can be used to obtain the result: (15) Among them, f k Let Mfro be the number of frontier points, and Mfro be the current frontier point cloud. topo The second sensing radius is a preset value, which can be the same as the first sensing radius r, n k Provide the location information for the newly added node k.

[0238] By determining the number of leading points using location information and the current leading point cloud in the current semantic point cloud, discrete geometric features can be transformed into continuous and computable topological state variables, providing a new dimension of input for intelligent decision-making. This enables robots to have metacognitive abilities such as "knowing where they are, how much is unknown around them, and whether it is worthwhile to continue exploring," thereby improving navigation accuracy.

[0239] In one possible implementation, the above steps add the newly added node and its attribute information to the robot's environmental memory topology information from the previous time step to obtain the environmental memory topology information for the current time step, including: Add the newly added node and its attribute information to the robot's environmental memory topology information in the previous time step to obtain the current candidate environmental memory topology information; determine whether to merge nodes in the current candidate environmental memory topology information; if yes, merge nodes in the current candidate environmental memory topology information to obtain the environmental memory topology information for the current time step; if no, use the current candidate environmental memory topology information as the environmental memory topology information for the current time step.

[0240] Optionally, the newly added node and its attribute information can be added to the robot's environmental memory topology information in the previous time step to obtain the current candidate environmental memory topology information.

[0241] Optionally, all nodes in the current candidate environment memory topology information, except for the newly added node, are traversed. For the current node that is traversed, it is determined whether to merge the newly added node and the current node in the current candidate environment memory topology information based on the distance between the newly added node and the current node and the relative relationship between the newly added node and the current node.

[0242] For example, if the spatial distance between the newly added node and the current node is less than a preset distance threshold, and there are no obstacles between the newly added node and the current node, then it is determined to merge the newly added node and the current node.

[0243] Specifically, it can be obtained by referring to the following formula (16): (16) Where, n i For the location information of the newly added node i, n j d represents the position information of the current node j. merge The distance threshold is set to a preset value, Mobs represents the current obstacle point cloud, and Z represents the distance threshold. floor Z is the preset ground level elevation. ceiling For the preset ceiling elevation, z w For point cloud X w The height value, L (n i , n j) represents the straight-line path between the newly added node i and the current node j.

[0244] By measuring the distance between the newly added node and the current node, as well as their relative relationship, we can determine whether to merge the newly added node and the current node in the current candidate environment's memory topology information. This method can quickly filter out obviously irrelevant distant nodes, ensuring computational efficiency, and can also completely avoid erroneous merging of geometrically adjacent but semantically conflicting nodes, making the merging result cognitively reasonable and task-safe.

[0245] Optionally, if it is determined to merge the current node and the newly added node, then the current node and the newly added node are merged according to the attribute information of the current node and the attribute information of the newly added node to obtain the merged node and the attribute information of the merged node. Then, the current node, the newly added node, the attribute information of the current node and the attribute information of the newly added node are deleted from the current candidate environment memory topology information, and the merged node and the attribute information of the merged node are added.

[0246] For example, the identifiers of the current node and the newly added node are compared, and the identifier of the node that was created earlier is retained as the identifier of the merged node.

[0247] For example, the two-dimensional centroids of the current node and the newly added node are calculated based on the position information of the current node and the position information of the newly added node, and used as the position information of the merged node.

[0248] For example, based on the location information of the merged node, the neighbor object category information of the merged node is recalculated with reference to the aforementioned S802.

[0249] For example, the room type of the earlier created node is retained as the room type of the merged node.

[0250] For example, based on the location information of the merged nodes, the number of leading edge points of the merged nodes is recalculated with reference to the aforementioned S804.

[0251] Optionally, if it is determined not to merge, the current candidate environment memory topology information is used as the environment memory topology information for the current time step.

[0252] By merging nodes of the current candidate environment memory topology information, the environment memory topology information of the current time step is obtained. Through spatial distance-driven, attribute-aware, and configurable topology compression, it is possible to avoid memory explosion while ensuring that key semantic information is not lost.

[0253] This application also provides a robot, such as... Figure 10 As shown, Figure 10The schematic diagram of the robot provided in this application embodiment includes: a processor 1001 and a memory 1002, and optionally, a bus 1003. The memory 1002 stores machine-readable instructions executable by the processor 1001. When the robot is running, the processor 1001 and the memory 1002 communicate through the bus 1003, and the processor 1001 executes the machine-readable instructions to perform the steps of the robot movement control method described above.

[0254] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the robot movement control method described above.

[0255] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A robot movement control method, characterized in that, include: At the current time step, acquire the robot's current task information, the current panoramic image corresponding to the current waypoint, and the historical navigation information corresponding to the current waypoint; The environmental memory topology information of the current time step is determined, and the current task information, the current panoramic image, the historical navigation information and the environmental memory topology information are input into a pre-trained visual language model. The visual language model predicts node recommendation information, preferred motion direction and target state information. The target state information is used to indicate the completion state of the current task information in the current panoramic image. The environmental memory topology information is used to indicate the environmental semantics of the robot at the current time step. Based on the node recommendation information, the preferred movement direction, and the target state information, the next waypoint of the current waypoint is determined, and the robot is controlled to move to the next waypoint.

2. The robot movement control method according to claim 1, characterized in that, Determining the next waypoint of the current waypoint based on the node recommendation information, the preferred direction of movement, and the target state information includes: Based on the target status information, determine the waypoint determination mode for the next waypoint; The next waypoint is determined based on the node recommendation information, the waypoint determination mode, and the preferred direction of movement.

3. The robot movement control method according to claim 2, characterized in that, The step of determining the next waypoint of the current waypoint based on the node recommendation information, the waypoint determination pattern, and the preferred direction of movement includes: Determine at least one candidate waypoint corresponding to the current waypoint; Based on the node recommendation information, the waypoint determination mode, and the preferred direction of movement, the target action parameters of each candidate waypoint are calculated. Based on the target actionable parameters of each of the candidate waypoints, the next waypoint of the current waypoint is determined from each of the candidate waypoints.

4. The robot movement control method according to claim 3, characterized in that, The step of determining the next waypoint of the current waypoint from among the candidate waypoints based on the target traversability parameters of each candidate waypoint includes: Determine the hazard score of each of the candidate waypoints, and based on the hazard score of each of the candidate waypoints, select at least one optional waypoint from the candidate waypoints; Obtain the maximum value of the target actionable parameter among the available waypoints, and obtain the target waypoint corresponding to the maximum value, and use the target waypoint as the next waypoint of the current waypoint.

5. The robot movement control method according to claim 4, characterized in that, The determination of the hazard score for each of the candidate waypoints includes: Identify all obstacle points in the current panoramic image; Based on each of the candidate waypoints and all the obstacle points, the shortest distance between each of the candidate waypoints and all the obstacle points is calculated; Based on each of the candidate waypoints and all the obstacle points, the maximum and minimum distances from each candidate point to the obstacle are calculated. The hazard score of each candidate waypoint is calculated based on the shortest distance, the maximum distance, and the minimum distance relative to all obstacle points.

6. The robot movement control method according to claim 3, characterized in that, The step of calculating the target actionable parameters for each candidate waypoint based on the node recommendation information, the waypoint determination pattern, and the preferred direction of movement includes: If the waypoint determination mode is exploration mode, the directional guidance parameters of the candidate waypoints are determined according to the preferred direction of movement; Based on the node recommendation information, determine the node guidance parameters for the candidate waypoints; Based on the current frontier point cloud in the current semantic point cloud, determine the frontier exploration parameters of the candidate waypoints; Based on the historical navigation information, determine the historical avoidance parameters of the candidate waypoints; The sum of the direction guidance parameters, the node guidance parameters, the frontier exploration parameters, and the historical avoidance parameters is calculated as the target actionable parameters for the candidate waypoints.

7. The robot movement control method according to claim 3, characterized in that, The step of calculating the target actionable parameters for each candidate waypoint based on the node recommendation information, the waypoint determination pattern, and the preferred direction of movement includes: If the waypoint determination mode is non-exploration mode, the directional guidance parameters of the candidate waypoints are determined according to the preferred direction of movement; Determine the proximity guidance parameters of the candidate waypoints; The sum of the directional guidance parameters and the neighboring guidance parameters is calculated as the target actionable parameters for the candidate waypoints.

8. The robot movement control method according to claim 1, characterized in that, The determination of the environmental memory topology information at the current time step includes: Based on the image information and attitude information of the robot at the current waypoint, the current semantic point cloud is determined, and the current semantic point cloud includes sub-point clouds of multiple dimensions that are spatially registered. Based on the current waypoint and the current semantic point cloud, determine the environmental memory topology information for the current time step.

9. The robot movement control method according to claim 8, characterized in that, The current semantic point cloud includes: the current scene base point cloud, the current object point cloud, the current navigable point cloud, the current obstacle point cloud, and the current front edge point cloud; The step of determining the current semantic point cloud based on the robot's image information and attitude information at the current waypoint includes: Based on the image information and the posture information, the robot incrementally updates the historical scene base point cloud of the previous time step to obtain the current scene base point cloud. Based on the image information and the posture information, the robot incrementally updates the historical object point cloud of the previous time step to obtain the current object point cloud. Based on the current scene base point cloud and the current object point cloud, the robot's historical navigable point cloud at the previous time step is updated to obtain the current navigable point cloud; The current obstacle point cloud is determined based on the current scene base point cloud; The current front-line point cloud is determined based on the current scene base point cloud, the current navigable point cloud, and the current obstacle point cloud.

10. A robot, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when the robot is running, are executed by the processor to perform the steps of the robot motion control method as described in any one of claims 1 to 9.

Images