A machine character object navigation method based on target-oriented semantic exploration

Abstract

This invention relates to the field of robot navigation technology, specifically providing a robot object navigation method based on goal-oriented semantic exploration. The method includes: acquiring RGB-D images and poses; constructing a dynamically updated multi-channel semantic map through differentiable geometric projection; determining whether a reliable observation exists in the target object's channel on the map; if so, direct localization; otherwise, inputting the map, target category, and historical trajectory into a goal-oriented policy network trained by deep reinforcement learning; using prior spatial layout of observed objects to predict the unknown region where the unobserved target is located as a long-range target point; and finally, using deterministic path planning to generate a collision-free path and control the robot's movement. This invention achieves active semantic exploration, improves navigation efficiency and robustness, and can adapt to dynamic environments and occluded scenarios.

Description

Technical Field

[0001] This invention belongs to the field of robot navigation technology, specifically relating to a robot object navigation method based on goal-oriented semantic exploration. Background Technology

[0002] In the field of autonomous robot navigation, object-target navigation requires robots to autonomously navigate to instances of a given object category (such as a television) in an unknown environment. Existing methods are mainly divided into end-to-end reinforcement learning methods and modular methods based on geometry maps. While end-to-end methods can learn implicit policies, they suffer from low sample efficiency, poor generalization ability, and difficulty in supporting long-distance planning. Although geometry map-based methods can efficiently construct environment maps and plan paths, their maps only contain obstacle and free space information, lacking semantic understanding of object categories and their spatial layout.

[0003] Therefore, when the target object is not observed, the robot can only adopt a blind full-coverage exploration strategy. It cannot use the co-occurrence priors and spatial positional relationships between existing objects in the scene (such as sofas and TV cabinets) and the target object to infer the area where the target is most likely to appear, resulting in low exploration efficiency and excessive navigation time. Summary of the Invention

[0004] To address the aforementioned shortcomings of existing technologies, this invention provides a robot object navigation method based on goal-oriented semantic exploration to solve the aforementioned technical problems.

[0005] In a first aspect, the present invention provides a robot object navigation method based on goal-oriented semantic exploration, comprising: The robot acquires RGB-D images and pose information, and generates a top-down multi-channel semantic map through differentiable geometric projection. The multi-channel semantic map includes object channels corresponding to multiple preset object categories and at least one obstacle channel. If it is confirmed that there are no reliable observation points in the object channel corresponding to the target object category in the multi-channel semantic map that meet the preset confidence conditions, then the multi-channel semantic map, the target object category, and the robot's historical trajectory are input into the pre-trained target guidance policy network. The target guidance policy network predicts the location of the unknown area where the target object is located based on the prior knowledge of the category distribution and spatial layout of the observed objects in the multi-channel semantic map, and uses the unknown area location as the long-range target point. Based on the long-range target point and the obstacle channels in the multi-channel semantic map, a collision-free path is generated from the robot's current position to the long-range target point.

[0006] In one optional implementation, RGB-D images and pose information acquired by the robot are obtained, and a top-down multi-channel semantic map is generated through differentiable geometric projection, including: Perform instance segmentation on the color image in the RGB-D image and predict the semantic category of each pixel to obtain the semantic label of each pixel; A 3D point cloud is generated from the depth image in the RGB-D image, and the semantic label of each pixel is associated with each point in the 3D point cloud to generate a point cloud with semantic labels. Based on the pose information, the point cloud with semantic labels is projected onto a top-down two-dimensional grid map through differentiable geometric transformation. By accumulating the number of points in each grid cell whose semantic labels belong to different semantic categories, a multi-channel projected semantic map is generated. The projected semantic map is denoised, and then fused with the historically accumulated global semantic map according to the robot's pose change to obtain a stable global semantic map.

[0007] In an optional implementation, a multi-channel projected semantic map is generated by accumulating the number of points whose semantic labels belong to different semantic categories within each grid cell, including: For each 3D point in the semantically labeled point cloud, calculate the continuous coordinates of the 3D point in a top-down 2D grid map; Identify multiple grid cells adjacent to the continuous coordinates, and calculate the distance from the 3D point to the center of each adjacent grid cell; Each adjacent grid cell is assigned a corresponding weight based on the distance, where the smaller the distance, the greater the weight, and the sum of the weights of all adjacent grid cells is 1. The semantic labels of the three-dimensional points are accumulated into the count values ​​of the corresponding semantic category channels of the corresponding adjacent grid cells according to the weights. After traversing all three-dimensional points, the cumulative results of each grid cell in each semantic category channel constitute the multi-channel projected semantic map.

[0008] In one optional implementation, the denoised projected semantic map is fused with the historically accumulated global semantic map based on the robot's pose changes, including: Based on the change between the robot's current pose and the pose corresponding to the historically accumulated global semantic map, a spatial transformation is performed on the historically accumulated global semantic map to obtain a global semantic map aligned with the current observation viewpoint. For each semantic category channel, an exponential moving average method is used to weight and sum the denoised projected semantic map and the aligned global semantic map according to a preset fusion coefficient, so as to update the global semantic map at the current time. The preset fusion coefficient has a greater value in the obstacle channel than in the object channel. In this process, unobserved grid cells are marked as unknown states, and grid cells marked as unknown states do not participate in the weighted summation.

[0009] In an optional implementation, it further includes: For grid cells with non-zero activation values ​​in the object channel, calculate the confidence index of the grid cell; the confidence index includes at least one of the following: The grid cell is observed as the historical observation count of the target object category; The observation consistency score of the grid cell is determined by the ratio of the number of times the grid cell is observed as the target object category to the number of times it is observed as other categories or non-target categories. The question asks whether there exists a collision-free path from the robot's current position to the corresponding position of the grid cell, and the cost of such a path. This is used to represent the probability of the existence of a target object inferred based on the current semantic map. The probability of the existence of the target object is obtained by global average pooling of the entire object channel corresponding to the target object category. When the probability of the existence of the target object is greater than a first threshold, it is determined to be a reliable observation point state. When it is less than a second threshold, it is determined to be an unobserved state. When it is between the first threshold and the second threshold, it is determined to be an ambiguous state. When the reliability index meets the corresponding preset threshold condition, the location corresponding to the grid cell is determined to be the reliable observation point; If there is a reliable observation point in the object channel corresponding to the target object category in the multi-channel semantic map that meets the preset confidence condition, then the location corresponding to the reliable observation point is taken as the long-range target point.

[0010] In an optional implementation, for the ambiguous state, the method further includes: The input data of the target-oriented strategy network is fine-tuned, including: multiplying the corresponding channel value of the grid cells in the multi-channel semantic map whose probability of the existence of the target object is within the fuzzy interval by an amplification factor greater than 1 before inputting them into the convolutional neural network; in addition to the target object category, the categories of interference objects similar to the target object are also input into the target-oriented strategy network for comparative learning; and adding the robot's dwell time and observation frequency characteristics near the fuzzy area to the historical trajectory. The output layer of the target-oriented strategy network is switched to simultaneously output two-dimensional coordinate offset and action type discrete variables, wherein the action type discrete variables include at least: When the target object is occluded, predict the observation location that can increase the proportion of visible pixels of the target object; When there are multiple candidate grid cells, select the observation position that can distinguish between the real target object and the false detection object, and output the candidate position that has high accessibility and conforms to the prior spatial layout of the target object. When the confidence score of the grid cell containing the target object fluctuates beyond a preset threshold, the position that is close to the grid cell and maintains a safe distance is output to trigger the local planning module to control the robot to approach and reconfirm through high-resolution perception data.

[0011] In one optional implementation, during the training process, when the robot performs an active confirmation action in the ambiguous state, if the probability of the existence of the target object increases to the level of a reliable observation point that meets the preset confidence condition, a positive confirmation reward is given; if the confirmation result is a false detection, a negative reward is given, and the false detection area is marked as an excluded area, switching to the unknown area exploration mode.

[0012] In an optional implementation, the goal-oriented policy network includes: The word embedding encoding module is used to map the natural language name of the target object category into a fixed-dimensional semantic embedding vector through a pre-trained word embedding model; The trajectory encoding module is used to normalize and concatenate the robot's current pose with the historical poses of the past preset number of steps to obtain the trajectory feature vector; The spatial feature extraction module is used to input the multi-channel semantic map into a convolutional neural network to extract spatial feature maps. The feature fusion module is used to flatten the spatial feature map and concatenate it with the semantic embedding vector and the trajectory feature vector in the feature dimension to obtain a joint feature vector; The coordinate mapping module is used to input the joint feature vector into a fully connected network, map it into a two-dimensional coordinate offset relative to the robot's current position, and superimpose the two-dimensional coordinate offset with the robot's current position to obtain a long-range target point in the world coordinate system.

[0013] In an optional implementation, the training method for the goal-oriented policy network includes: A training dataset containing multiple indoor scenes is generated in a simulation environment, with target objects randomly placed in each scene; The parameters of the target-oriented policy network are updated using a near-end policy optimization algorithm; Interact with the environment once every 25 steps, calculate the cumulative reward and perform a gradient backpropagation; The cumulative reward is obtained by weighted summation of distance reduction reward, information gain reward, collision penalty and target discovery reward.

[0014] In an optional implementation, based on the long-range target point and the obstacle channels in the multi-channel semantic map, a collision-free path is generated from the robot's current position to the long-range target point, including: Obstacle channels are extracted from the multi-channel semantic map to generate a binary obstacle cost map; The binary obstacle cost map is inflated to extend the obstacle boundaries outward by a preset safe distance. Using the robot's current position as the source point and the long-range target point as the destination point, the minimum arrival time field from the source point to the destination point is solved on the expanded obstacle cost map using the fast travel method, and the initial global path is obtained by backtracking along the gradient descent direction. The initial global path is smoothed using B-spline curves to generate a smooth path with continuous curvature.

[0015] Secondly, the present invention provides a robot object navigation system based on goal-oriented semantic exploration, comprising: The map building module is used to acquire RGB-D images and pose information collected by the robot, and generate a top-down multi-channel semantic map through differentiable geometric projection; the multi-channel semantic map includes object channels corresponding to multiple preset object categories and at least one obstacle channel. The semantic prediction module is used to confirm that there are no reliable observation points in the object channel corresponding to the target object category in the multi-channel semantic map that meet the preset confidence conditions. Then, the multi-channel semantic map, the target object category, and the robot's historical trajectory are input into the pre-trained target guidance policy network. The target guidance policy network predicts the location of the unknown area where the target object is located based on the prior knowledge of the category distribution and spatial layout of the observed objects in the multi-channel semantic map, and uses the location of the unknown area as a long-range target point. The path generation module is used to generate a collision-free path from the robot's current position to the long-range target point based on the long-range target point and the obstacle channels in the multi-channel semantic map.

[0016] Thirdly, a device is provided, comprising: Memory for storing robot object navigation programs based on goal-oriented semantic exploration; A processor is configured to implement the steps of the goal-oriented semantic exploration-based robot object navigation method provided in the first aspect when executing the goal-oriented semantic exploration-based robot object navigation program.

[0017] Fourthly, a computer-readable storage medium is provided, on which a robot object navigation program based on goal-oriented semantic exploration is stored, wherein when the robot object navigation program based on goal-oriented semantic exploration is executed by a processor, the robot object navigation program based on goal-oriented semantic exploration implements the steps of the robot object navigation method based on goal-oriented semantic exploration provided in the first aspect.

[0018] The beneficial effects of this invention are that the robot object navigation method based on goal-oriented semantic exploration provided by this invention significantly improves the efficiency and robustness of robot object navigation in unknown environments by introducing a hierarchical navigation architecture based on goal-oriented semantic exploration. Specifically, the goal-oriented policy network can utilize the prior knowledge of the category distribution and spatial layout of observed objects in a multi-channel semantic map to intelligently predict the most likely area where the target object will appear when it has not been observed, thereby replacing the traditional blind full-coverage exploration strategy and greatly shortening the navigation time. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0021] Figure 2 This is a schematic flowchart illustrating the method for constructing a multichannel semantic map according to an embodiment of the present invention.

[0022] Figure 3 This is a diagram illustrating the prediction principle of a goal-oriented policy network according to an embodiment of the present invention.

[0023] Figure 4 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0027] The robot object navigation method based on goal-oriented semantic exploration provided in this embodiment of the invention is executed by a computer device, and correspondingly, the robot object navigation system based on goal-oriented semantic exploration runs in the computer device.

[0028] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The executing entity can be a robot object navigation system based on goal-oriented semantic exploration. Depending on different requirements, the order of the steps in this flowchart can be changed, and some can be omitted.

[0029] like Figure 1 As shown, the method includes: S1. Acquire RGB-D images and pose information collected by the robot, and generate a top-down multi-channel semantic map through differentiable geometric projection; the multi-channel semantic map includes object channels corresponding to multiple preset object categories and at least one obstacle channel; S2. If it is confirmed that there are no reliable observation points in the multi-channel semantic map corresponding to the target object category that meet the preset confidence conditions, then the multi-channel semantic map, the target object category, and the robot's historical trajectory are input into the pre-trained target guidance policy network. The target guidance policy network predicts the location of the unknown area where the target object is located based on the prior knowledge of the category distribution and spatial layout of the observed objects in the multi-channel semantic map, and uses the unknown area location as a long-range target point. S3. Based on the long-range target point and the obstacle channels in the multi-channel semantic map, generate a collision-free path from the robot's current position to the long-range target point.

[0030] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and its specific implementation plan in a non-limiting manner. Please refer to [reference needed]. Figure 2 .

[0031] The robot is equipped with an RGB-D camera (such as an Intel RealSense D435i) and an odometry / localization system (such as SLAM-based pose estimation) to acquire color images in real time at a frequency of 10Hz. Depth images and the corresponding robot pose The semantic mapping module constructs and dynamically updates the global semantic map according to the following steps.

[0032] Step 1: First-person semantic instance segmentation Color image Input a pre-trained instance segmentation model (such as Mask R-CNN) and output the semantic category label for each pixel. ,in This represents the preset total number of semantic categories (including obstacle categories). For each detected object instance, all pixels within it are assigned the corresponding category label, while background pixels are marked as unknown.

[0033] Step 2: Generation of 3D point clouds with semantic labels Using camera intrinsic matrix Each valid pixel in the depth image and its depth value The three-dimensional points are obtained by back-projecting them into the camera coordinate system. :

[0034] At the same time, the semantic tags obtained in step 1 Linking these 3D points creates a point cloud with semantic labels. .

[0035] Step 3: Differentiable geometric projection generates local semantic maps Let the resolution of the top-down two-dimensional grid map be... (e.g., 0.05m / grid), map size is First, transform the point cloud from the camera coordinate system to the world coordinate system:

[0036] in To determine the robot's current pose The constructed 4×4 transformation matrix. For each 3D point. Considering only its horizontal coordinate Calculate continuous coordinates on the map:

[0037] in This represents the minimum value of the map boundary. Take the integer grid index. And determine the four grid cells adjacent to that point. Calculate the distance from this point to the center of each adjacent grid cell, and define the weights using bilinear interpolation: Let... , The four weights are as follows:

[0038] Obviously Furthermore, the weight is inversely proportional to the distance. The semantic category of this 3D point is determined. The weights are accumulated and added to the corresponding semantic channels of the corresponding grid cells according to the aforementioned weights. Let... For each grid cell and category channels The accumulated value is:

[0039] After traversing all 3D points, a multi-channel projected semantic map of the current frame is obtained. Since the weight calculation is continuously differentiable, this projection operation is differentiable throughout, allowing gradient backpropagation.

[0040] Step 4: Single-frame noise reduction Will Input a lightweight fully convolutional network (e.g., 3 convolutional layers, 64 channels per layer, 3×3 kernel size), output a denoised confidence map. It is used to eliminate isolated noise caused by single-frame segmentation errors or occlusions.

[0041] Step 5: Integration with the global semantic map Let the global semantic map at the previous time step be... Based on the change between the robot's current pose and its historical pose. ,right Perform a bilinear interpolation spatial transformation to obtain a map aligned with the current observation viewpoint. Then for each semantic channel Updated using exponential moving average:

[0042] in This is the fusion coefficient. For obstacle channels (such as...) ),Pick To maintain the stability of obstacle information; for ordinary object channels (such as chairs, beds), take This allows object positions to be dynamically updated as the environment changes. Grid cells that have never been observed are marked as unknown and do not participate in the weighted summation; their values ​​remain at their initial values ​​(usually 0). This ultimately yields the global semantic map for the current time step. This map not only contains obstacle and free space information, but also the distribution probability of multiple object categories, providing rich structured input for the upper-level strategy module.

[0043] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0044] The robot has obtained the global semantic map at the current moment through the semantic mapping module. ,in The total number of semantic categories. The map grid resolution is set to (e.g., 0.05m per grid). Given the target object category (e.g., a television), the target-oriented semantic strategy module determines the long-range target point according to the following steps.

[0045] Step 1: Determining Reliable Observation Points First, examine the object channels in the global semantic map that correspond to the target object category. The activation status is determined. Grid cells with an activation value of 0 are considered unobserved points and are not included in the analysis.

[0046] For grid cells with non-zero activation values, their confidence index is calculated. In this embodiment, the confidence index includes at least one or more combinations of the following: Historical observation count: Record grid cell The cumulative number of times the target object was observed as such in history is .like (like If ), then the condition is satisfied.

[0047] Observational consistency score: The consistency score is defined as...

[0048] in This represents the number of times the grid cell is observed as a non-target category. It is a very small positive number. If (like If ), then the condition is satisfied.

[0049] Reachability determination: from the robot's current position to the world coordinates of this grid cell Pathfinding is performed using obstacles and pathways in the semantic map. If a collision-free path exists and its travel cost is... (like If ), then the condition is satisfied.

[0050] When any of the above confidence indicators meets the corresponding preset threshold, the location corresponding to the grid cell is determined to be a reliable observation point. If there are multiple reliable observation points, the point closest to the robot or with the highest activation value is selected as the target point.

[0051] If at least one reliable observation point exists, then the world coordinates of that reliable observation point are directly used as the long-range target point. No policy network needs to be invoked. The deterministic local policy module then plans the path and controls the robot to reach that point.

[0052] Step 2: Policy Network Prediction under Unobserved or Unreliable Conditions If there are no reliable observation points in the multi-channel semantic map that meet the preset confidence conditions for the object channel corresponding to the target object category (i.e., either all activation values ​​are 0, or the confidence of all non-zero activation values ​​is lower than the threshold), then the pre-trained target-oriented policy network is invoked to predict the location of the unknown area where the target object is most likely to be found, based on the prior knowledge of the category distribution and spatial layout of the observed objects in the current semantic map.

[0053] Please refer to Figure 3 The goal-oriented strategy network specifically includes the following sub-modules: Word embedding encoding module: Inputs the natural language name of the target object category (e.g., television) into a pre-trained word embedding model (e.g., Word2Vec or GloVe), mapping it to a fixed-dimensional semantic embedding vector. In this embodiment, .

[0054] Trajectory encoding module: Obtains the robot's current pose And the past Historical position of step ( The pose coordinates are normalized (e.g., divided by the map size), and then the current pose is concatenated with historical poses to form a trajectory feature vector.

[0055] Spatial feature extraction module: extracts global semantic map Input a convolutional neural network (CNN). This example uses two convolutional layers: First layer: Input channel 64 output channels, convolution kernel Step size 1, padding 1, activation function ReLU; Second layer: 64 input channels, 128 output channels, convolutional kernel Step size 2, padding 1, activation function ReLU.

[0056] Output spatial feature map .

[0057] Feature fusion module: flattens the spatial feature map into a one-dimensional vector. Then with semantic embedding vector Trajectory feature vector By concatenating the features along their dimensions, we obtain the joint feature vector:

[0058] in .

[0059] Coordinate mapping module: Combines joint feature vectors Input a two-layer fully connected network (MLP): First layer: The activation function is ReLU; Second layer: The activation function is ReLU; Output layer: Maps 256-dimensional features to two-dimensional coordinate offsets. .

[0060] Final long-range target point for:

[0061] Step 3: Training Method for Policy Network The goal-oriented policy network is trained using the Proximal Policy Optimization (PPO) algorithm in deep reinforcement learning. The specific steps are as follows: Thousands of scenes with different interior layouts, including living rooms, bedrooms, and kitchens, are generated in a simulation environment (such as AI2-THOR or Habitat). Target objects (such as televisions, chairs, beds, etc.) are randomly placed in each scene, and the robot's starting position is randomly initialized.

[0062] The cumulative reward is calculated after each interaction (after the robot performs an action), and is defined as:

[0063] in: This refers to the reduction in the Euclidean distance between the robot and the nearest target object before and after the action is performed. Information gain reward, defined as the reduction in map entropy, encourages robots to explore unknown areas; This is a collision indicator function (set to 1 if a collision occurs, otherwise set to 0). A sparse reward (set to +5) is triggered when the target first appears in the semantic map. The weighting coefficients are 0.1, 1.0, and 0.5 respectively in this embodiment.

[0064] The network parameters are updated using the Proximal Policy Optimization (PPO) algorithm, which interacts with the environment every 25 steps to collect trajectory data and calculate the Generalized Advantage Estimation (GAE), followed by gradient backpropagation. During training, a curriculum learning strategy is employed: gradually transitioning from simple scenes (single room, single target) to complex scenes (multi-story buildings, multiple interfering objects). Domain randomization (randomly changing lighting, texture, and object positions) is also used to enhance the model's transferability from simulation to real-world environments.

[0065] Once training converges, the target-oriented policy network can intelligently predict the most likely location of unobserved targets in actual navigation based on the input semantic map, target category, and historical trajectory, and output it as a long-range target point to the downstream local planning module.

[0066] The core innovation of this application's target-oriented semantic policy network lies in introducing a deep reinforcement learning paradigm into object target navigation, achieving a technological leap from passive recognition to active decision-making. Traditional supervised learning methods can only identify existing target distributions in a map and require manually labeled ground truth data, proving ineffective in unobserved areas and ambiguous situations. The policy network takes a multi-channel semantic map, target category embeddings, and historical trajectories as input. After spatial feature extraction via CNN, it is fused with trajectory and semantic vectors, and finally mapped to long-range target point coordinates through a fully connected layer. Training employs the PPO algorithm, with a reward function that integrates distance reduction, information gain, collision penalty, and target discovery reward. No manual labeling is required; the network autonomously learns scene co-occurrence priors and exploration strategies through simulated interaction. The significant advantages of this implementation are: active exploration capability, able to predict target locations in unobserved areas; strong robustness, effectively distinguishing real targets from interference through reachability and historical trajectories; closed-loop decision-making, with each action dynamically adjusted based on the latest map; high sample efficiency, with modular design reducing reinforcement learning complexity and seamless transferability to real robot platforms.

[0067] Based on the above implementation, this implementation further proposes an optimized active confirmation mechanism for ambiguous states (i.e., the probability of the existence of the target object is within a preset ambiguous range and there is no reliable observation point to confirm it), in order to solve the perceptual ambiguity problem caused by partial occlusion, interference from similar objects, or single-frame detection noise.

[0068] Step 1: Three-level confidence level division Let the global multi-channel semantic map output by the semantic mapping module be... ,in This represents the total number of semantic categories. For the target object category... Calculate its global existence probability:

[0069] This involves performing global average pooling on the target category channels. A first threshold is set. (e.g., 0.8) and the second threshold (e.g., 0.2), the current state is divided into three levels: Observed status: At this point, if there are reliable observation points on the map that meet the preset confidence level conditions, the long-range target point will be output directly.

[0070] Unobserved state: Since there are no reliable observations on the map, the policy network is directly invoked for semantic prior exploration.

[0071] Fuzzy state: If a suspected target exists but the confidence level is insufficient, the active confirmation mechanism of this implementation method is triggered.

[0072] Step 2: Fine-tuning the input data in a fuzzy state When the determination is in an ambiguous state, the target-oriented policy network performs the following fine-tuning processing on the input data: For grid cells where the probability of the target object existing falls within the fuzzy range, multiply their channel values ​​by an amplification factor. (This embodiment takes) This forces the network to focus on these areas of high uncertainty.

[0073] In addition to the target object category, the categories of visually similar interfering objects (e.g., for a target television, a poster is introduced as an interfering category) are also encoded as embedding vectors and input into the network along with the target category embedding, so that the network can learn to distinguish between real targets and false detections at the feature level.

[0074] In the historical trajectory feature vector, add the robot's cumulative dwell time near the fuzzy region. and number of observations As an additional feature dimension:

[0075] This feature indicates that if the target is not identified after lingering in an ambiguous area for too long, a change in strategy (such as moving closer or switching the exploration direction) is needed.

[0076] Step 3: Output switching and active confirmation in fuzzy state In the ambiguous state, the output layer of the target guidance strategy network switches from conventional two-dimensional coordinate offset prediction to simultaneously outputting two-dimensional coordinate offsets. and discrete variables of action type .in They represent: : Proceed to the output point for final confirmation (applicable when the target location has been basically determined); : Go to the output point to complete the view (applicable when the target is partially obscured); : Go to the output point for close-range re-sensing (suitable for situations with large confidence fluctuations).

[0077] Specifically, the active confirmation actions corresponding to the discrete variables of action types include: View completion: When the target object is occluded by other objects (such as a sofa), the policy network predicts an observation position that can increase the proportion of visible pixels of the target object and guides the robot to move to that position to eliminate the occlusion.

[0078] Candidate disambiguation: When multiple candidate grid cells exist in the semantic map (e.g., both the TV channel and the poster channel are active), the policy network selects the observation location with the highest discriminative power. This selection comprehensively considers accessibility (whether a collision-free path exists) and prior spatial layout of the target object (e.g., there is usually open space in front of the TV, while the poster is close to the wall), prioritizing the output of candidate locations with high accessibility and conforming to the prior.

[0079] Near-field resenting: the confidence score of the grid cell containing the target object. If fluctuations exceed a preset threshold across multiple consecutive frames, the system is deemed to be perceptually unstable. In this case, the policy network outputs a position close to the grid cell while maintaining a safe distance (e.g., 0.5m), triggering the local planning module to control the robot to approach and reconfirm using high-resolution RGB-D raw data, rather than relying solely on a low-resolution global map.

[0080] Final long-range target point and action type Common output, local planning module according to Adjust motion parameters (such as approach speed, stopping distance, etc.).

[0081] Step 4: Design of Training Rewards for Fuzzy States To support the aforementioned proactive confirmation behavior, in the reinforcement learning training of the policy network, in addition to the basic reward... Information gain reward In addition to collision penalties and target discovery rewards, the following additional features are added: Confirm Reward :

[0082] Obstruction Removal Rewards If, after the robot moves, the visible pixel percentage of the target object increases by more than 5%, then... .

[0083] The total reward function is:

[0084] In this embodiment, the weighting parameter is taken as follows: .

[0085] Step 5: Dynamic Arbitration and Strategy Switching Set a decision entropy threshold; after multiple consecutive active confirmation actions, the uncertainty of the candidate region (in terms of...) is... If the variance or entropy measure of confidence score does not decrease significantly, the system determines that the region is a false detection or noise. At this time, the region is marked as excluded in the semantic map, and the system switches back to the unobserved state to call the goal-oriented policy network to explore semantic priors, guiding the robot to other unexplored regions.

[0086] The aforementioned arbitration mechanism can be formally described as: if an active confirmation action is performed. After that ( ), still If the regional confidence entropy does not decrease by more than 10%, then set a mask:

[0087] This causes the policy network to ignore this region in subsequent decisions.

[0088] Traditional object navigation methods typically simplify the existence of targets on a map into a binary decision of presence or absence. This coarse-grained state division is prone to misjudging presence or underjudging absence when faced with partial occlusion, interference from similar objects, or noise in single-frame detection, leading to navigation failure. The target-oriented semantic strategy module proposed in this invention introduces a three-level confidence partitioning mechanism, dividing target existence into three levels. For different levels, the policy network employs three differentiated inference modes. Especially in ambiguous states, instead of blindly outputting target points, it outputs confirmation actions, including sub-strategies such as view completion, candidate disambiguation, and near-field re-perception, guiding the robot to actively eliminate perceptual uncertainty.

[0089] Meanwhile, targeted confirmation rewards and occlusion resolution rewards were designed during the training phase to encourage the model to perform effective confirmation behaviors in a blurred state and to mark and exclude false detection areas. A dynamic arbitration mechanism was also used to achieve smooth switching between strategies.

[0090] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0091] Step 1: Obstacle Cost Map Construction From the multi-channel semantic map Extract obstacle channels. Typically, channel 0 or a merged channel containing categories such as walls, furniture, and impassable areas is used as an obstacle identifier. Generate a binary obstacle cost map. ,in:

[0092] Step 2: Obstacle Inflation Process To ensure the robot maintains a safe distance from obstacles, morphological dilation is performed on the binary obstacle cost map. Let the robot radius be... (In this embodiment, 0.3m is used), and the map resolution is... (In this embodiment, we take 0.05m / grid), so the number of grids corresponding to the expansion radius is: Using a radius of circular structural elements To expand:

[0093] in Indicates Centered on, with radius The neighborhood grid set. After expansion, all grids with a distance less than [a certain value] from the obstacle. All grid cells are marked as impassable, thus providing a safe boundary for subsequent path planning.

[0094] Step 3: Fast Marching Method (FMM) Global Path Search Based on the robot's current position The corresponding grid is the source point, and the long-range target point is... Using the corresponding grid as the endpoint, the Eikonal equation is solved on the inflated cost map using the fast traversal method:

[0095] in Indicates the distance from the source point to the point Minimum arrival time, This is a function of propagation speed. In this embodiment, in free space ( )Pick Take in the obstacle area The arrival time field was obtained by numerically solving using the upwind difference scheme. .

[0096] Then, from the finish line To begin, we backtrack along the gradient descent direction to the source point, i.e., we solve the following differential equation:

[0097] Obtain the initial global path .

[0098] Step 4: Smoothing the B-spline curve path To satisfy robot kinematic constraints (such as curvature continuity), a cubic uniform B-spline curve is used to smooth the initial path. Let the control point sequence be... (Taken from Then, the B-spline curve is defined as:

[0099] in The basis functions are cubic B-spline functions. The control points are adjusted using interpolation or approximation methods to make the curve... While maintaining a shape similar to the original path, it features continuous curvature and no redundant inflection points. The smoothed path. This final global path is output to the subsequent local obstacle avoidance control module.

[0100] In some embodiments, the robot object navigation system based on goal-oriented semantic exploration may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the robot object navigation system based on goal-oriented semantic exploration may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) Functionality of robot object navigation based on goal-oriented semantic exploration.

[0101] In this embodiment, the robot object navigation system based on goal-oriented semantic exploration can be divided into multiple functional modules according to the functions it performs, such as... Figure 4 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and is stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0102] The map building module is used to acquire RGB-D images and pose information collected by the robot, and generate a top-down multi-channel semantic map through differentiable geometric projection; the multi-channel semantic map includes object channels corresponding to multiple preset object categories and at least one obstacle channel. The semantic prediction module is used to confirm that there are no reliable observation points in the object channel corresponding to the target object category in the multi-channel semantic map that meet the preset confidence conditions. Then, the multi-channel semantic map, the target object category, and the robot's historical trajectory are input into the pre-trained target guidance policy network. The target guidance policy network predicts the location of the unknown area where the target object is located based on the prior knowledge of the category distribution and spatial layout of the observed objects in the multi-channel semantic map, and uses the location of the unknown area as a long-range target point. The path generation module is used to generate a collision-free path from the robot's current position to the long-range target point based on the long-range target point and the obstacle channels in the multi-channel semantic map.

[0103] Figure 5 The robot object navigation method based on goal-oriented semantic exploration provided in this application embodiment can be applied to devices. Those skilled in the art will understand that the device structure involved in the embodiments of this invention does not constitute a limitation on the device. The device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. Specifically, the device 500 may include: a processor 510, a memory 520, and a communication unit 530. These components communicate through one or more buses. Those skilled in the art will understand that the server structure shown in the figures does not constitute a limitation on the invention; it may be a bus topology or a star topology, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0104] The memory 520 can be used to store execution instructions of the processor 510. The memory 520 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 520 are executed by the processor 510, the device 500 is able to perform some or all of the steps in the above method embodiments.

[0105] The processor 510 serves as the control center of the storage device, connecting various parts of the electronic device via various interfaces and lines. It executes software programs and / or modules stored in the memory 520, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of integrated circuits, such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 510 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0106] The communication unit 530 is used to establish a communication channel, enabling the storage device to communicate with other devices. It can receive user data sent by other devices or send user data to other devices.

[0107] The present invention also provides a computer medium, wherein the computer medium may store a program, which, when executed, may include some or all of the steps provided in the embodiments of the present invention. The medium may be a magnetic disk, an optical disk, a read-only memory, or a random access memory, etc.

[0108] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a medium such as a USB flash drive, mobile hard drive, read-only memory, random access memory, magnetic disk, or optical disk, or any other medium capable of storing program code. It includes several instructions to cause a computer device (which may be a personal computer, server, or a second device, network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0109] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0110] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0111] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0113] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A robot object navigation method based on goal-oriented semantic exploration, characterized in that, include: The robot acquires RGB-D images and pose information, and generates a top-down multi-channel semantic map through differentiable geometric projection. The multi-channel semantic map includes object channels corresponding to multiple preset object categories and at least one obstacle channel; If it is confirmed that there are no reliable observation points in the object channel corresponding to the target object category in the multi-channel semantic map that meet the preset confidence conditions, then the multi-channel semantic map, the target object category, and the robot's historical trajectory are input into the pre-trained target guidance policy network. The target guidance policy network predicts the location of the unknown area where the target object is located based on the prior knowledge of the category distribution and spatial layout of the observed objects in the multi-channel semantic map, and uses the unknown area location as the long-range target point. Based on the long-range target point and the obstacle channels in the multi-channel semantic map, a collision-free path is generated from the robot's current position to the long-range target point.

2. The method according to claim 1, characterized in that, The system acquires RGB-D images and pose information gathered by the robot, and generates a top-down multi-channel semantic map through differentiable geometric projection, including: Perform instance segmentation on the color image in the RGB-D image and predict the semantic category of each pixel to obtain the semantic label of each pixel; A 3D point cloud is generated based on the depth image in the RGB-D image, and the semantic label of each pixel is associated with each point in the 3D point cloud to generate a point cloud with semantic labels. Based on the pose information, the point cloud with semantic labels is projected onto a top-down two-dimensional grid map through differentiable geometric transformation. By accumulating the number of points in each grid cell whose semantic labels belong to different semantic categories, a multi-channel projected semantic map is generated. The projected semantic map is denoised, and then fused with the historically accumulated global semantic map according to the robot's pose change to obtain a stable global semantic map.

3. The method according to claim 2, characterized in that, By accumulating the number of points whose semantic labels belong to different semantic categories within each grid cell, a multi-channel projected semantic map is generated, including: For each 3D point in the semantically labeled point cloud, calculate the continuous coordinates of the 3D point in a top-down 2D grid map; Identify multiple grid cells adjacent to the continuous coordinates, and calculate the distance from the 3D point to the center of each adjacent grid cell; Each adjacent grid cell is assigned a corresponding weight based on the distance, where the smaller the distance, the greater the weight, and the sum of the weights of all adjacent grid cells is 1. The semantic labels of the three-dimensional points are accumulated into the count values ​​of the corresponding semantic category channels of the corresponding adjacent grid cells according to the weights. After traversing all three-dimensional points, the cumulative results of each grid cell in each semantic category channel constitute the multi-channel projected semantic map.

4. The method according to claim 2, characterized in that, The denoised projected semantic map is fused with the historically accumulated global semantic map based on the robot's pose changes, including: Based on the change between the robot's current pose and the pose corresponding to the historically accumulated global semantic map, a spatial transformation is performed on the historically accumulated global semantic map to obtain a global semantic map aligned with the current observation viewpoint. For each semantic category channel, an exponential moving average method is used to weight and sum the denoised projected semantic map and the aligned global semantic map according to a preset fusion coefficient, so as to update the global semantic map at the current time. The preset fusion coefficient has a greater value in the obstacle channel than in the object channel. In this process, unobserved grid cells are marked as unknown states, and grid cells marked as unknown states do not participate in the weighted summation.

5. The method according to claim 1, characterized in that, The method further includes: For grid cells with non-zero activation values ​​in the object channel, calculate the confidence index of the grid cell; the confidence index includes at least one of the following: The grid cell is observed as the historical observation count of the target object category; The observation consistency score of the grid cell is determined by the ratio of the number of times the grid cell is observed as the target object category to the number of times it is observed as other categories or non-target categories. The question asks whether there is a collision-free path from the robot's current position to the corresponding position of the grid cell, and the cost of such a path. This is used to represent the probability of the existence of a target object inferred based on the current semantic map. The probability of the existence of the target object is obtained by global average pooling of the entire object channel corresponding to the target object category. When the probability of the existence of the target object is greater than a first threshold, it is determined to be a reliable observation point state. When it is less than a second threshold, it is determined to be an unobserved state. When it is between the first threshold and the second threshold, it is determined to be an ambiguous state. When the reliability index meets the corresponding preset threshold condition, the location corresponding to the grid cell is determined to be the reliable observation point; If there is a reliable observation point in the object channel corresponding to the target object category in the multi-channel semantic map that meets the preset confidence condition, then the location corresponding to the reliable observation point is taken as the long-range target point.

6. The method according to claim 5, characterized in that, For the aforementioned ambiguous state, the method further includes: The input data of the target-oriented strategy network is fine-tuned, including: multiplying the corresponding channel value of the grid cells in the multi-channel semantic map whose probability of the existence of the target object is within the fuzzy interval by an amplification factor greater than 1 before inputting them into the convolutional neural network; in addition to the target object category, the categories of interference objects similar to the target object are also input into the target-oriented strategy network for comparative learning; and adding the robot's dwell time and observation frequency characteristics near the fuzzy area to the historical trajectory. The output layer of the target-oriented strategy network is switched to simultaneously output two-dimensional coordinate offset and action type discrete variables, wherein the action type discrete variables include at least: When the target object is occluded, predict the observation location that can increase the proportion of visible pixels of the target object; When there are multiple candidate grid cells, select the observation position that can distinguish between the real target object and the false detection object, and output the candidate position that has high accessibility and conforms to the prior spatial layout of the target object. When the confidence score of the grid cell containing the target object fluctuates beyond a preset threshold, the position that is close to the grid cell and maintains a safe distance is output to trigger the local planning module to control the robot to approach and reconfirm through high-resolution perception data.

7. The method according to claim 6, characterized in that, During the training process, when the robot performs an active confirmation action in the ambiguous state, if the probability of the existence of the target object increases to the level of a reliable observation point that meets the preset confidence conditions, a positive confirmation reward is given; if the confirmation result is a false detection, a negative reward is given, and the false detection area is marked as an excluded area, switching to the unknown area exploration mode.

8. The method according to claim 1, characterized in that, The goal-oriented strategy network includes: The word embedding encoding module is used to map the natural language name of the target object category into a fixed-dimensional semantic embedding vector through a pre-trained word embedding model; The trajectory encoding module is used to normalize and concatenate the robot's current pose with the historical poses of the past preset number of steps to obtain the trajectory feature vector; The spatial feature extraction module is used to input the multi-channel semantic map into a convolutional neural network to extract spatial feature maps. The feature fusion module is used to flatten the spatial feature map and concatenate it with the semantic embedding vector and the trajectory feature vector in the feature dimension to obtain a joint feature vector; The coordinate mapping module is used to input the joint feature vector into a fully connected network, map it into a two-dimensional coordinate offset relative to the robot's current position, and superimpose the two-dimensional coordinate offset with the robot's current position to obtain a long-range target point in the world coordinate system.

9. The method according to claim 8, characterized in that, The training method for the goal-oriented policy network includes: A training dataset containing multiple indoor scenes is generated in a simulation environment, with target objects randomly placed in each scene; The parameters of the target-oriented policy network are updated using a near-end policy optimization algorithm; Interact with the environment once every 25 steps, calculate the cumulative reward and perform a gradient backpropagation; The cumulative reward is obtained by weighted summation of distance reduction reward, information gain reward, collision penalty and target discovery reward.

10. The method according to claim 1, characterized in that, Based on the long-range target point and the obstacle channels in the multi-channel semantic map, a collision-free path is generated from the robot's current position to the long-range target point, including: Obstacle channels are extracted from the multi-channel semantic map to generate a binary obstacle cost map; The binary obstacle cost map is inflated to extend the obstacle boundaries outward by a preset safe distance. Using the robot's current position as the source point and the long-range target point as the destination point, the minimum arrival time field from the source point to the destination point is solved on the expanded obstacle cost map using the fast travel method, and the initial global path is obtained by backtracking along the gradient descent direction. The initial global path is smoothed using B-spline curves to generate a smooth path with continuous curvature.

Images