Quadruped robot recharging positioning method and device, robot, and medium

Abstract

The present disclosure provides a quadruped robot charging positioning method, device, robot and medium, etc., in the movement process of the quadruped robot, the original radar point cloud data and image data collected are acquired, and feature extraction is respectively performed; the node features in the radar features are aligned with the shallow feature maps in the image features, the edge features in the radar features are aligned with the deep feature maps in the image features, the aligned multi-modal features are obtained, and based on the aligned multi-modal features, feature fusion processing is performed to obtain fusion features; a space-time memory system is constructed based on the fusion features, charging pile identification positioning is performed based on the space-time memory system, a charging pile position positioning result is generated, and the quadruped robot is guided to move to the charging pile position for charging. The dependence on priori map and artificial annotation can be eliminated, real-time multi-modal fusion perception is realized, the differentiated positioning accuracy requirement in the multi-stage charging process is met, and the perception result time sequence consistency is maintained in the robot movement process.

Description

Technical Field

[0001] This disclosure relates to the field of robotics technology, and in particular to a method, apparatus, robot, and medium for recharging and positioning a quadruped robot. Background Technology

[0002] With the rapid development of quadruped robot technology, achieving high precision and autonomous recharging has become a key aspect in ensuring their endurance and operational reliability. As a type of system with complex motion capabilities and irregular gait characteristics, quadruped robots not only rely on global navigation path planning for their recharging tasks, but also require stable and accurate local positioning and posture alignment within a few meters of their end effector.

[0003] The relevant autonomous recharging and localization technology for quadruped robots mainly relies on known maps, where the locations of various charging stations are pre-marked. The robot then navigates to the target charging station for charging. This method, because it heavily depends on prior map knowledge and manual annotation, lacks adaptive and self-updating capabilities. If the location of the charging station or the environment changes, manual maintenance, updating, or reconstruction of the map is required. If the map is not updated in a timely manner, it will be unable to accurately navigate to the target charging station, thus failing to complete the charging task. Summary of the Invention

[0004] This disclosure provides at least one method, apparatus, robot, and medium for controlling a quadruped robot. This disclosure provides a method for recharging and positioning a quadruped robot, including: During the movement of the quadruped robot, raw radar point cloud data and image data collected by the quadruped robot are acquired, and feature extraction is performed on the raw radar point cloud data and image data respectively to determine radar features and image features. The node features in the radar features are aligned with the shallow feature maps in the image features, and the edge features in the radar features are aligned with the deep feature maps in the image features to obtain aligned multimodal features. Based on the aligned multimodal features, feature fusion processing is performed to obtain fused features. The number of convolutional layers generating the shallow feature maps is less than the number of convolutional layers generating the deep feature maps. A spatiotemporal memory system is constructed based on the fused features. The spatiotemporal memory system is used to maintain temporal continuity and scene geometry. Based on the spatiotemporal memory system, charging pile identification and positioning are performed to generate charging pile location results. Based on the charging pile location results, the quadruped robot is guided to move to the charging pile location for charging.

[0005] The above scheme aligns and fuses radar features and image features at different levels. Through this multimodal data fusion processing, it can make full use of the precise spatial information of radar point cloud data and the rich semantic information of image data, adapt to the positioning needs at different distance stages, and improve the accuracy and robustness of charging pile identification. By constructing a spatiotemporal memory system, it can maintain the temporal consistency of perception results during robot movement and effectively cope with various interferences in dynamic environments.

[0006] In one optional implementation, feature extraction of the original radar point cloud data includes: voxelizing the original radar point cloud data to obtain a voxel grid; the voxelization process involves dividing the point cloud points corresponding to the original radar point cloud data into a uniform cubic grid in three-dimensional space; identifying non-empty voxels in the voxel grid; constructing an octree data structure for each non-empty voxel; and performing hierarchical analysis on the octree data structure to obtain the radar features of the radar feature points.

[0007] The above implementation method, through voxelization and octree data structure analysis, can effectively reduce the size of the original point cloud data and reduce computational complexity, while retaining the key geometric information in the point cloud data, laying the foundation for subsequent feature extraction.

[0008] In one optional implementation, an octree data structure is constructed for each non-empty voxel, and hierarchical analysis is performed on the octree data structure, including: calculating the covariance matrix based on the point set within each non-empty voxel and performing eigenvalue decomposition to obtain multiple eigenvalues; determining the structural complexity of the point cloud of the non-empty voxel region based on the ratio of the largest eigenvalue to the smallest eigenvalue among the multiple eigenvalues; and performing recursive segmentation when the structural complexity exceeds a preset complexity threshold until the node satisfies any of the following termination conditions: planar attribute, linear attribute, reaching the maximum recursion depth, or the number of points within the node is less than the minimum number of points threshold, thereby determining radar feature points.

[0009] The above implementation method, through covariance matrix analysis and eigenvalue decomposition, can adaptively evaluate the structural complexity of point cloud regions, recursively subdivide complex regions, and retain simple regions, thereby achieving adaptive simplification of point cloud data and improving the representativeness and accuracy of feature points.

[0010] In one alternative implementation, determining radar feature points until a node satisfies any termination condition includes: calculating the centroid coordinates of the point set within the node that satisfies the termination condition as the representative position of the node; calculating the principal direction vector of the node as the normal vector based on the covariance matrix of the point set; and combining the representative position and the normal vector to form a representative feature point, thereby obtaining an initial set of radar feature points.

[0011] The above implementation method can generate highly representative feature points by calculating the centroid coordinates and principal direction vector of the nodes, effectively expressing the position and orientation information of the point cloud region, and providing high-quality basic data for subsequent feature matching and fusion.

[0012] In one optional implementation, after obtaining the initial radar feature point set, the method further includes: calculating the normal angle between adjacent feature points based on the normal vectors of adjacent feature points in the initial feature point set; calculating the Mahalanobis distance between the centroids of adjacent feature points based on the covariance matrix of adjacent feature points in the initial feature point set; and when the normal angle between adjacent feature points is less than a set angle and the Mahalanobis distance is less than a set distance threshold, weighted averaging and merging the covariance matrices of the adjacent feature points to generate new representative feature points, thereby obtaining an optimized feature point set.

[0013] The above implementation method can identify and merge feature points belonging to the same macrostructure by analyzing the normal angle and Mahalanobis distance between feature points, thereby reducing redundant feature points, improving the compactness and representativeness of the feature set, and maintaining the integrity of the geometric structure.

[0014] In one optional implementation, the node features and edge features in the radar features are determined according to the following steps: the k nearest neighbor algorithm is used to find k nearest neighbors for each radar feature point and establish directed connections to construct a graph structure; the three-dimensional coordinates of each radar feature point are used as the initial feature vector of the node; the initial feature vectors and adjacency relationships of each node in the graph structure are input into a PCA-GM enhanced GATv2 network for graph feature learning; and the node features of each node and the edge features between adjacent nodes are obtained through multi-layer graph attention calculation.

[0015] The above implementation method, by constructing a graph structure and applying an advanced graph neural network, can fully consider the topological relationships between feature points, learn richer feature representations, and generate node features and edge features, providing structured radar feature information for subsequent multimodal fusion.

[0016] In one optional implementation, feature extraction of the image data includes: performing multi-level convolutional processing using a ResNet-18 network based on the image data to obtain feature maps of different resolutions; and constructing a multi-level feature map structure based on the feature maps of different resolutions to obtain the image features.

[0017] The above implementation method, by employing a mature deep convolutional neural network, is able to extract multi-level feature representations from image data. Shallow feature maps retain rich spatial details, while deep feature maps contain abstract semantic information, providing rich visual features for multimodal fusion.

[0018] In one optional implementation, feature fusion processing includes: based on the aligned multimodal features, defining a corresponding rectangular fusion region in the image feature map with the horizontal coordinates of the radar feature points as the center and a preset radius range; based on the image features and radar features within the rectangular fusion region, using the image features as a query vector and the radar features as key-value pairs, calculating attention weights; and based on the attention weights, performing weighted fusion of the image features and radar features to obtain the fused features.

[0019] The above implementation defines the fusion region centered on radar feature points and uses an attention mechanism to calculate weights, which enables precise alignment and adaptive fusion of radar and visual features, effectively solving the problem of scale and density differences between the two modalities and generating fused features with both spatial accuracy and semantic richness.

[0020] In one optional implementation, the image features and radar features are weighted and fused based on the attention weights, including: weighting and summing the node features of the shallow feature map and the radar feature points based on the attention weights to enhance the spatial positioning features; and weighting and summing the edge features of the deep feature map and the radar feature points based on the attention weights to extract global structural context features; and concatenating the enhanced spatial positioning features and the global structural context features to obtain the fused features.

[0021] The above implementation distinguishes between processing node features and shallow feature maps, and edge features and deep feature maps. It can adopt matching fusion strategies for features at different levels, which not only retains accurate spatial positioning capabilities, but also integrates rich structural context information to generate more comprehensive and effective fused features.

[0022] In one alternative implementation, constructing a spatiotemporal memory system includes: constructing a temporal memory stream by storing feature information of continuous fusion feature frames in a real-time frame queue based on the fusion features; constructing a spatial map layer by geometric structure modeling and optimization based on the fusion features; and obtaining the spatiotemporal memory system by performing dual-axis fusion based on the temporal memory stream and the spatial map layer through a keyframe bridging mechanism.

[0023] The above implementation method can simultaneously maintain the temporal continuity of appearance and the geometric consistency of scene by constructing the temporal memory stream and spatial map layer in parallel, providing multi-dimensional reference information for charging pile identification and improving the stability and reliability of identification.

[0024] In one optional implementation, the spatiotemporal memory system is obtained by performing dual-axis fusion based on the temporal memory stream and the spatial map layer through a keyframe bridging mechanism. This includes: based on the temporal memory stream, when a fused feature frame with a confidence level greater than a set confidence level is detected, the fused feature frame is used as a visual keyframe, and a keyframe trigger signal is generated; based on the keyframe trigger signal, the construction of corresponding keyframes in the spatial map layer is triggered, and spatial keyframes are obtained by marginalizing and compressing statistical distribution features in the space; based on the statistical distribution features, multiple spatial map layer update tasks are accumulated to a preset batch size and then a batch matrix operation is performed uniformly to obtain the spatiotemporal memory system.

[0025] The above implementation method, through keyframe triggering mechanism and batch processing strategy, can improve computational efficiency while ensuring spatiotemporal consistency, reduce redundant update operations, achieve efficient spatiotemporal memory system maintenance, and meet the performance requirements of real-time recharge positioning.

[0026] In one optional implementation, the target identification and positioning of charging piles based on the spatiotemporal memory system includes: matching charging pile targets using historical appearance templates in the time memory stream and geometric structure information in the spatial map layer based on the spatiotemporal memory system; when the target matching confidence is lower than a preset threshold, retrieving historical appearance templates in the time memory stream and estimating the possible target area in the spatial map layer; within the possible target area, determining the location of the charging pile through intersection filtering and fine matching of the time memory stream and the spatial map layer, thereby obtaining the charging pile location positioning result.

[0027] By combining information from the time memory stream and the spatial map layer, the above implementation method can maintain its positioning capability by relying on historical memory and geometric calculation when the charging pile is temporarily obscured or its features are not obvious, thereby achieving continuous and stable tracking of the charging pile and improving the success rate of autonomous recharging.

[0028] This disclosure also provides a quadruped robot recharging and positioning device, including: The feature extraction module is used to acquire the original radar point cloud data and image data collected by the quadruped robot during the movement of the quadruped robot, and to extract features from the original radar point cloud data and image data respectively to determine radar features and image features. The feature fusion module is used to align the node features in the radar features with the shallow feature maps in the image features, and to align the edge features in the radar features with the deep feature maps in the image features, to obtain aligned multimodal features. Based on the aligned multimodal features, feature fusion processing is performed to obtain fused features. The number of convolutional layers used to generate the shallow feature maps is less than the number of convolutional layers used to generate the deep feature maps. A spatiotemporal memory construction module is used to construct a spatiotemporal memory system based on the fusion features, wherein the spatiotemporal memory system is used to maintain temporal continuity and scene geometric structure. The positioning and guidance module is used to identify and locate the charging pile based on the spatiotemporal memory system, generate the charging pile location result, and guide the quadruped robot to move to the charging pile location for charging based on the charging pile location result.

[0029] This disclosure also provides a quadruped robot, which is equipped with a lidar, a camera, and a controller. The lidar is used to collect raw radar point cloud data during the movement of the quadruped robot; the camera is used to collect image data during the movement of the quadruped robot. The controller is used to acquire the original radar point cloud data and image data, and to extract features from the original radar point cloud data and image data respectively to determine radar features and image features; align the node features in the radar features with the shallow feature maps in the image features, and align the edge features in the radar features with the deep feature maps in the image features to obtain aligned multimodal features, and perform feature fusion processing based on the aligned multimodal features to obtain fused features; wherein the number of convolutional layers generating the shallow feature maps is less than the number of convolutional layers generating the deep feature maps; construct a spatiotemporal memory system based on the fused features, the spatiotemporal memory system is used to maintain temporal continuity and scene geometry; perform charging pile identification and positioning based on the spatiotemporal memory system, generate charging pile location positioning results, and guide the quadruped robot to move to the charging pile location for charging based on the charging pile location positioning results.

[0030] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the quadruped robot recharge and positioning method as described in any of the above embodiments.

[0031] The specific implementation and effects of the above-mentioned device, quadruped robot and computer-readable storage medium are described in the aforementioned method description, and will not be repeated here.

[0032] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0033] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0034] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0035] Figure 1 A flowchart illustrating a quadruped robot recharging and positioning method provided in this embodiment of the disclosure; Figure 2 This is a schematic diagram illustrating the process of feature extraction from raw radar point cloud data according to an embodiment of this disclosure; Figure 3 This is a flowchart illustrating the hierarchical analysis of an octree data structure according to an embodiment of this disclosure. Figure 4 This is a schematic diagram illustrating the process of feature extraction from image data according to an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the feature fusion process in an embodiment of the present disclosure; Figure 6 A framework diagram for constructing a spatiotemporal memory system; Figure 7 A flowchart illustrating the process of constructing a spatiotemporal memory system: Figure 8 This is a schematic diagram of the process for identifying and locating charging pile targets based on a spatiotemporal memory system. Figure 9 This is a schematic diagram of a quadruped robot recharging and positioning device provided in an embodiment of the present disclosure; Figure 10 This is a diagram showing the external shape and structural modules of a quadruped robot provided in an embodiment of this disclosure. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0037] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0038] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0039] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0040] In related quadruped robot autonomous recharging and localization technologies, the main approach relies on feature registration schemes based on known maps. These methods assume that the location, shape, and other features of the charging station have been manually labeled on the global map. The recharging process is essentially a two-step combination of "navigation to the target point + local alignment." Typical implementations include geometric registration based on LiDAR and appearance matching based on vision. The main drawback of these schemes is their strong dependence on prior map knowledge and manual labeling, lacking adaptive and self-updating capabilities. Once the location of the charging station or the environment changes, manual maintenance, updating, or reconstruction of the map is required. Furthermore, affordable LiDAR point clouds are relatively sparse, failing to provide sufficient point cloud density for perception in areas exceeding a certain distance (e.g., beyond 10 meters).

[0041] In addition, there is a learning-based single-modal perception scheme. This type of method uses deep learning networks to extract more robust features for charging pile identification and registration optimization. Although multimodal data may be used during the training phase, it often degenerates into single-modal (usually a pure vision solution) operation during deployment, with other sensors only serving as backup inputs. The main drawback of this type of scheme is its lack of generalization ability. It requires retraining for specific charging pile models, deployment scenarios, and onboard sensors, and it cannot meet the different perception requirements for charging pile positioning at different distance stages in actual deployment.

[0042] Therefore, in the field of autonomous recharging of quadruped robots, this disclosure provides a technical solution that eliminates the reliance on prior maps and manual annotations, achieves true real-time multimodal fusion perception, meets the differentiated positioning accuracy requirements during recharging at multiple stages (far, medium, and near), and maintains temporal consistency of perception results during robot movement.

[0043] like Figure 1 As shown, this disclosure provides a method for recharging and positioning a quadruped robot, including: Step S1: During the movement of the quadruped robot, acquire the original radar point cloud data and image data collected by the quadruped robot, and extract features from the original radar point cloud data and image data respectively to determine radar features and image features.

[0044] In this embodiment, the quadruped robot collects point cloud data of the surrounding environment via its onboard LiDAR sensor and simultaneously acquires image data via a camera during movement. For example, the LiDAR can be a multi-line LiDAR with a horizontal field of view of 360 degrees and a vertical field of view of 30 degrees, with a sampling frequency of 10Hz, generating approximately 100,000 point cloud data points per second. The camera can be a 1080P resolution RGB camera with a sampling frequency of 30Hz. The quadruped robot's control system synchronizes the data collected by these two different sensors according to timestamps, ensuring that the acquisition time difference between the point cloud data and image data is within 10 milliseconds, facilitating subsequent multimodal fusion processing. The original LiDAR point cloud data typically contains a large number of noise points and redundant information, while image data may suffer from blurring, overexposure, or underexposure due to factors such as lighting and viewing angle. Therefore, feature extraction is required for both types of data to obtain more stable and representative features.

[0045] Step S2: Align the node features in the radar features with the shallow feature maps in the image features, and align the edge features in the radar features with the deep feature maps in the image features to obtain aligned multimodal features. Based on the aligned multimodal features, perform feature fusion processing to obtain fused features. The number of convolutional layers used to generate the shallow feature maps is less than the number of convolutional layers used to generate the deep feature maps.

[0046] In this embodiment, radar features and image features have different data structures and representations, requiring feature alignment for effective fusion. Radar features are typically sparse three-dimensional point sets, while image features are dense two-dimensional feature maps. A spatial correspondence between the two is established by projecting the radar point cloud onto the camera coordinate system. Specifically, node features in radar features contain spatial location and local geometric attribute information, making them more suitable for alignment with shallow feature maps in image features that contain detailed textures and edge information; while edge features in radar features reflect the topological relationships and global structural information between point clouds, making them more suitable for alignment with deep feature maps in image features that express semantic concepts and structural information. For example, shallow feature maps can be generated using the first three convolutional layers of a ResNet-18 network, resulting in a total of three convolutional layers and higher spatial resolution; deep feature maps can be generated using all five convolutional layers of a ResNet-18 network, resulting in a total of five convolutional layers and lower spatial resolution but richer semantic information. This hierarchical alignment strategy fully utilizes the complementary advantages of the two modalities.

[0047] Step S3: Construct a spatiotemporal memory system based on fusion features. The spatiotemporal memory system is used to maintain temporal continuity and scene geometry.

[0048] In this embodiment, the spatiotemporal memory system is a dual-axis fusion memory architecture, consisting of a memory stream on the time axis and a map layer on the spatial axis. The fused features are first organized into a time series and stored in a fixed-length queue according to the acquisition time sequence to form a temporal memory stream, maintaining continuous observation data for the most recent N frames (e.g., N=30). Simultaneously, the fused features are also mapped into three-dimensional space, forming a spatial map layer through geometric constraints and pose optimization, representing the static geometric structure of the scene. The spatiotemporal memory system bridges the temporal memory stream and the spatial map layer through a keyframe mechanism. When a high-quality fused feature frame is detected, keyframe generation is triggered, and both subsystems are updated simultaneously, ensuring spatiotemporal consistency. This design enables the system to maintain both temporal continuity and spatial consistency of perception, maintaining stable perception capabilities even when the quadruped robot is moving at high speed or the environment has dynamic occlusion.

[0049] Step S4: Based on the spatiotemporal memory system, identify and locate the charging pile, generate the charging pile location result, and guide the quadruped robot to move to the charging pile location for charging based on the charging pile location result.

[0050] In practical implementation, charging pile identification and localization can adopt a multi-stage progressive strategy, divided into two stages: coarse localization and fine localization. In the coarse localization stage, historical observation data from the spatiotemporal memory system can be used to identify the approximate location of the charging pile within a relatively large search range through template matching, with an error controlled within 1 meter. In the fine localization stage, the search range is narrowed, and the precise pose of the charging pile can be calculated through multi-constraint optimization, combining appearance information from the temporal memory stream and geometric information from the spatial map layer, with an error controlled at the centimeter level. Finally, the charging pile location result is converted into motion control commands for the quadruped robot. Through path planning and posture adjustment, the robot is guided to precisely move to the charging pile location and complete the charging docking.

[0051] Generally, traditional radar point cloud feature extraction methods directly process the 3D coordinate data of sparse point clouds. This method is extremely sensitive to noise and outliers; even small coordinate deviations can lead to significant changes in feature representation. Secondly, it fails to effectively capture the geometric relationships within the point cloud, ignoring the spatial topological connections between points. This disclosure presents a robust feature extraction method that can extract representative feature points from sparse radar point clouds, effectively eliminating redundant data while preserving key geometric information, thus providing high-quality feature representations for subsequent multimodal fusion tasks. Figure 2 As shown in this embodiment of the disclosure, feature extraction of the original radar point cloud data includes: Step S1.1: Perform voxelization on the original radar point cloud data to obtain a voxel grid; voxelization is to divide the point cloud points corresponding to the original radar point cloud data into a uniform cubic grid in three-dimensional space.

[0052] In this embodiment, voxelization is a key step in point cloud data dimensionality reduction and regularization. The original radar point cloud can first be voxelized, dividing the three-dimensional space into a uniform cubic mesh. Each voxel contains a set of spatially close radar point cloud points; the choice of voxel size requires a trade-off between computational efficiency and feature preservation.

[0053] In one alternative implementation, considering that the LiDAR commonly used in quadruped robots can generate relatively dense point clouds at close range (<10 meters) but become sparse at long range, an adaptive voxel size strategy can be adopted to balance computational efficiency and feature extraction accuracy. Specifically, for example, within a 3-meter radius around the robot, the voxel size can be set to 0.05 meters; within a 3-8 meter radius, the voxel size can be set to 0.1 meters; and beyond 8 meters, the voxel size can be set to 0.2 meters. The voxelization process involves: first, calculating the spatial bounding box of the point cloud, then dividing the bounding box into a regular cubic mesh; next, traversing each point cloud point and calculating its voxel index based on its 3D coordinates (x, y, z). , where s is the voxel size; finally, point cloud points belonging to the same voxel are grouped together. Voxelization greatly reduces the amount of subsequent computation. For example, the original point cloud may contain 100,000 points, while after voxelization, the effective number of voxels can usually be reduced to the thousands.

[0054] Step S1.2: Identify non-empty voxels in the voxel grid, construct an octree data structure for each non-empty voxel, and perform hierarchical analysis on the octree data structure to obtain the radar features of the radar feature points.

[0055] In this embodiment, a non-empty voxel refers to a voxel containing at least one point cloud point. For noisy points in the original point cloud, statistical filtering can be used: if a voxel contains fewer than three point cloud points, it is considered a noisy voxel and filtered out. For the remaining non-empty voxels, an octree data structure is constructed for each voxel. An octree is a three-dimensional spatial partitioning structure where each node represents a spatial region, and each node has eight child nodes, corresponding to the eight quadrants of three-dimensional space. The construction process adopts a top-down recursive approach: first, the entire voxel is used as the root node, and then further subdivision is determined based on the distribution characteristics of the point cloud. The maximum depth of the octree is set to 4 to balance computational complexity and representational accuracy. The octree structure effectively organizes the spatial distribution of the point cloud data, providing a foundation for subsequent hierarchical analysis.

[0056] Furthermore, an octree data structure is constructed for each non-empty voxel, and a hierarchical analysis is performed on the octree data structure, such as... Figure 3 As shown, it includes: Step S1.2.1: Based on the point set within each non-empty voxel, calculate the covariance matrix and perform eigenvalue decomposition to obtain multiple eigenvalues.

[0057] In this embodiment of the disclosure, for each point set within a non-empty voxel First, calculate the centroid coordinates of the point set. Then calculate the covariance matrix. The covariance matrix C is a... A symmetric matrix can be obtained by eigenvalue decomposition, yielding three eigenvalues. and the corresponding feature vector These three eigenvalues ​​reflect the dispersion of the point set along three principal directions, while the eigenvectors represent these three principal directions. The magnitude ratio of the eigenvalues ​​can characterize the geometric properties of the point cloud: if... Then the point cloud is approximately linearly distributed; if If the point cloud is approximately planar, then the point cloud is approximately planar; if The point cloud then has an approximately spherical distribution. This geometric feature extraction method based on Principal Component Analysis (PCA) can effectively identify structural features in the environment, such as walls, edges, and corners.

[0058] Step S1.2.2: Determine the structural complexity of the point cloud in the non-empty voxel region based on the ratio of the largest eigenvalue to the smallest eigenvalue.

[0059] In this embodiment of the disclosure, the structural complexity ρ is defined as the ratio of the largest eigenvalue to the smallest eigenvalue: This ratio reflects the degree of anisotropy in the point cloud distribution. A larger ratio indicates a more concentrated distribution and simpler structure in a particular direction; a smaller ratio indicates a more uniform distribution and more complex structure. For example, a complexity threshold can be set. ,when When, it is considered that the point cloud structure of the current voxel region is simple enough to be represented by a single feature point; when At that time, it was determined that the point cloud structure of the current voxel region was too complex and needed to be further subdivided to capture more refined structural features. This adaptive subdivision strategy based on structural complexity can both preserve key structural information in the environment and effectively control the total number of feature points, thereby improving the efficiency of subsequent processing.

[0060] Step S1.2.3: When the structural complexity exceeds the preset complexity threshold, recursive segmentation is performed until the node satisfies any of the following termination conditions: planar attribute, linear attribute, reaches the maximum recursion depth, or the number of points in the node is less than the minimum number of points threshold, and the radar feature points are determined.

[0061] Here, the segmentation process is carried out recursively step by step. For example, if the structural complexity of the current node is... Then, the node is moved along its direction of maximum variance (corresponding to the eigenvector). Divide the node into two child nodes at its midpoint, and repeat the above process for each child node; if If the segmentation stops, then the segmentation stops. In addition to structural complexity, other termination conditions are set to control the recursion depth and ensure feature quality: (1) Planar properties: for example, if and (1) If the point cloud is approximately planar, then the segmentation is stopped; (2) Linear attributes: for example, if If the point cloud is considered to be approximately linearly distributed, segmentation is stopped; (3) Maximum recursion depth: For example, if the current recursion depth reaches a preset value of 8, segmentation is stopped; (4) Minimum number of points threshold: For example, if the number of points in the current node is less than 4, segmentation is stopped. When any termination condition is met, the current node is marked as a leaf node, and a representative feature point is calculated based on the points in that node. Through this adaptive recursive segmentation strategy, the density of feature points can be automatically adjusted according to the complexity of the environmental structure, using fewer feature points in simple structural areas and more feature points in complex structural areas, thereby efficiently expressing environmental features.

[0062] When the recursive process terminates, a marginalization operation can be performed on the point set within the final node. This operation is essentially a statistical generalization of the local point cloud distribution. By extracting centroid feature points, it achieves the transformation from the original dense point cloud to a sparse feature point cloud while preserving key geometric structure information. Specific implementations include: Step S1.2.3.1: Based on the set of points within a node that satisfies the termination condition, calculate the centroid coordinates of the set of points as the representative position of the node.

[0063] For example, for the set of points within a node that satisfies the termination condition. Calculate its centroid coordinates The centroid coordinates serve as the representative location of the node. They minimize the mean square distance from the point set to the representative point, representing the optimal geometric position of the point set. To improve computational accuracy, outlier detection can be performed on the point set to remove potential outliers. Specifically, this can be achieved by calculating the Mahalanobis distance from each point to the centroid. , This is the inverse of the covariance matrix C. If this distance exceeds a preset threshold (e.g., 2.5), the point is marked as an outlier and excluded from the calculation. Then, the centroid coordinates are recalculated using the remaining points to obtain a more robust and representative location.

[0064] Step S1.2.3.2: Based on the covariance matrix of the point set, calculate the principal direction vector of the node as the normal vector.

[0065] In practical implementation, the principal direction vector of a node can be determined based on the covariance matrix C of the point set and the eigenvectors obtained from eigenvalue decomposition. For planar nodes ( and Select the corresponding smallest eigenvalue. eigenvectors As a normal vector, it represents the direction of the normal to the plane; for linear nodes... Select the corresponding largest eigenvalue eigenvectors As a normal vector, it represents the principal direction of the line; for point nodes (other cases), the corresponding largest eigenvalue is selected. eigenvectors As the normal vector. To ensure consistency in the direction of the normal vector, a global consistency strategy is adopted: for planar nodes, the normal vector direction should point to the open side of the space; for linear nodes, the normal vector direction should be consistent with the normal vectors of adjacent nodes. This consistency process is crucial for subsequent feature matching and fusion.

[0066] Step S1.2.3.3: Combine the representative positions and normal vectors to form representative feature points, and obtain the initial radar feature point set.

[0067] In this embodiment of the disclosure, each representative feature point consists of the following attributes: (1) three-dimensional position coordinates (2) Normal vector , (3) The covariance matrix C represents the spatial distribution of the point set; (4) Eigenvalues (5) Point set size n, representing the number of original point cloud points represented by the feature point; (6) Geometric type This represents the geometric attributes of the feature point. These attributes together constitute a rich feature description, capable of comprehensively expressing the geometric characteristics and statistical properties of the feature point. Through recursive partitioning of the octree, a set of initial radar feature points is finally obtained, typically the number of which is a fraction of the original point cloud points. This greatly reduces the computational complexity of subsequent processing.

[0068] The initial radar feature points in the aforementioned initial radar feature point set may suffer from uneven spatial distribution and geometric redundancy. For example, the same macroscopic plane (such as the ground or a wall) may be segmented into multiple local small planes due to voxel partitioning. These small planes are geometrically continuous but spatially incorrectly split. Therefore, embodiments of this disclosure refine and merge feature points based on geometric continuity and statistical scaling to ensure that the feature points are spatially uniformly distributed and geometrically complete. Thus, in optional embodiments of this disclosure, after obtaining the initial radar feature point set, the following may be included: Step S1.3: Calculate the angle between the normals of adjacent feature points in the initial feature point set.

[0069] In practice, for each pair of adjacent feature points in the initial feature point set Their normal vectors can be calculated. The angle between , where "·" represents the vector dot product operation. Normal angle. The range is [0, π], when the two normal vectors are in the same direction. When the directions are completely opposite Adjacent feature points are defined as: two feature points that are adjacent in location, or the Euclidean distance between two feature points is less than a preset threshold. (For example, set to twice the size of the original voxel). The included normal angle reflects the orientation difference of the local surfaces represented by the two feature points, and is an important basis for determining whether they belong to the same structural plane.

[0070] Step S1.4: Calculate the Mahalanobis distance between the centroids of the feature points based on the covariance matrix of adjacent feature points in the initial feature point set.

[0071] In this embodiment, Mahalanobis distance takes into account the spatial distribution characteristics of feature points and is more suitable than Euclidean distance for expressing the "statistical distance" between non-uniformly distributed point sets. For two feature points... Their Mahalanobis distance is defined as: ,in and These are the position coordinates of two feature points. and These are their covariance matrices. Mahalanobis distance takes into account the shape and orientation of the point set, making the distance contribution along the principal direction of the point set smaller and the distance contribution perpendicular to the principal direction of the point set larger, thus more accurately reflecting the "closeness" of two feature points when considering statistical distribution.

[0072] Step S1.5: When the angle between the normals of two feature points is less than a set angle (e.g., 10 degrees) and the Mahalanobis distance is less than a set distance threshold (e.g., 3), the covariance matrices of the two feature points are weighted and averaged to generate new representative feature points, thus obtaining an optimized feature point set.

[0073] In this embodiment, the merging process employs an iterative strategy: First, an adjacent feature point graph can be constructed, where nodes are feature points and edges represent two feature points satisfying the merging condition (angle between normals). And Mahalanobis distance Then, iteratively perform the following steps until there are no more mergeable point pairs: select the node with the highest degree in the graph (i.e., the feature point with the most mergeable neighbors), merge it with all its matching neighbors, and update the graph structure. Merging two feature points... The specific method is as follows: (1) Position coordinates: That is, a weighted average based on the size of the corresponding point set ( for The number of point clouds in the corresponding point set, for (2) Covariance matrix: (The number of points in the point cloud corresponding to the point set) ,in and These represent the covariance matrices of the point sets represented by the two feature points, reflecting the distribution shape and orientation of the point cloud in three-dimensional space. This formula considers the internal covariance of the two point sets and the distance between the point sets; (3) Normal vector: (3) Normalized sum of two normal vectors; (4) Size of the point set: This merging strategy further reduces the number of feature points while preserving key structural information. The merged feature point set is typically smaller than the initial set. However, their expressive abilities remain largely unchanged.

[0074] The purpose of this step is to determine whether two feature points belong to the same macrostructure. When two feature points simultaneously satisfy both the conditions of normal consistency and statistical borderline, they can be considered to belong to the same macrostructure, and at this time they can be merged to reduce redundant feature points.

[0075] By implementing the above methods, while maintaining planar continuity and overall structural integrity, local redundant feature points can be effectively reduced, thereby generating a sparse and geometrically complete set of feature points, providing robust geometric primitives for subsequent registration, reconstruction and optimization.

[0076] Based on the radar feature point optimization strategy described above, the node features and edge features in the radar features can specifically be the node features of each radar feature point in the optimized feature point set and the edge features between adjacent radar feature points. In specific implementation, the node features and edge features in the radar features can be determined according to the following steps: Step S1.6: Based on the optimized feature point set, use the k nearest neighbor algorithm to find k nearest neighbors for each radar feature point and establish directed connections to construct a graph structure.

[0077] In this embodiment of the disclosure, a graph structure is used to capture the spatial relationships and topological structure between feature points. Feature points serve as nodes in the graph structure, and the initial feature of each node is its three-dimensional coordinate values, using k-nearest neighbors (k-nearest neighbors). The algorithm constructs directed edge connections. For each feature point, it finds its k nearest neighbors in the Euclidean distance space and establishes directed connections from that point to each nearest neighbor. The computational complexity of the graph is controlled by adjusting the value of k.

[0078] In specific implementation, k-nearest neighbors (k-nearest neighbors) are used. The algorithm, for each feature point Find k nearest neighbors Where k can be set to, for example, 6-10. The distance metric used is weighted Euclidean distance: W is a diagonal weight matrix, typically with greater weights in the horizontal direction (x, y) than in the vertical direction (z), to accommodate the characteristic that horizontal structures are more important than vertical structures in an environment. For each pair of feature points... Establish a directed connection ,if yes One of its k-nearest neighbors, then ,otherwise In this way, the set of feature points is organized into a directed graph G=(V,E), where V is the set of feature points and E is the set of directed connections. The graph structure effectively represents the local adjacency relationships and global topological structure of the point cloud, providing a foundation for subsequent graph feature learning.

[0079] Step S1.7: Use the three-dimensional coordinates of each radar feature point in the optimized feature point set as the initial feature vector of the node. Input the initial feature vectors and adjacency relationships of each node in the graph structure into the PCA-GM enhanced GATv2 network for graph feature learning. Calculate the node features of each node and the edge features between adjacent nodes through multi-layer graph attention.

[0080] In this embodiment of the disclosure, graph feature learning employs a graph attention network v2 (GATv2) architecture enhanced by principal component analysis-graph matching (PCA-GM).

[0081] In practical implementation, a two-layer GATv2 network architecture can be adopted. The first layer maps the 3D node coordinate features to a 64-dimensional space, using four independent attention heads, each learning a different node relationship pattern and activated by LeakyReLU. The second layer maps the 64-dimensional features to a 128-dimensional space, using four attention heads and activated by LeakyReLU.

[0082] Here, a fully connected layer maps 3D coordinate features to a 64-dimensional space. The choice of 64 dimensions strikes a balance between "expressive power" and "computational efficiency": it is sufficient to capture the local geometric properties of nodes (such as the distance and angular relationships between adjacent nodes) while avoiding computational redundancy caused by excessive dimensionality (adapting to the real-time inference requirements of quadruped robots). A "multi-attention head" mechanism is adopted, where each attention head independently learns a "node relationship pattern" (e.g., head 1 learns "node-to-node distance correlation", head 2 learns "consistency of node normal vectors", head 3 learns "continuity of the plane to which the node belongs", and head 4 learns "correlation between the node and the global structure"). The outputs of the four attention heads are fused by "concatenation and averaging" to ensure coverage of multiple types of relationship features between nodes and avoid the bias of a single attention head. A nonlinear transformation is introduced through the LeakyReLU activation function to address the problem that "linear mapping cannot capture complex geometric relationships". The advantage of LeakyReLU over traditional ReLU is that it preserves small gradients for negative inputs (e.g., outputting -0.01 instead of 0 when the input is -0.1), avoiding the "activation failure" of some node features due to negative inputs. It is especially suitable for sparse radar point cloud scenarios (where some node-related features may be negative, such as the inverse normal vector).

[0083] The second layer of GATv2, building upon the "local relational features" of the first layer, further learns the association between nodes and the global graph structure, generating high-dimensional features with "global context." The 64-dimensional intermediate features are mapped to a 128-dimensional space. The 128-dimensional design aims to incorporate "global structural information" (such as the macroscopic shape of the entire radar point cloud, and the geometric distinction between charging piles and the background), laying the foundation for subsequent "global graph feature" extraction. Four attention heads are continued, but the learning objective shifts to "global association" (e.g., head 1 learns the "association between nodes and the graph center region," head 2 learns the "association between nodes and charging pile candidate regions," head 3 learns the "difference between nodes and background interference regions," and head 4 learns the "stability of nodes in time-series frames"). Furthermore, the LeakyReLU activation function is maintained consistent with the first layer to ensure the coherence of feature learning and avoid abrupt changes in feature distribution due to activation function changes.

[0084] The role of PCA-GM is to "correct the attention weights of GATv2"—avoiding GATv2 relying solely on data-driven learning (which may ignore geometric rationality, such as connecting distant nodes in space). By introducing "geometric priors of radar point clouds," it ensures the "spatial rationality" and "numerical stability" of the attention weights. The specific process can be divided into three steps: 1) Node similarity calculation: Through the "Query-Key" mechanism, the "feature similarity" of any two nodes is calculated, which provides a basis for subsequent attention weights.

[0085] Specifically, Q / K is first generated through a linear transformation: the node features of the current layer (64 dimensions for the first layer and 128 dimensions for the second layer) are mapped to a "query vector (Q)" and a "key vector (K)" respectively through two independent learnable linear layers; where Q represents "what related features the current node needs", and K represents "what related features other nodes can provide". Then, a similarity matrix is ​​calculated: the correlation between Q and K can be calculated using "dot product similarity". The formula for calculating the correlation between Q and K using "dot product similarity" is: ; where Q i Q is the query vector for the i-th node. j It is the key vector of the j-th node, d is the vector dimension (d=64 in the first layer, d=128 in the second layer), and the denominator is... This is to avoid excessively high similarity values ​​due to high dimensionality and to prevent gradient explosion. Finally, the original similarity matrix is ​​generated: the similarity of all nodes forms an N×N matrix (N is the total number of nodes in the graph structure), and the matrix element sim(i,j) represents the feature correlation between the i-th node and the j-th node.

[0086] 2) Structural constraint fusion: The original attention calculation of GATv2 may result in "spatial unreasonable association" (for example, two nodes that are spatially far apart are given high attention weights due to accidental similarity of features), while the geometric association of radar point clouds only exists in "spatial adjacent nodes" (such as adjacent feature points on the same plane, or consecutive nodes on the same edge). Therefore, it is necessary to introduce "k-nn adjacency matrix" as a constraint.

[0087] Specifically, based on the graph structure described above, an N×N binary adjacency matrix A is constructed. If the j-th node is the k nearest neighbor (the k nodes with the closest spatial distance) of the i-th node, then A... i,j =1, otherwise A i,j =0. Multiply the original similarity matrix and the k-nn adjacency matrix element-wise, using the following formula: This operation ensures that only spatially adjacent nodes ( ) retain their similarity values, and nodes that are not spatially adjacent ( The similarity value is set to 0 to completely filter out invalid associations, making the attention calculation conform to the geometric logic of the radar point cloud.

[0088] 3) Double-random normalization: If the similarity matrix after structural constraints is directly used as attention weights, there may be a problem where the sum of row / column weights is not 1 (e.g., the sum of similarity of all adjacent nodes of a node is 5, while the sum of similarity of another node is 0.5), leading to an unbalanced weight distribution and affecting the stability of subsequent feature fusion. Based on this, Sinkhorn iteration can be used to ensure numerical stability and reasonable weights.

[0089] Through iterative normalization, the attention weight matrix satisfies the "double random matrix" property—the weights in each row sum to 1 (the sum of attention allocated by each node to its neighboring nodes is 1), and the weights in each column sum to 1 (the sum of attention received by each node from its neighboring nodes is 1). The iterative process includes: performing Softmax normalization on each row of the constrained similarity matrix so that the sum of each row is 1; performing Softmax normalization on each column of the row-normalized matrix so that the sum of each column is 1; repeating "row normalization → column normalization" until the sum of the rows and the sum of the columns of the matrix are close to 1 (the error is less than a preset threshold), then stopping the iteration to obtain the final attention weight matrix.

[0090] Double randomization avoids the problem of excessive concentration of weights in some nodes or neglect of weights in others, ensuring that the attention allocation of each node is both reasonable and stable, and providing a reliable weight foundation for subsequent feature fusion.

[0091] The PCA-GM module is not independent of GATv2, but rather integrated into the GATv2 network via residual connections. Its core purpose is to "preserve the integrity of the original GATv2 features while introducing geometric constraints," avoiding feature information loss due to module integration. The output features of a GATv2 layer are element-wise added to the features optimized by the attention weights in PCA-GM to obtain the final output features of that layer. This operation retains the flexibility of data-driven approaches while supplementing the reliability of geometric constraints. Residual connections allow gradients to propagate directly back through the "addition path," avoiding the "feature learning failure" caused by gradual gradient decay in deep networks (such as two-layer GATv2), making it particularly suitable for low-computing scenarios at the end effector of quadruped robots (where complex gradient compensation mechanisms are not required).

[0092] The integration of the PCA-GM module is essentially an application of optimal transport theory in graph neural networks: by optimizing attention weights, the optimal allocation of node features in the graph structure is achieved. The core of optimal transport is to find the optimal path for feature transport under constraints (such as geometric adjacency). PCA-GM ensures that node features are only transported to the most needed neighboring nodes through structural constraint fusion (geometric adjacency constraint) and double random normalization (weight balance constraint), avoiding feature waste or mistransmission, and ultimately enhancing the overall expressive power of the graph structure (such as more accurately distinguishing the geometric boundaries between charging piles and the background).

[0093] The graph neural network enhanced by PCA-GM is used to output two types of 128-dimensional features: "node-level features" and "global graph features (edge ​​features)". These correspond to the different needs of "close-range precise docking" and "long-range global recognition" during the recharging process of the quadruped robot, respectively, and achieve "multi-scale coverage".

[0094] Node-level features are precise representations of local geometry. Each radar feature point corresponds to a 128-dimensional vector containing the node's "local geometric information" (such as distance to neighboring nodes, normal vector consistency, and curvature of the plane it belongs to). This is suitable for the "near-range phase" of recharging. "At this point, the quadruped robot needs millimeter-level positioning accuracy to complete the charging interface docking. Node-level features can accurately reflect the local geometric details of the charging pile interface (such as the edge position of the interface and the flatness of the plane), providing a geometric reference for visual features (such as the texture details of RGB images) and ensuring the posture alignment accuracy during docking."

[0095] Global graph features (edge ​​features) provide an overall perception of the macroscopic structure. Attention pooling is used to aggregate the 128-dimensional features of all nodes: based on the attention weights of each node (optimized by PCA-GM), the node features are weighted and summed to generate a 128-dimensional global vector. The global graph features contain "macroscopic structural information of the entire radar point cloud" (such as the overall shape of the charging pile, its spatial relationship with the surrounding environment, and the density distribution of the point cloud). This is suitable for the "long-distance stage" of recharging (…). "At long distances, it is necessary to quickly identify the general direction of the charging pile. Global graph features can match the "macro structure" with historical templates (such as the typical geometric outline of the charging pile). At medium distances, it is necessary to determine the type of charging pile and the approximate location of the interface. Global graph features can provide "overall semantics" (such as distinguishing the macro differences between the charging pile and the wall and furniture), providing geometric support for visual semantic features (such as the deep semantics of ResNet-18).

[0096] The aforementioned node-level features and global graph features are not used independently, but work together in the subsequent "multimodal fusion module": node-level features are fused with visual shallow features (texture details) to enhance "local positioning accuracy"; global graph features (edge ​​features) are fused with visual deep features (semantic information) to enhance "global recognition robustness"; ultimately achieving full-stage positioning requirement coverage from "far to mid to near", solving the pain points of traditional single-modal perception that are "either insufficient in accuracy or poor robustness".

[0097] In implementation, feature extraction is performed on the image data, such as... Figure 4 As shown, it includes: Step S1.8: Based on the image data, use the ResNet-18 network to perform multi-level convolution processing to obtain feature maps of different resolutions.

[0098] In this embodiment, image feature extraction employs a pre-trained ResNet-18 network, which may contain one initial convolutional layer and four residual blocks. The input image size is, for example, [size missing]. First, the pixels are standardized: the pixel values ​​are scaled to the [0,1] range, and then the mean is subtracted by each channel and divided by the standard deviation. The processed image is then fed into the ResNet-18 network to extract multi-scale feature maps from different layers: (1) Shallow feature maps For example, the output from the first residual block has a size of ( (1) Spatial resolution (width × height) of the feature map, 64 represents the number of feature channels, preserving rich spatial details and texture information; (2) Mid-layer feature map For example, the output from the third residual block has a size of (3) Deep feature map : Output from the last residual block, size is This multi-scale feature extraction strategy contains high-level semantic information but has lower spatial resolution. Shallow feature maps are generated, for example, by three convolutional layers (one initial convolutional layer and two convolutional layers in the first residual block), while deep feature maps are generated, for example, by nine convolutional layers (including all residual blocks), satisfying the requirement that "the number of convolutional layers generating shallow feature maps is less than the number of convolutional layers generating deep feature maps." This strategy enables the simultaneous utilization of both detailed texture information and high-level semantic information in the image.

[0099] Step S1.9: Based on feature maps of different resolutions, construct a multi-level feature map structure to obtain image features.

[0100] In this embodiment, the multi-level feature map structure adopts the design concept of Feature Pyramid Network (FPN) to integrate feature maps of different resolutions. The specific steps are as follows: (1) Processing deep feature maps Perform a 2x upsampling to obtain the same as Same size Feature map (2) and Element-wise addition is performed to obtain a fused feature map. (3) To Perform a 2x upsampling to obtain the same as Same size Feature map (4) and Feature fusion is performed to obtain the final multi-scale feature map F. This top-down path enhances the flow of semantic information in the feature hierarchy, enabling the lower-level feature maps to contain high-level semantic information while preserving their rich spatial details. Ultimately, two types of image features are obtained: (1) shallow feature maps. (1) mainly includes spatial details and texture information; (2) deep feature map It mainly contains semantic concepts and structural information. These two types of features will be aligned and fused with the node features and edge features in the radar features, respectively.

[0101] Traditional multimodal fusion methods typically rely on intermediate representations (such as depth maps or segmentation maps) as bridges to obtain dense features through interpolation or upsampling. This approach suffers from two drawbacks: firstly, the sparsity and noise of the intermediate representations propagate to the final features, leading to a degradation in feature quality; secondly, the multi-stage processing flow is inefficient and struggles to meet the response requirements of real-time recharging tasks for robots. This disclosure presents an efficient single-stage feature-level fusion method that directly establishes cross-modal associations between graph-structured radar features and multi-scale image features, achieving deep fusion of geometric structure and texture information. This provides robust feature representations for subsequent spatiotemporally consistent charging pile localization.

[0102] In this embodiment of the disclosure, feature fusion processing is performed, such as... Figure 5 As shown, it includes: Step S2.1: Based on the aligned multimodal features, with the horizontal coordinates of the radar feature points as the center, set a preset radius range and delineate the corresponding rectangular fusion region in the image feature map.

[0103] In this embodiment, the node features and edge features of the graph structure are paired and fused with adjacent levels in the image pyramid. The node features are aligned with the image detail layer, and the edge features are aligned with the semantic layer, forming a complementary enhancement of geometry and texture.

[0104] Based on the aligned multimodal features, the first step in feature fusion is to determine the corresponding region of the radar feature point in the image feature map. For each radar feature point... First, the coordinates are transformed from the radar coordinate system to the camera coordinate system, and then projected onto the image plane through the camera intrinsic parameter matrix to obtain the two-dimensional pixel coordinates. Considering the sparsity and uncertainty of radar point clouds, feature fusion cannot rely solely on precise projection points; instead, a fusion region should be defined centered on the projection points. Specifically, for shallow feature maps... (size is) ), fusion region radius Set to 8 pixels; for deep feature maps (size is) ), fusion region radius Set to 4 pixels. The blending area is defined as... A square region centered at a point with a side length of 2r. If the fusion region exceeds the feature map boundary, appropriate cropping is performed. This projection-based feature correspondence strategy establishes a spatial correlation between radar features and image features, providing a foundation for subsequent feature fusion.

[0105] Step S2.2: Based on the image features and radar features within the rectangular fusion region, use the image features as the query vector and the radar features as the key-value pairs to calculate the attention weights.

[0106] In this embodiment of the disclosure, feature fusion employs an attention mechanism to achieve adaptive fusion of radar features and image features. For each radar feature point... And its fusion region R in the feature map, representing the image features within the region as , where f(u,v) is the feature vector at position (u,v). Then, the radar features are calculated. Attention weights between (node ​​features or edge features) and each image feature f(u,v) within the region:

[0107] in, This is the projection matrix, projecting radar features into the same feature space as image features. `d` is the feature dimension, and the softmax function ensures that the sum of attention weights at all locations within the region is 1. This attention mechanism, based on dot product similarity, adaptively identifies the parts of image features most relevant to radar features, effectively handling the correspondence uncertainty and feature differences between the two modalities. The attention weights reflect the degree of response of image features to radar features at each location; a larger value indicates a higher correlation between the features at that location and the radar features.

[0108] Step S2.3: Based on attention weights, perform weighted fusion of image features and radar features to obtain fused features.

[0109] In this embodiment, the final step of feature fusion is to weight and combine image features and radar features based on attention weights. Different fusion strategies are used to process different types of feature pairs: node features with shallow feature maps, and edge features with deep feature maps.

[0110] Furthermore, based on attention weights, image features and radar features are weighted and fused, including: Step S2.3.1: Based on attention weights, perform weighted summation of node features of shallow feature map and radar feature points to enhance spatial positioning features; and, based on attention weights, perform weighted summation of edge features of deep feature map and radar feature points to extract global structural context features.

[0111] In this embodiment, the calculation formula for fusing node features and shallow feature maps is as follows: ; in, It is the fused node features (output). These are the nodal features of radar feature points. It represents the feature at position (u,v) in the shallow feature map. It is a balance factor (e.g., set to 0.5). This represents the summation of features within a region. This is for attention weights. This weighted summation operation enhances the spatial localization capability of node features, enabling them to accurately locate using detailed texture information in the image. Shallow detail fusion provides a foundation for close-range precise docking by enhancing spatial localization accuracy.

[0112] The formula for fusing edge features and deep feature maps is as follows:

[0113] in, These are the edge features between radar feature points. It is the feature at position (u,v) in the deep feature map. This is a balancing factor (usually set to 0.3). This fusion operation enhances the semantic expressiveness of edge features, enabling them to utilize high-level structural information in the image to describe the relationships between feature points. Deep structural fusion, by extracting global structural context, can support long-distance charging pile identification.

[0114] The aforementioned hierarchical control strategy uses the output of each fusion layer as the input for the next layer, forming a bottom-up feature enhancement process that ensures efficient integration of multi-scale information. This hierarchical fusion strategy fully leverages the complementary advantages of the two modalities: radar features provide accurate 3D geometric information, while image features provide rich appearance and semantic information. The fused features retain the key information of the original features while adding cross-modal complementary information, improving the expressive power and robustness of the features.

[0115] Step S2.3.2: Concatenate the enhanced spatial localization features and the global structural context features to obtain the fused features.

[0116] In this embodiment of the disclosure, the final fused feature consists of enhanced node features and edge features. For each radar feature point... Its fusion characteristics are represented as follows: ; in, It is an enhanced node feature. It is an enhanced edge feature connected to neighbor node j. Let be the set of neighbors of node i, and ";" denotes feature concatenation operation. Considering that the number of neighbors of each node may be different, attention pooling is used to convert the variable-length edge feature set into a fixed-length feature vector:

[0117] in, It is a pooling operation based on an attention mechanism, which can adaptively focus on the most important edge features and output a fixed-dimensional feature vector. This combined feature includes both the local attributes of the node and the relationship between the node and its neighbors, providing a comprehensive expression of the feature point and its context.

[0118] The shallow fusion features obtained through the above method can support near-range millimeter-level positioning, while the deep fusion features can meet the requirements of long-range rapid identification and adapt to the multi-stage accuracy requirements of recharging tasks.

[0119] In the autonomous recharging task of quadruped robot dogs, it is typically necessary to quickly identify charging piles from a distance of 5 meters and maintain stable positioning while in continuous movement, until millimeter-level precise docking is achieved. This process places extremely high demands on the spatiotemporal continuity of environmental perception. Multiple challenges arise in real dynamic environments: dynamic disturbances: changes in lighting, partial occlusion, and dynamic background objects cause frequent fluctuations in the appearance of the charging pile; the impact of its own motion: bumps and pitch rotations during walking cause instantaneous changes in the camera's perspective and radar echo structure, resulting in drastic drift of target features; short-term instability: in scenarios involving rapid turning or jumping, the target may temporarily leave the field of view, and traditional methods cannot maintain positioning continuity. Traditional Simultaneous Localization and Mapping (SLAM) systems focus on maintaining spatial topology, constructing a spatial hierarchy from real-time frames to a global map, but lack the ability to adapt to temporal changes in target appearance. Temporal memory modules focus on the temporal consistency of multimodal features, maintaining target association even with drastic appearance changes, but lack integration with the geometric constraints of the spatial map. To this end, this disclosure proposes a spatiotemporal memory hierarchical fusion architecture based on keyframe bridging, which deeply couples temporal and spatial information, uses keyframes as spatiotemporal association nodes, and establishes a bidirectional mapping relationship between the temporal memory stream and the spatial map layer to achieve unified maintenance of appearance consistency and geometric consistency.

[0120] like Figure 6 The diagram illustrates the construction of a spatiotemporal memory system, primarily building a hierarchical structure across two dimensions: time and space, bridged by keyframes. Figure 7 As shown, the process of constructing a spatiotemporal memory system includes: Step S3.1: Based on the fusion features, construct a temporal memory stream by storing the feature information of consecutive frames in a real-time frame queue; and, based on the fusion features, construct a spatial map layer by geometric structure modeling and optimization.

[0121] In this embodiment of the disclosure, the spatiotemporal memory system consists of two subsystems: a temporal memory stream and a spatial map layer, which maintain the continuity and consistency of perceived information from both temporal and spatial dimensions.

[0122] The time memory stream uses a real-time frame queue to store the fusion features of the most recent N frames (usually N=30, corresponding to 3 seconds of observation data) in chronological order. Specifically, it is implemented as a fixed-length circular queue. When a new frame arrives, the oldest frame is removed from the queue, and the new frame is added to the tail of the queue. Each frame contains the following information: (1) timestamp t; (2) robot pose. (3) Fusion feature set (4) Confidence score The main function of temporal memory stream is to maintain the temporal continuity of perception results, and it can handle dynamic environmental factors such as short-term occlusion and changes in viewpoint, providing stable target tracking capabilities.

[0123] The spatial map layer adopts a geometric structure modeling approach, mapping fused features to a global coordinate system to construct a spatial representation of the scene. Specific implementations include: (1) Point cloud registration: using the Iterative Closest Point (ICP) algorithm to register point clouds across consecutive frames and estimate relative pose transformations; (2) Pose map optimization: constructing a pose map and using the g2o library for backend optimization to reduce accumulated errors; (3) Feature point management: maintaining a global feature point map, including adding new feature points, updating existing feature points, and removing unstable feature points; (4) Spatial indexing: using an octree or k-dimensional (KD) tree to establish a spatial index, improving feature point query efficiency. The main function of the spatial map layer is to maintain the geometric consistency of the scene and provide accurate spatial positioning capabilities.

[0124] These two subsystems complement each other: the temporal memory stream provides short-term continuous observations, suitable for handling dynamic changes; the spatial map layer provides long-term structural constraints, suitable for handling static environments. By combining these two different types of memory, spatial consistency can be maintained while perceptual continuity is preserved, providing a stable and reliable foundation for charging pile identification and positioning.

[0125] Step S3.2: Based on the temporal memory stream and the spatial map layer, perform dual-axis fusion through a keyframe bridging mechanism to obtain the spatiotemporal memory system.

[0126] In this embodiment of the disclosure, the keyframe bridging mechanism is a core component connecting the temporal memory stream and the spatial map layer. By selectively mapping key observations on the time axis to the spatial axis, it enables collaborative updates and information sharing between the two subsystems.

[0127] Furthermore, based on the temporal memory stream and spatial map layer, a dual-axis fusion is performed through a keyframe bridging mechanism to obtain a spatiotemporal memory system, including: Step S3.2.1: Based on the time memory stream, when a fused feature frame with a confidence level greater than the set confidence level is detected, the fused feature frame is used as a visual key frame, and a key frame trigger signal is generated.

[0128] In this embodiment, keyframe selection is based on two criteria: feature quality and information gain. A confidence score can be calculated for each frame in the temporal memory stream. The score can be determined by the following factors: (1) Number of feature points: the more feature points, the greater the information content; (2) Distribution of feature points: the more uniform the distribution of feature points, the wider the coverage; (3) Stability of feature points: the higher the consistency of feature point matching in consecutive frames, the better the stability; (4) Uniqueness of feature points: the greater the difference between feature points and historical observations, the higher the information gain. These factors are combined in the form of a weighted sum to form the final confidence score: .

[0129] in, These are weighting parameters, typically set to 0.2, 0.3, 0.3, 0.2. When the confidence score... When the threshold τ is exceeded (usually set to 0.7), a keyframe trigger signal is generated, marking the current frame as a keyframe (visual keyframe). Furthermore, the minimum time interval (usually 0.5 seconds) and minimum spatial interval (usually 0.5 meters) between keyframes can be set to prevent keyframes from becoming too densely packed. The keyframe trigger signal can contain information such as frame index, timestamp, and robot pose, used for subsequent spatial keyframe construction.

[0130] Step S3.2.2: Based on the keyframe trigger signal, trigger the construction of the corresponding keyframe in the spatial map layer, and obtain the spatial keyframe by marginalizing and compressing the statistical distribution features in the space.

[0131] In specific implementation, when a keyframe trigger signal is received, the spatial map layer initiates the construction process of the spatial keyframe. The spatial keyframe is a spatial mapping of the keyframe in the temporal memory stream, containing richer geometric structure information and statistical characteristics. The construction process includes the following steps: (1) Feature point transformation: transforming the fused feature points in the keyframe to the global coordinate system; (2) Local map extraction: extracting a local map with a radius of, for example, 5 meters around the keyframe from the spatial map layer; (3) Feature matching: using the KNN (k-nearest neighbor) algorithm to find the point in the local map that best matches the feature points of the keyframe; (4) Pose refinement: based on the feature matching results, using the Random Sample Consensus (RANSAC) algorithm to estimate a more accurate pose transformation; (5) Statistical feature calculation: calculating the statistical distribution features of the matched feature point pairs, including mean, covariance, principal component direction, etc.

[0132] The edge detection operation filters the real-time frame queue of the spatial map layer, discarding redundant frames with low confidence and poor feature quality (such as frames affected by occlusion or noise), and retaining only frames with high confidence and good feature quality as keyframes. The spatial statistical distribution features carried by the keyframes are statistical summaries of multiple frames in the real-time frame queue. Instead of retaining the complete point cloud distribution of each frame, it integrates core statistical information such as centroid and covariance of multiple frames to form compressed features that combine spatiotemporal continuity and geometric accuracy.

[0133] The calculation of statistical distribution characteristics employs a Bayesian framework, combining prior knowledge with observed data to obtain the posterior distribution. A specific formula is as follows:

[0134]

[0135] in, Prior and The priors are the previously estimated mean and covariance. Observation and The observations are the mean and covariance of the current observations. Posterior and The posterior is the updated mean and covariance. This feature representation based on statistical distribution not only compresses the amount of data but also retains uncertainty information, thus enabling adaptive adjustment of the confidence level for different features according to the reliability of the observations.

[0136] Step S3.2.3: Based on statistical distribution characteristics, accumulate multiple spatial map layer update tasks to a preset batch size and then perform batch matrix operations in a unified manner to obtain a spatiotemporal memory system.

[0137] In this embodiment, to improve computational efficiency, a batch processing strategy can be adopted, accumulating multiple spatial map layer update tasks to a certain number (e.g., 5-10) before performing batch matrix operations. Batch processing is mainly for computationally intensive operations, such as covariance matrix inversion and eigenvalue decomposition, which can be efficiently executed in parallel on modern GPUs. Specifically, the CUDA Basic Linear Algebra Subprograms (cuBLAS) and CUDA Solver Library (cuSOLVER) can be used to fully utilize GPU-accelerated matrix computation. The choice of batch size needs to balance real-time performance and computational efficiency: too small a batch results in high startup overhead and low efficiency; too large a batch increases processing latency and affects real-time performance. Experiments have shown that, under the hardware configuration of the optional implementation of this disclosure, a batch size of 8 achieves the optimal balance, and the processing latency can be controlled within 50 milliseconds.

[0138] After batch matrix operations are completed, the updated statistical distribution features are integrated into the spatial map layer, while keyframes in the temporal memory stream associated with these features are also updated. This bidirectional update mechanism ensures the consistency between the temporal memory stream and the spatial map layer, enabling the two subsystems to mutually correct and complement each other. The final spatiotemporal memory system, through a keyframe bridging mechanism, organically combines temporal continuity and spatial consistency to form a comprehensive and stable perceptual representation of the environment, providing a reliable foundation for subsequent charging pile identification and positioning.

[0139] Charging pile target identification and positioning based on spatiotemporal memory system, such as Figure 8 As shown, it includes: Step S4.1: Based on the spatiotemporal memory system, use the historical appearance templates in the time memory stream and the geometric structure information in the spatial map layer to match the charging pile target.

[0140] In this embodiment, the charging pile target matching adopts a multimodal fusion approach, utilizing both visual appearance features and geometric structure features for matching. First, historical appearance templates of the charging piles are extracted from the time-memory stream. Each template This includes visual feature descriptions of the charging stations. These templates can be initialized manually with annotations when the robot first uses the charging station, or they can be gradually accumulated through online learning. Then, regions matching the templates are searched in the fused features of the current observation frame, and a matching score is calculated.

[0141] Here, r represents the candidate region in the current frame, and similarity is the feature similarity function, typically using cosine similarity. Simultaneously, a geometric structure template G for the charging pile is extracted from the spatial map layer, containing the charging pile's shape, size, and local environmental features. A structure matching the geometric template is searched in the point cloud of the current frame, and the geometric matching degree is calculated.

[0142] in, The point cloud registration score, based on the ICP algorithm, reflects the degree of matching of the point cloud structure. Finally, the appearance matching score and geometric matching score are combined to obtain the comprehensive matching score.

[0143] Here, α is a balancing parameter, for example, set to 0.6. The region with the highest matching degree is identified as the charging pile target. This multimodal fusion matching strategy utilizes both the rich appearance information of visual features and the stable structural information of geometric features, and can maintain stable recognition performance under different environmental conditions (such as changes in lighting and partial occlusion).

[0144] Step S4.2: When the target matching confidence is lower than the preset threshold, retrieve the historical appearance template in the time memory stream and infer the possible area of ​​the target in the spatial map layer.

[0145] In this embodiment of the disclosure, when the confidence level of the charging pile target matching is lower than a preset threshold... When the value is set to 0.7 (for example), the current observations are considered insufficient to reliably identify the charging station, and a recovery strategy needs to be initiated. The recovery strategy includes two aspects: historical appearance retrieval and spatial location estimation.

[0146] Historical appearance retrieval: This involves searching the historical frames of the time-memory stream for the most recent high-confidence observations and extracting their appearance features as enhancement templates. Specifically, iterate through the most recent M frames (typically M=50) of historical observations, identify the set of frames with confidence levels exceeding a threshold, extract appearance features from these frames, and cluster them to obtain K representative templates (e.g., K=3). These enhancement templates can adapt to changes in environment and perspective, improving the robustness of template matching.

[0147] Spatial location estimation: Utilizing pose information from the spatial map layer and the historical location of the charging station, the possible location area of ​​the charging station at the current moment is estimated. Specifically, assuming the charging station is stationary, the robot's current pose is used to estimate the location area of ​​the charging station. Historical location and orientation of charging piles Calculate the predicted location of the charging station in the current field of view: Then, taking into account the uncertainty of pose estimation, a search area is set around the predicted position. .

[0148] By combining historical appearance retrieval and spatial location estimation, the system can effectively address situations with insufficient observation and maintain continuous tracking of charging stations. This recovery strategy is crucial for the autonomous recharging of quadruped robots in complex environments, enabling them to cope with challenging scenarios such as short-term occlusion, changes in lighting, and sudden changes in perspective.

[0149] Step S4.3: Within the possible target area, the location of the charging pile is determined by the intersection filtering and fine matching of the time memory stream and the spatial map layer, and the location result of the charging pile is obtained.

[0150] In this embodiment of the disclosure, determining the precise location of the charging station requires fine-grained matching and filtering within the target potential area. The intersection area of ​​the temporal memory stream and the spatial map layer prediction results can be calculated first: ;in, It is a region predicted based on temporal memory stream. It is a region extrapolated based on the spatial map layer. The intersection region I represents a high-probability region jointly recognized by the two subsystems, which can effectively reduce false alarms and mismatches.

[0151] Within the intersection region, a multi-level fine-matching strategy can be employed to determine the precise location and orientation of the charging pile, including: coarse registration: using the Sample Consensus Initial Alignment (SAC-IA) algorithm to coarsely register the candidate point cloud and the template point cloud to obtain a preliminary pose estimate; fine registration: based on the coarse registration result, using the Iterative Closest Point (ICP) algorithm to perform fine registration and iteratively optimize the pose estimate; geometric verification: calculating the overlap rate and normal consistency of the registered point cloud to ensure the accuracy of the geometric structure matching; semantic verification: using the semantic information in the fused features to verify whether the candidate region contains key components of the charging pile (such as charging ports, indicator lights, etc.); temporal consistency check: comparing the continuity between the current estimate and historical estimates to filter out abnormal estimates that do not conform to motion constraints.

[0152] After multiple levels of screening and verification, the final location of the charging station is obtained. This final location result may include: the charging station's six-degree-of-freedom pose, the uncertainty covariance matrix of the pose estimation, and the confidence score of the location result. This information will be used for subsequent path planning and precise docking control.

[0153] This disclosure achieves autonomous recharging and positioning of the quadruped robot by acquiring raw radar point cloud data and image data during its movement, performing feature extraction and hierarchical fusion on these data, constructing a spatiotemporal memory system, and using this system for charging station identification and positioning. This disclosure employs multimodal hierarchical fusion and spatiotemporal memory hierarchical fusion technologies, improving the accuracy and robustness of positioning, adapting to recharging needs at different distance stages, and maintaining temporal consistency of perception results during robot movement.

[0154] like Figure 9 As shown in the figure, this disclosure also provides a quadruped robot recharging and positioning device 900, the device comprising: The feature extraction module 91 is used to acquire the original radar point cloud data and image data collected by the quadruped robot during the movement of the quadruped robot, and to extract features from the original radar point cloud data and image data respectively to determine radar features and image features. The feature fusion module 92 is used to align the node features in the radar features with the shallow feature maps in the image features, and to align the edge features in the radar features with the deep feature maps in the image features, to obtain aligned multimodal features. Based on the aligned multimodal features, feature fusion processing is performed to obtain fused features. The number of convolutional layers used to generate the shallow feature maps is less than the number of convolutional layers used to generate the deep feature maps. Spatiotemporal memory construction module 93 is used to construct a spatiotemporal memory system based on fusion features. The spatiotemporal memory system is used to maintain temporal continuity and scene geometric structure. The positioning and guidance module 94 is used to identify and locate the charging pile based on the spatiotemporal memory system, generate the charging pile location result, and guide the quadruped robot to move to the charging pile location for charging based on the charging pile location result.

[0155] The functions of each module of the device correspond to the steps of the aforementioned method embodiments, and can be understood by referring to the relevant descriptions of the method embodiments, which will not be repeated here.

[0156] The feature extraction module 91 can include a point cloud processing unit and an image processing unit. The point cloud processing unit is responsible for processing the raw radar point cloud data through voxelization, octree construction, and feature point extraction to generate radar features. The image processing unit is responsible for processing the image data through ResNet feature extraction and multi-level feature map construction to generate image features. These two units can work in parallel to improve processing efficiency.

[0157] The feature fusion module 92 may include a feature alignment unit and an attention fusion unit. The feature alignment unit is responsible for establishing the spatial correspondence between radar features and image features and determining the fusion region; the attention fusion unit is responsible for calculating attention weights and performing weighted fusion operations to generate fused features.

[0158] The spatiotemporal memory construction module 93 can include a temporal memory unit and a spatial map unit. The temporal memory unit is responsible for maintaining the temporal memory stream and handling the temporal relationships between consecutive frames; the spatial map unit is responsible for maintaining the spatial map layer and managing global feature points and geometric structures. These two units are connected and collaboratively updated through a keyframe bridging unit.

[0159] The positioning and guidance module 94 may include a target matching unit, a position estimation unit, and a path planning unit. The target matching unit is responsible for identifying the charging pile and estimating its position; the position estimation unit is responsible for estimating the possible position of the charging pile when observation is insufficient; and the path planning unit is responsible for generating motion control commands for guiding the robot based on the positioning results.

[0160] For details on the implementation of each module, please refer to the specific description of the aforementioned methods. The specific implementation of the aforementioned methods can be achieved through each module, which will not be elaborated here.

[0161] like Figure 10 As shown, this embodiment of the present disclosure also provides a quadruped robot, which is equipped with a lidar 101, a camera 102 and a controller 103; The lidar 101 is used to acquire raw radar point cloud data during the movement of the quadruped robot; the camera 92 is used to acquire image data during the movement of the quadruped robot. The controller 103 is used to acquire raw radar point cloud data and image data, and to extract features from the raw radar point cloud data and image data respectively to determine radar features and image features; aligning the node features in the radar features with the shallow feature maps in the image features, and aligning the edge features in the radar features with the deep feature maps in the image features, to obtain aligned multimodal features, and performing feature fusion processing based on the aligned multimodal features to obtain fused features; wherein the number of convolutional layers generating the shallow feature maps is less than the number of convolutional layers generating the deep feature maps; constructing a spatiotemporal memory system based on the fused features, the spatiotemporal memory system is used to maintain temporal continuity and scene geometric structure; performing charging pile identification and positioning based on the spatiotemporal memory system, generating charging pile location positioning results, and guiding the quadruped robot to move to the charging pile location for charging based on the charging pile location positioning results.

[0162] The functions performed by controller 103 correspond to the steps of the aforementioned method embodiments, and can be understood by referring to the relevant descriptions of the method embodiments, which will not be repeated here.

[0163] In this embodiment of the disclosure, the specific implementation of the quadruped robot includes the following components: The quadruped mechanical body 100 features a lightweight design with three joints on each of its four legs. Each joint is equipped with a high-precision servo motor and encoder, supporting multiple gaits (such as crawling and trotting), with a maximum walking speed of up to 2m / s, and adapting to different terrains.

[0164] LiDAR 101: Mounted on the upper part of the robot, it uses a 16-line or 32-line LiDAR with a horizontal field of view of 360 degrees, a vertical field of view of 30 degrees, a measurement range of 100 meters, a point cloud acquisition frequency of 10Hz, and a resolution better than 2cm, providing accurate three-dimensional geometric information for recharge positioning.

[0165] Camera 102: Mounted at the front of the robot, it is a 1080P high-definition RGB camera with a 90-degree field of view and a frame rate of 30fps. It is equipped with automatic exposure and white balance functions, providing rich visual information for recharging and positioning.

[0166] Controller 103: It adopts an embedded computing platform, such as NVIDIA Jetson AGX Xavier, equipped with an 8-core ARM CPU, a 512-core CUDA GPU and 32GB of memory, running the Ubuntu 18.04 operating system and ROS (Robot Operating System), and is responsible for sensor data processing, algorithm execution and motion control.

[0167] Power system 104: Includes a high-energy-density lithium battery pack with a rated capacity of approximately 4400mAh, an operating voltage of 24V, and supports fast charging. It can work continuously for 2-3 hours after being fully charged.

[0168] Communication system 105: Includes a Wi-Fi module (supporting 802.11ac) and a 4G / 5G module, enabling data exchange with remote control terminals and cloud services, and supporting remote monitoring and task assignment.

[0169] Other sensors, including IMU (Inertial Measurement Unit), odometry, and contact sensors, provide the robot with additional status information and environmental awareness.

[0170] Through the coordinated operation of these hardware components, the quadruped robot achieves autonomous movement, environmental perception, and precise recharging. Data collected by LiDAR and cameras is processed by the controller to generate fused features and a spatiotemporal memory system for identifying and locating charging stations. Based on the location results, the controller generates motion control commands to drive the joint motors of the four legs, guiding the robot to move precisely to the charging station location and complete autonomous recharging.

[0171] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the quadruped robot. In other embodiments of this application, the quadruped robot may include more or fewer components than illustrated, or combine some components, or separate some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0172] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the quadruped robot recharge and positioning method as described in any of the above embodiments.

[0173] Computer-readable storage media can be any usable medium accessible to a computer or a data storage device such as a server or data center that integrates one or more usable media. Usable media can be magnetic media, such as hard disks, floppy disks, and magnetic tapes; optical media, such as DVD-ROM, DVD-RAM, DVD-RW, DVD+RW, CD-ROM, CD-R, CD-RW, MO (magneto-optical) storage media; and semiconductor storage media, such as flash memory, EEPROM, Dynamic Random Access Memory (DRAM), and Static Random Access Memory (SRAM).

[0174] The computer program can be written in various computer programming languages, including but not limited to C, C++, Python, and custom messages and services under the ROS framework. When the computer program is executed by the processor, it implements the various steps of the quadruped robot recharging and positioning method in the embodiments of this disclosure.

[0175] This disclosure also provides a computer program product, which stores a computer program. When the computer program is run by a processor, it executes the steps of the quadruped robot recharging and positioning method provided in any of the above embodiments of this disclosure. For details, please refer to the above method embodiments, which will not be repeated here.

[0176] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0177] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and apparatuses described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0178] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0179] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0180] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0181] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for recharging and positioning a quadruped robot, characterized in that, include: During the movement of the quadruped robot, the original radar point cloud data and image data collected by the quadruped robot are acquired, and feature extraction is performed on the original radar point cloud data and image data respectively to determine radar features and image features. The node features in the radar features are aligned with the shallow feature maps in the image features, and the edge features in the radar features are aligned with the deep feature maps in the image features to obtain aligned multimodal features. Based on the aligned multimodal features, feature fusion processing is performed to obtain fused features. The number of convolutional layers used to generate the shallow feature maps is less than the number of convolutional layers used to generate the deep feature maps. Based on the fusion features, a temporal memory stream is constructed by storing the feature information of continuous fusion feature frames in a real-time frame queue; and, based on the fusion features, a spatial map layer is constructed through geometric structure modeling and optimization; based on the temporal memory stream and the spatial map layer, a dual-axis fusion is performed through a keyframe bridging mechanism to obtain a spatiotemporal memory system; the spatiotemporal memory system is used to maintain temporal continuity and scene geometry. Based on the spatiotemporal memory system, the charging pile is identified and located, generating a charging pile location result. Based on the charging pile location result, the quadruped robot is guided to move to the charging pile location for charging. The process of obtaining the spatiotemporal memory system by performing dual-axis fusion based on the temporal memory stream and the spatial map layer through a keyframe bridging mechanism includes: based on the temporal memory stream, when a fused feature frame with a confidence level greater than a set confidence level is detected, the fused feature frame is used as a visual keyframe, and a keyframe trigger signal is generated; based on the keyframe trigger signal, the construction of the corresponding keyframe in the spatial map layer is triggered, and spatial keyframes are obtained by marginalizing and compressing statistical distribution features in the space; based on the statistical distribution features, multiple spatial map layer update tasks are accumulated to a preset batch size and then batch matrix operations are performed uniformly to obtain the spatiotemporal memory system.

2. The method according to claim 1, characterized in that, Feature extraction is performed on the original radar point cloud data, including: The original radar point cloud data is voxelized to obtain a voxel grid; the voxelization process involves dividing the point cloud points corresponding to the original radar point cloud data into a uniform cubic grid in three-dimensional space. Non-empty voxels in the voxel grid are identified, an octree data structure is constructed for each non-empty voxel, and hierarchical analysis is performed on the octree data structure to obtain the radar features of the radar feature points.

3. The method according to claim 2, characterized in that, An octree data structure is constructed for each non-empty voxel, and a hierarchical analysis is performed on the octree data structure, including: Based on the point set within each non-empty voxel, the covariance matrix is ​​calculated and eigenvalue decomposition is performed to obtain multiple eigenvalues. The structural complexity of the point cloud in the non-empty voxel region is determined based on the ratio of the largest to the smallest eigenvalue among the plurality of eigenvalues. When the structural complexity exceeds a preset complexity threshold, recursive segmentation is performed until the node satisfies any of the following termination conditions: planar attribute, linear attribute, reaches the maximum recursion depth, or the number of points in the node is less than the minimum number of points threshold, and radar feature points are determined.

4. The method according to claim 3, characterized in that, Radar feature points are determined until any termination condition is met at a node, including: Based on the set of points within a node that satisfies the termination condition, the centroid coordinates of the set of points are calculated as the representative position of the node. Based on the covariance matrix of the point set, the principal direction vector of the node is calculated as the normal vector; The representative locations and the normal vectors are combined to form representative feature points, thus obtaining an initial set of radar feature points.

5. The method according to claim 4, characterized in that, After obtaining the initial radar feature point set, it also includes: Based on the normal vectors of adjacent feature points in the initial radar feature point set, calculate the angle between the normal vectors of adjacent feature points; Based on the covariance matrix of adjacent feature points in the initial radar feature point set, the Mahalanobis distance between the centroids of adjacent feature points is calculated. When the angle between the normals of adjacent feature points is less than a set angle and the Mahalanobis distance is less than a set distance threshold, the covariance matrices of the adjacent feature points are weighted and averaged to generate new representative feature points, thus obtaining an optimized feature point set.

6. The method according to any one of claims 1 to 5, characterized in that, The node features and edge features in the radar features are determined according to the following steps: The k-nearest neighbor algorithm is used to find k nearest neighbors for each radar feature point and establish directed connections to construct a graph structure; The three-dimensional coordinates of each radar feature point are used as the initial feature vector of the node. The initial feature vectors and adjacency relationships of each node in the graph structure are input into the graph attention network GATv2 enhanced by principal component analysis-graph matching (PCA-GM) for graph feature learning. The node features of each node and the edge features between adjacent nodes are obtained through multi-layer graph attention calculation.

7. The method according to claim 1, characterized in that, Feature extraction of the image data includes: Based on the image data, a ResNet-18 network is used for multi-level convolution processing to obtain feature maps of different resolutions. Based on the feature maps of different resolutions, a multi-level feature map structure is constructed to obtain the image features.

8. The method according to claim 1, characterized in that, Feature fusion processing is performed, including: Based on the aligned multimodal features, a preset radius range is set with the horizontal coordinates of the radar feature points as the center, and a corresponding rectangular fusion region is delineated in the image feature map. Based on the image features and radar features within the rectangular fusion region, the image features are used as query vectors, and the radar features are used as key-value pairs to calculate attention weights. Based on the attention weights, image features and radar features are weighted and fused to obtain the fused features.

9. The method according to claim 8, characterized in that, Based on the attention weights, image features and radar features are weighted and fused, including: Based on the attention weights, the node features of the shallow feature map and the radar feature points are weighted and summed to enhance the spatial positioning features; and based on the attention weights, the edge features of the deep feature map and the radar feature points are weighted and summed to extract the global structural context features. The enhanced spatial localization features and the global structural context features are concatenated to obtain the fused features.

10. The method according to claim 1, characterized in that, Based on the spatiotemporal memory system, target identification and positioning of charging piles are performed, including: Based on the spatiotemporal memory system, the historical appearance templates in the time memory stream are used to match the geometric structure information in the spatial map layer to perform charging pile target matching. When the target matching confidence is lower than a preset threshold, historical appearance templates are retrieved from the time memory stream and the possible target area is estimated in the spatial map layer. Within the target potential area, the location of the charging pile is determined by the intersection filtering and fine matching of the time memory stream and the spatial map layer, thus obtaining the location result of the charging pile.

11. A quadruped robot recharging and positioning device, characterized in that, The device includes: The feature extraction module is used to acquire the original radar point cloud data and image data collected by the quadruped robot during the movement of the quadruped robot, and to extract features from the original radar point cloud data and image data respectively to determine radar features and image features. The feature fusion module is used to align the node features in the radar features with the shallow feature maps in the image features, and to align the edge features in the radar features with the deep feature maps in the image features, to obtain aligned multimodal features. Based on the aligned multimodal features, feature fusion processing is performed to obtain fused features. The number of convolutional layers used to generate the shallow feature maps is less than the number of convolutional layers used to generate the deep feature maps. A spatiotemporal memory construction module is used to construct a temporal memory stream based on the fusion features by storing feature information of continuous fusion feature frames in a real-time frame queue; and to construct a spatial map layer based on the fusion features through geometric structure modeling and optimization; and to obtain a spatiotemporal memory system by performing dual-axis fusion based on the temporal memory stream and the spatial map layer through a keyframe bridging mechanism; the spatiotemporal memory system is used to maintain temporal continuity and scene geometry; wherein, when obtaining the spatiotemporal memory system by performing dual-axis fusion based on the temporal memory stream and the spatial map layer through a keyframe bridging mechanism, the module is used to, based on the temporal memory stream, when a fusion feature frame with a confidence level greater than a set confidence level is detected, use the fusion feature frame as a visual keyframe and generate a keyframe trigger signal; based on the keyframe trigger signal, trigger the construction of corresponding keyframes in the spatial map layer, and obtain spatial keyframes by marginalizing and compressing statistical distribution features in the space; based on the statistical distribution features, accumulate multiple spatial map layer update tasks to a preset batch size and then uniformly perform batch matrix operations to obtain the spatiotemporal memory system; The positioning and guidance module is used to identify and locate the charging pile based on the spatiotemporal memory system, generate the charging pile location result, and guide the quadruped robot to move to the charging pile location for charging based on the charging pile location result.

12. A quadruped robot, characterized in that, The quadruped robot is equipped with a lidar, a camera, and a controller. The lidar is used to collect raw radar point cloud data during the movement of the quadruped robot; the camera is used to collect image data during the movement of the quadruped robot. The controller is used to acquire the original radar point cloud data and image data, and to extract features from the original radar point cloud data and image data respectively to determine radar features and image features; align the node features in the radar features with the shallow feature maps in the image features, and align the edge features in the radar features with the deep feature maps in the image features to obtain aligned multimodal features; and perform feature fusion processing based on the aligned multimodal features to obtain fused features; wherein the number of convolutional layers generating the shallow feature maps is less than the number of convolutional layers generating the deep feature maps; based on the fused features, a temporal memory stream is constructed by storing the feature information of continuous fused feature frames in a real-time frame queue; and, based on the fused features, a spatial map layer is constructed by geometric structure modeling and optimization; based on the temporal memory stream and the spatial map layer, a dual-axis fusion is performed through a keyframe bridging mechanism to obtain a spatiotemporal memory system; When the spatiotemporal memory system is obtained by performing dual-axis fusion based on the temporal memory stream and the spatial map layer through a keyframe bridging mechanism, it is used to take a fused feature frame with a confidence level greater than a set confidence level as a visual keyframe and generate a keyframe trigger signal when a fused feature frame with a confidence level greater than a set confidence level is detected based on the temporal memory stream. Based on the keyframe trigger signal, the construction of the corresponding keyframe in the spatial map layer is triggered, and the spatial keyframe is obtained by marginalizing and compressing the statistical distribution features in the space. Based on the statistical distribution features, multiple spatial map layer update tasks are accumulated to a preset batch size and then batch matrix operations are performed uniformly to obtain the spatiotemporal memory system. The spatiotemporal memory system is used to maintain temporal continuity and scene geometry. Based on the spatiotemporal memory system, charging pile identification and positioning are performed to generate charging pile location positioning results, and based on the charging pile location positioning results, the quadruped robot is guided to move to the charging pile location for charging.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the quadruped robot recharge positioning method as described in any one of claims 1 to 10.

Images