An integrated communication and positioning method for intelligent machine swarms

Abstract

The application discloses a communication and positioning integrated method of intelligent machine groups, which is used for multi-intelligent robot cooperation. The method comprises the following steps: an intelligent robot constructs a query vector from current radar point cloud data perceived by a medium-term feature extraction and interaction module point, calculates spatial importance information, extracts medium-term features for communication and interaction; a base station clusters the intelligent robots through an optimal communication link construction module, distributes optimal communication links among the robots in the cluster, and supplements the communication links of the inter-cluster boundary; and the intelligent robot processes the medium-term feature time sequence sent by a cooperative positioning object and the feature map sequence collected by itself through a time-sequencing relative positioning module, and obtains the relative position relationship among the robots. The method considers the use of time-sequencing information, realizes real-time operation under objective communication condition limitation, realizes efficient and high-precision continuous cooperative positioning in the time-space dimension by fusing cross-device features, and improves the registration precision.

Description

Technical Field

[0001] This invention belongs to the field of multi-intelligent robot collaboration technology, specifically relating to an integrated communication and positioning method for a group of intelligent robots. Background Technology

[0002] In the field of intelligent robotics, to enable robots to perform safe, reliable, and highly accurate tasks, they often need to have high positioning accuracy within the corresponding environment. Current robot localization methods mainly rely on SLAM (Simultaneous Localization and Mapping) algorithms based on high-precision maps. This method pre-builds a high-precision map of the scene and performs map-level calculations and relocalization based on real-time perception data during operation.

[0003] To achieve the above goals, commonly used devices such as GPS and inertial measurement units (IMUs) have unavoidable problems at the device level, such as difficulty in meeting accuracy requirements or accumulating errors. Currently, the common approach is to rely on high-precision maps, that is, matching sensor data with a pre-built high-precision map to obtain absolute coordinates, such as latitude and longitude, in the map coordinate system. However, while pre-built map-based positioning schemes can help improve positioning speed and accuracy, this map-dependent approach means that the robot's collaborative algorithms cannot operate directly without a map. Due to discrepancies between the pre-built map and the distribution of perceived data, in highly interference-prone environments such as underground, mines, and mountainous areas, and under limitations such as lighting, rapid environmental changes, and insufficient positioning device accuracy, intelligent robots struggle to perform highly reliable operations under these conditions, thus limiting their application scenarios.

[0004] In scenarios involving multi-robot collaborative systems, multi-robot system algorithms typically focus more on the relative positional relationships between robots than their absolute positions in the global coordinate system. Only the relative positional relationship between neighboring robots and the master robot is needed to complete the coordinate system transformation and alignment, thereby enabling intelligent decision-making from the master robot's perspective. Therefore, collaborative localization algorithms that do not rely on high-precision maps have received widespread attention and research in the field of autonomous driving. Current related work focuses on registering the point clouds of both robots and deriving the rotation matrix between the coordinate systems of the two point clouds to achieve map-free relative position localization. However, such work lacks consideration for dynamic operating conditions. In the continuous localization scenario addressed by this invention, the shared point cloud data during collaborative perception between robots exhibits temporal correlation, as does the corresponding relative positional relationship. Existing work lacks consideration for temporal sequence, primarily focusing on registration of multi-angle point clouds within a single frame. During continuous relative localization, due to the strong temporal correlation between consecutive point cloud frames and relative positions, continuous temporal information strongly correlated with relative positions is often ignored. This often leads to a significant waste of computational and communication resources in order to ensure the reliability of continuous localization. Furthermore, current point cloud registration algorithms are typically based on the original point cloud data of both parties. Although this information contains the largest amount of data, the sheer volume of point cloud data inevitably leads to substantial communication resource consumption in real-world scenarios, making it difficult to guarantee the real-time performance of collaborative localization and limiting its usability in real-world situations.

[0005] Chinese invention patent CN117870652A, published on April 12, 2024, discloses a land-air online cooperative positioning method based on radar, inertial, and visual fusion. In the relative positioning process, both the UAV and the unmanned vehicle (UAV) achieve their own positioning through the fusion of airborne radar and inertial units. Simultaneously, the UAV observes the relative position of the UAV, and then the UAV and UAV point cloud fusion registration method is used to achieve relative pose estimation. However, this technical solution requires too many types of sensing data. Besides radar point cloud data, it requires multiple sensing data sources and land-air cooperative equipment to achieve highly stable relative positioning. Although sufficient and rich sensing data can bring higher accuracy to cooperative positioning, in real-world scenarios, a trade-off must often be made with the communication and computational load of the equipment and the environment. As the number of devices in the system increases, the excessive demand for sensing data means both increased system costs and a non-linear increase in peak communication and computational resource consumption, which is often unacceptable in practical scenarios.

[0006] Chinese invention patent application CN117156415A, published on December 1, 2023, discloses a communication resource management method for vehicle-to-everything (V2X) sensing and collaboration. This method estimates vehicle speed using location data obtained through wireless sensing and predicts the vehicle's position at the next moment based on a sensing estimation model. Based on the prediction results, a clustering method is used to manage vehicles corresponding to user terminals in clusters. The system throughput is calculated based on the clustering management results and the achievable communication rate. Maximizing system throughput is the optimization objective, thus constructing a resource allocation optimization problem. While this technical solution optimizes communication resources by maximizing system throughput, the clustering method relies solely on device location. However, in scenarios where there may be a large amount of invalid communication, although maximizing throughput fully utilizes resources, it neglects the attention and management of invalid communication. This makes it difficult for indicators to directly reflect the problem of wasted communication resources.

[0007] Chinese invention patent application CN117930219A, published on April 26, 2024, discloses a method for swarm UAV positioning based on millimeter-wave radar. Addressing navigation failure scenarios, it utilizes millimeter-wave radar technology to solve the swarm positioning problem. It estimates the expected relative position based on the swarm's state before navigation failure and then combines the radar and inertial measurement unit of a single device to achieve relative positioning. However, this technical solution relies on the navigation failure scenario and depends on the swarm's position and movement relationship before navigation failure. For scenarios where there is no high-precision map in the initial state or where relative positioning starts from the initial state, the initial expected value is difficult to obtain, making this technical solution unsuitable for such scenarios.

[0008] Chinese invention patent application CN116977963A, published on October 31, 2023, discloses a method and system for multimodal collaborative perception of autonomous driving based on the BEV perspective. To address the poor collaborative perception effect during the fusion process of multimodal data in autonomous driving, this technical solution extracts point cloud BEV features, image BEV features, and depth BEV features from acquired raw point cloud data and multi-view camera image data. Then, it fuses these BEV features to obtain multimodal BEV fusion features. A multi-task head processes these multimodal BEV fusion features to obtain 3D object detection results and BEV perspective semantic segmentation results. This technical solution uses point cloud and visual information fusion to extract mid-term features. The neural network for extracting mid-term features is trained only using the results of downstream semantic segmentation and object detection. The supervision signal for mid-term fusion aims to maximize the accuracy of downstream tasks, neglecting the individual needs of each collaborative participant. For example, the accuracy requirements of autonomous robots for near-distance objects are often higher than those for distant objects.

[0009] Existing technologies typically start from the single frame of raw perception data held by both parties and run the subsequent registration and positioning process directly. This will lead to a large waste of computing and communication resources in the context of temporal positioning. Therefore, it is necessary to build a set of intelligent machine group communication and positioning methods that meet the constraints of objective communication conditions and reduce the waste of computing and communication resources. Summary of the Invention

[0010] To address the problems existing in the prior art, this invention provides an integrated communication and positioning method for intelligent machine swarms. Based on the actual communication capabilities and positioning needs of individual intelligent machines, communication-friendly mid-term features are extracted. The selection and scheduling of collaborative positioning objects for individuals are completed at the base station. Individual intelligent machines estimate the relative positional relationship between machines based on the point cloud information of both parties, realizing real-time operation under objective communication constraints. By integrating cross-device features in the spatiotemporal dimension, efficient and high-precision continuous collaborative positioning is achieved, improving registration accuracy.

[0011] This invention discloses an integrated communication and positioning method for a swarm of intelligent machines. The application scenario includes multiple intelligent machines and multiple base stations. Each intelligent machine is equipped with a mid-term feature extraction and interaction module, as well as a temporal relative positioning module. Each base station is equipped with an optimal communication link construction module. The method of this invention includes the following steps:

[0012] Step 1: The intelligent robot perceives the radar using LiDAR. Its intermediate feature extraction and interaction module processes the perceived radar point cloud data, constructs a query vector, and broadcasts it. Intermediate features are extracted for interaction. Each intelligent robot's intermediate feature extraction and interaction module constructs a multi-resolution feature map from its perceived radar point cloud data and then constructs and broadcasts the query vector. When robot A interacts with robot B using the query vector, robot A's intermediate feature extraction and interaction module calculates the spatial importance of its multi-resolution feature map to robot B based on robot B's query vector. It then generates intermediate features with spatial importance from the multi-resolution feature map. Robot A concatenates the intermediate features and the spatial importance features as the interaction features and sends them back to robot B.

[0013] Step 2: The base station first clusters the intelligent robots using the optimal communication link construction module, then allocates optimal communication links within each cluster, and finally completes the communication links at the inter-cluster boundaries. Each intelligent robot initially attempts to connect to all base stations via broadcast, then retains only the communication link with the highest signal strength, grouping robots connected to the same base station into one cluster. The optimal communication link construction module calculates the distance score between robots within a cluster based on signal strength and spatial importance, and then assigns cooperative localization targets to robots within the cluster according to the principle of maximizing the sum of the bidirectional distance scores. For robots located at cluster boundaries, the distance score between them and robots located at adjacent cluster boundaries is calculated, and the communication links for robots located at cluster boundaries are optimized based on the distance scores. The base station feeds back the cooperative localization allocation results to the robots, and direct communication links are established between the two robots in cooperative localization.

[0014] Step 3: The intelligent robot receives the intermediate feature time sequence sent by the collaborative localization object. Based on its own acquired multi-resolution feature map sequence and the intermediate feature sequence transmitted by the collaborative object, the robot performs point cloud registration through the temporal relative localization module to obtain the relative positional relationship between the robots. The temporal relative localization module includes two feature extraction structures, a feature fusion unit, and a localization output network. Robot A acquires a multi-resolution feature map sequence of its own over a continuous sampling time and acquires an intermediate feature sequence from robot B for the corresponding continuous sampling time. The multi-resolution feature map sequence and the intermediate feature sequence are respectively input into a feature extraction structure to extract an aggregated spatiotemporal feature map. The feature fusion unit fuses the aggregated spatiotemporal feature maps output by the two feature extraction structures and inputs the fused features into the localization output network. The localization output network outputs the relative positional relationship between the two robots at the current moment.

[0015] In existing cooperative localization technologies similar to this invention, the utilization of temporal information is generally not considered. To ensure cooperative performance, multi-sensor perception or external observer information must be introduced, and the stability of the output results is difficult to guarantee. The method of this invention achieves efficient and rapid cooperative localization and communication in multi-robot group interconnection scenarios, and compared with existing technologies, it has the following advantages and positive effects:

[0016] (1) Considering the limitations of communication bandwidth, the method of this invention designs a mid-term feature extraction and interaction module. This module adaptively combines the robot's localization requirements to perform sparse mid-term feature extraction and interaction on the original point cloud information, thereby controlling the communication scale required for the collaborative localization algorithm. Existing relative localization methods typically require the integration of sensor data from multiple sensors and devices operating in various environments. However, in practical applications, due to limitations in communication bandwidth and device computing units, excessive sensor data can lead to the amount of data transmitted in real-time exceeding the system's communication and computing capacity, thus causing communication and computing bottlenecks. Therefore, this invention proposes a mid-term feature extraction network based on the actual task requirements of the collaborating parties, and combines this with the local personalized requirements of the localization task to complete the interaction and extraction of sparse feature maps between robots. This assists in achieving communication- and computing-friendly collaborative localization while ensuring task accuracy. Through efficient mid-term feature extraction and interaction strategies, this invention can flexibly adapt to various communication resource scenarios simply by adjusting the sparsity of the feature maps during the interaction process.

[0017] (2) The method of this invention, through an optimal communication link construction module based on signal strength and feature space importance, achieves local optimal cooperative link construction in local clusters and global cluster coverage of boundary conditions, minimizing the waste of communication and computing resources caused by distance factors. Existing communication link construction methods typically address resource scheduling optimization problems with the goal of optimizing system throughput and resource utilization. However, in the large-scale robot relative positioning scenario addressed by this invention, relative positioning is performed only through perception data with overlapping conditions, and most links do not meet this condition due to physical distance, resulting in a significant waste of communication and computing resources. Considering that the expected goal of the relative positioning method is to maximize the average effective information about the relative positional relationship between robots within the system and other robots in the communication topology, the communication link construction method in this scenario should, based on the unified completion of intra-cluster communication relay through direct links from communication base stations, design and execute connection allocation and partial clustering algorithms with strategies for minimizing the relative distance between robots within the cluster and optimizing the connectivity between nodes at the edge conditions between clusters. The method of this invention avoids the huge uplink bandwidth pressure caused by completely point-to-point communication within the system by constructing an optimal communication link module. At the same time, it avoids the problem of a large amount of communication and computing resources being wasted due to the strong dependence of the cooperative positioning algorithm on data overlap by using intra-cluster scheduling and inter-cluster completion methods.

[0018] (3) The method of this invention improves the accuracy of positioning and registration by incorporating the use of historical point cloud information and the temporal information of historical prediction results through a temporal relative positioning module based on continuous communication frames. Existing relative positioning methods do not consider the impact of temporal information on positioning, and mainly rely on single-frame perception data to complete point cloud registration and subsequent positioning processes. During robot operation, both actual perception and relative positioning data have strong temporal correlations, which will be reflected on both the input and output sides of the positioning algorithm. The scheme that only completes registration based on single-frame perception information completely ignores continuous frame information. From an information theory perspective, in order to ensure the consistent performance of the positioning algorithm, additional computation or communication is required to compensate for the loss of information. At the same time, with the assistance of the above-mentioned communication and intermediate feature strategies, the robot can efficiently receive continuous intermediate feature frame sequences from multiple parties, creating conditions for the utilization of temporal information. The method of this invention supplements and aggregates the information contained in the perception information in the spatiotemporal dimension, and performs time decay output aggregation based on confidence weighting based on historical prediction results to achieve highly reliable relative positioning.

[0019] (4) The method of the present invention realizes real-time operation under objective communication conditions, improves the utilization rate of information from a single sensing data source, and effectively ensures the stability of the output of the collaborative algorithm based on the confidence-time decay output strategy, thus expanding the application scenarios of collaborative positioning. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a scenario where a group of intelligent machines are interconnected.

[0021] Figure 2 This is a framework diagram of the integrated communication and positioning method for intelligent machine swarms of the present invention;

[0022] Figure 3 This is a diagram illustrating the implementation framework of the intermediate feature extraction and interaction module in the method of this invention.

[0023] Figure 4 This is a framework diagram of the implementation of cross-robot joint information aggregation by the temporal relative positioning module in the method of this invention. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0025] like Figure 1As shown, in a scenario of interconnected intelligent machine swarms, intelligent robots acquire and process radar point cloud perception images. These robots can communicate with neighboring robots within their communication range. Intelligent robots connected to the same base station are grouped into a cluster, and the robots within the cluster and the base station form a subnet. This invention provides an integrated communication and positioning method for intelligent machine swarms. First, considering the recent actual positioning needs of the intelligent machine swarm, intermediate features are extracted from the original perception information and the actual needs of neighboring robots to reduce communication requirements. Simultaneously, an optimal communication cooperation cluster is constructed based on optimal communication quality. Furthermore, the scheduling and allocation of connection and cooperation objects are completed at the base station. Finally, feature pairs between the master robot and neighboring robots are extracted based on continuous communication frame sequences. Point cloud registration is performed based on the local features of the two robots to obtain their relative positional relationship, thus achieving relative positioning.

[0026] like Figure 2 As shown, the integrated communication and positioning method for a swarm of intelligent machines in this embodiment of the invention mainly implements three functional modules based on a computer: a mid-term feature extraction and interaction module based on relative positioning requirements, an optimal communication link construction module based on signal strength and feature space importance, and a temporal relative positioning module based on continuous communication frames. Each intelligent machine is equipped with both the mid-term feature extraction and interaction module and the temporal relative positioning module, while each base station is equipped with the optimal communication link construction module. The implementation of this embodiment of the invention is described through the following three steps.

[0027] Step 1: Each intelligent machine extracts intermediate features from the radar sensing information through the intermediate feature extraction and interaction module.

[0028] The current intelligent machine is designated as the master robot. To control resource consumption during communication, this invention designs the master robot to complete relative localization based on feature maps, and uses feature map interaction during communication instead of raw point cloud data containing a lot of redundant information. Furthermore, to improve communication efficiency and ensure the accuracy of downstream localization tasks, feature extraction and interaction across robots and coordinate systems are achieved through a query-response feature map interaction method. Therefore, this invention's method designs an intermediate feature extraction and interaction module based on relative localization requirements. For example... Figure 2As shown, the intermediate feature extraction and interaction module of the main robot first constructs a multi-resolution feature map from the lidar point cloud data acquired locally, and then constructs a low-dimensional query vector based on real-time requirements for broadcasting. After interacting with the low-dimensional query vector with neighboring robots, the main robot constructs a voxelized importance feature map with spatial importance information based on the query vector, and generates a mask matrix based on the importance feature map in the spatial dimension. It further constructs a sparse feature map with spatial importance weights based on the multi-resolution feature map, thus realizing a communication-friendly intermediate feature extraction method that can be adjusted according to various communication loads of different sizes based on the scenario.

[0029] Based on the spatial importance of feature maps, a masking operation is further performed on the feature maps held by neighboring robots to prune them, thus retaining only the sparse feature maps corresponding to channels that are more important to the master robot, reducing the communication pressure in subsequent collaboration processes. Finally, the neighboring robots weight the sparse feature maps according to their importance, and send the sparse feature maps with implicit spatial importance back to the master robot, completing the mid-term feature extraction and interaction.

[0030] The implementation of the intermediate feature extraction and interaction module in this embodiment of the invention is as follows: Figure 3 As shown. To provide neural network feature maps that meet the actual communication needs of devices and are communication and computationally friendly, an adaptive feature extraction network needs to be constructed in conjunction with downstream positioning tasks. In this embodiment of the invention, the feature extraction network consists of two parts: a request encoder and a spatial local feature generator, and completes one round of data exchange through a query-response communication interaction.

[0031] During the collaborative localization process, the main robot first extracts the current point cloud into a multi-channel planar feature map from the BEV's perspective. Where H, W, and D represent the height, width, and depth of the feature map, respectively. This embodiment of the invention uses a PointPillars pre-trained network to voxelize the point cloud into a nine-dimensional feature tensor representation: x,y,z,r,x c ,y c ,z c ,x p ,y p Where x, t, a, r represent the three-dimensional spatial position and reflection intensity of a point cloud frame, x c ,y c ,z c x represents the centroid position of all points in the Pillar to which this point cloud belongs. p ,y p This represents the relative offset between the point and the centroid. A deep convolutional network is used to extract multi-resolution feature maps of the point cloud. Let F be the multi-resolution feature map extracted from the point cloud at time i. i .

[0032] This invention embodiment performs further feature interaction based on the feature map. The main robot will use the current feature map F i By requesting encoder G q Compressed into a communication-friendly N-dimensional query vector Q i as follows:

[0033]

[0034] Among them G q Through θ q Parameterized trainable feature encoder.

[0035] During the communication interaction, the query vector, as a feature reflecting the main robot's external needs, has already been used via G... q Compressing features to a low dimension ensures that they are broadcast to neighboring robots in the form of requests with low communication volume. Subsequently, the neighboring robots use these requests to further reduce the dimensionality of their own feature maps. For example... Figure 3 As shown, suppose a neighboring robot first uses the PointPillars pre-trained network to extract feature map F′ from its own perceived point cloud data. i Then for F′ i By requesting encoder G q Compressed into a low-dimensional query vector Q′ i And broadcast it.

[0036] During the communication interaction, the main robot sends the compressed low-dimensional query vector Q in real time. i Broadcast to all collaborating partners in the collaborative localization, namely neighboring robots, and simultaneously receive query vectors Q′ from neighboring robots. i The main robot combines the feature map F of its own point cloud. i Query vector Q′ with neighboring robots i Furthermore, the importance of voxelized features, which are spatially represented in the feature map, is determined in the spatial dimension. Specifically, a voxelization importance generator Φ is defined in the form of an object detection head using a single convolutional layer. v , Q′ i With F i After splicing, the spatial importance feature V is calculated. i as follows:

[0037]

[0038] Where Φ v Through θ v Parameterized trainable importance generator, V i Represents feature map F i The importance of each high-dimensional voxelized spatial feature to the neighboring robots. Generator Φv It is implemented with a plug-and-play target detection head structure, which can be replaced according to localization requirements such as prediction time or accuracy. V is selected using a top-k strategy. i The k values ​​with the largest weights are selected and transformed into a binary mask matrix M∈{0,1}. H×W In the mask matrix M, the values ​​corresponding to the k largest weights are set to 1, and the values ​​at the remaining positions are 0. A mask value of 1 represents F. i Features at corresponding locations are highly important to neighboring robots, requiring feature interaction. Connect M and F... i Element-wise dot product represents filtering the original feature map based on the importance of spatial features, retaining only the features corresponding to positions where the feature map mask matrix is ​​1, and discarding the features of voxels where the mask matrix is ​​0, which do not participate in subsequent feature interactions, resulting in a sparsed feature map F. i M Then F i M With V i Element-wise dot product, weighted by importance, yields the final intermediate feature C output by the intermediate feature extraction and interaction module. i .

[0039] Since the number of masks in M ​​directly affects the dimension of the sparse feature map used in the final communication, and further directly impacts the communication throughput, the sparse feature map participating in feature interaction can be changed by adjusting the value of k based on actual communication bandwidth conditions, thus flexibly adapting to various communication environments. Furthermore, note the neighboring robot V′ i The spatial importance information contained reflects C′ i The importance of each voxelization feature in C′ is therefore emphasized. i Spatial Importance Feature Map V′ i After being assembled, the features are transmitted back to the main robot as interactive features, completing one round of feature interaction process.

[0040] Step 2: The base station first clusters the intelligent robots using the optimal communication link construction module, allocates the optimal communication link within each cluster, and then completes the communication links at the boundaries between clusters.

[0041] In large-scale robot swarm cooperative localization scenarios, as the coverage area of ​​cooperative localization expands, point-to-point communication will be significantly limited by factors such as packet loss rate and communication bandwidth bottlenecks due to objective physical constraints, thus hindering its scalability. Furthermore, the mid-stage feature extraction and interaction modules distribute feature maps via broadcast, which would result in a significant waste of uplink bandwidth if point-to-point communication were used. Therefore, it is necessary to introduce a communication base station into the communication model, which will handle the reception, processing, and distribution of broadcast messages during communication. Considering that the ultimate goal of the cooperative system is to maximize the average amount of information in the effective relative localization results obtained by the robots within the system, the method of this invention aims to reduce invalid computation and communication. First, it allocates intra-cluster connections based on the base station, and then further completes the communication links that need to be established under boundary conditions through clustering.

[0042] The optimal communication link construction module first initializes clustering based on the spatial importance of the feature map and the signal strength between the robot and the base station. In the initial state, to build a communication link between the robot and the base station, the robot first broadcasts an attempt to connect to all base stations, and selects and retains the communication link with the highest signal strength based on a greedy strategy, while disconnecting from the remaining base stations. Simultaneously, based on the actual computational resource load of the robot group, the maximum number of connections *m* that the robot can actually support can be obtained, and the base station further completes the selection of the robot's collaborating partners and connection targets.

[0043] It is important to note that during the process of base station-assisted robot selection of communication targets, the lack of physical coordinates makes it difficult to directly determine the distance between robots. Furthermore, relative positioning algorithms based purely on perception information will obviously fail when the distance is too great due to the lack of overlap in perception information, and connections established under such conditions would waste significant communication and computing resources. To construct a highly reliable and resource-efficient communication link, a greedy approach should be taken to select the m nearest robots for each robot, which requires a certain degree of prediction of the relative distances between robots. During the initialization phase, the master robot broadcasts a query vector Q and calculates the voxelized importance V between robots, obtaining the voxelized importance of all other connectable robots. The master robot is labeled as 'a', its neighboring robots as 'b', and V... b The spatial importance features are calculated by neighboring robot b based on the query vector of master robot a; each V is calculated. b The elements and D b Using D b This indicates the importance of the feature map of the main robot a to the neighboring robot b, and D... b It is considered one of the factors reflecting the distance between neighboring robot b and the master robot. This is because the voxelization importance generator Φ v V is obtained based on training from downstream localization tasks.b This reflects the importance of the neighbor robot's entire feature map to the master robot.

[0044] For two known robots a and b within the same cluster, obtain their respective signal strengths with the base station, S. a S b Combining the free-space propagation model, the estimated distance between the two robots a and b can be obtained from the signal strength level. However, this value becomes misleading when robots are distributed at the same distance from the base station but in different directions. Therefore, this invention further calculates the distance fraction R from intelligent machine a to machine b using a weighted method. ab as follows:

[0045]

[0046] Where α is an adjustable weighting factor, ranging from (0,1), to control the weight of signal strength and bidirectional importance between devices in the distance scoring function, D ab The importance of the feature map representing robot b to robot a. Distance score R. ab The larger the value, the better the robot b coordinates with a for localization.

[0047] The distance scores between all devices within a cluster can be obtained using the above method. The base station then establishes a bidirectional connection for each robot based on the highest possible score. Within a base station's subnet, the optimal communication link construction module completes the connection scheduling process for robots within the cluster based on the distance scores between devices, assigning cooperative localization objects to robots and establishing communication interaction links. Assuming there are n robots within the base station subnet, and the maximum number of connections a single robot can support is determined to be m, the base station maximizes the bidirectional scores between devices within the cluster and R... sum The principle for assigning cooperative localization objects to robots within a cluster is as follows:

[0048] R sum =∑(R) ab +R ba ), a, b ∈ [1, n]

[0049] Among them, R ab R is the distance fraction from intelligent machine a to machine b. ba It is the distance fraction from intelligent machine b to machine a.

[0050] For a subnet of a base station, each robot needs to be set with m cooperative positioning objects. The base station feeds back the cooperative positioning allocation results to the robot, and a direct communication link is established between the two robots in cooperative positioning.

[0051] When a robot leaves the current subnet or an assigned communication link, the difference in signal strength between two robots is significant. When the distance is less than the set threshold, the base station needs to recalculate the distance score between the affected robots and reallocate the collaborative positioning objects and communication links.

[0052] After completing the connection allocation for intelligent robots within a cluster, two robots located at the boundaries of two communication clusters, despite being physically close, are assigned to different clusters and cannot establish communication with each other. To address this, the optimal communication link construction module of this invention uses cross-base station communication to perform inter-cluster cooperation and a certain degree of clustering, thus completing the inter-cluster boundary link. Considering the coverage of communication base stations, robots closer to the cluster center obviously do not need cross-cluster communication. Therefore, in this embodiment, robots whose signal strength S and all m distance scores R within the communication base station cluster are below a preset threshold are considered to be located at the cluster boundary. For robot a located at the base station boundary, robot b located at the boundary of the adjacent base station of that base station is selected, and the distance score between them is calculated again through cross-cluster communication. Based on the calculated distance score, it is determined whether to replace the communication link already established between robot a and robot b. For example, if the calculated distance score is higher than the distance score between robot a and a certain robot, a communication link can be established between robot a and robot b, replacing the existing communication link with a lower distance score between robot a and robot b. The distance score R from robot a to robot b is calculated across the cluster. ab The method is based on calculating the distance score between robots within the cluster, but the difference is that the signal strength is provided by the base station to which the robot is connected.

[0053] The optimal communication link construction module, constructed using the above method, enables low-latency long-range networking between intelligent machine groups and communication base stations. This ensures the real-time performance and reliability of collaborative positioning while reducing the consumption of communication and computing resources within the system.

[0054] Step 3: The intelligent machines interact with each other according to the established communication link. Each intelligent machine receives a continuous sequence of communication frames from its neighboring machine and determines the relative position between the two machines by performing point cloud registration through the time-series relative positioning module.

[0055] Through the above steps, the robot can efficiently communicate with its optimal neighboring robots, exchanging their real-time point cloud feature maps and their spatial importance features within the limits of computational and communication load. Subsequently, to achieve high-precision relative positioning, it is necessary to focus on the shared continuity between sparse feature map data and robot position information in terms of temporal relationship. Existing point cloud registration algorithms typically target single point cloud pairs at a specific time. While these methods can guarantee that the registration distance error reaches a statistically optimal solution, when directly applied to real-time positioning, their neglect of temporal continuity often leads to large inter-frame errors and a complete inability to guarantee the confidence level of the results when running on continuous point cloud sequences, making it difficult to guarantee their usability in continuous registration scenarios. To solve these problems, it is necessary to efficiently aggregate the joint information of cross-robot feature maps distributed in different feature spaces and possessing implicit temporal features, in order to align features from different sources and at different times into the same feature space.

[0056] The master robot receives interaction features from neighboring robots and extracts intermediate features calculated by the neighboring robots. It then uses the multi-resolution feature maps of the master robot at continuous time intervals and the intermediate features of the neighboring robots at continuous time intervals as input to the temporal relative localization module. The temporal relative localization module addresses the continuous feature maps of both the master robot and the neighboring robots. On one hand, it extracts intra-frame static attention from the feature map sequences through a self-attention mechanism, dividing the feature map sequence into a sequence of voxelized feature blocks that are continuous in the temporal dimension and independent in the spatial dimension, extracting spatiotemporal features from a local temporal to a global spatial perspective. On the other hand, it extracts inter-frame dynamic attention from the feature map based on the time-divided feature map, extracting spatiotemporal features from the global perspective of the feature map.

[0057] Then, a cross-attention mechanism is used to further fuse heterogeneous features from different robots. Finally, the features after aggregating multi-source temporal information are used for localization and adaptive confidence extraction, respectively, and a weighted output aggregation operation is performed based on historical localization and confidence results to complete the prediction process at one time step.

[0058] like Figure 4 As shown, the temporal relative localization module of this invention is based on an improved Transformer structure and includes a first feature extraction structure, a second feature extraction structure, a feature fusion unit, and a localization output network. Both the first and second feature extraction structures contain one one-dimensional (1-D) convolutional module and N encoding modules. Each encoding module is an improved version of the Transformer structure, with two multi-head self-attention modules at input, used to extract global and local spatiotemporal features respectively. Let F be the sequence of multi-resolution feature maps obtained by the main robot at time t, representing a continuous sampling time. i(i = t - T + 1, ..., t), obtain the interaction features C of a certain neighbor at the corresponding consecutive sampling times. i (i = t - T + 1, ..., t); Let the continuous sampling time length be T, the value of T can be adjusted. The continuous feature map F with time length T... i Input the first feature extraction structure to extract the aggregated spatiotemporal feature map of the main robot. Then, extract the continuous feature map C of time series length T. i The first feature extraction structure is used to extract the aggregated spatiotemporal feature maps of neighboring robots. The feature fusion unit contains N decoding modules, implemented using a Transformer architecture. These modules fuse the aggregated spatiotemporal feature maps of the main robot and neighboring robots, and then input the fused features into the localization output network. The localization output network outputs the relative positional relationship between the two robots at time t through a fully connected localization network, including the predicted position coordinates and rotation angle. The number N of the superimposed encoding or decoding modules is a positive integer and can be adjusted according to the required precision and time requirements.

[0059] Perform the following in the first feature extraction structure: (1) F i (i = t - T + 1, ..., t) are integrated into a spatiotemporal feature map S. f And it is divided into voxelized feature block sequences s in the spatial dimension. (f,j) ,j=1,2,…,H*W, perform 1-D convolution operation on the voxelized feature block sequence, aggregate the local spatial information of the block sequence in the depth dimension through 1-D convolution, and then input the aggregated local spatial feature sequence into a multi-head self-attention module, calculate the mutual relationship between feature blocks through time-level multi-head temporal attention, that is, obtain the local spatiotemporal features fused with intra-frame attention; (2) F i (i=t-T+1,…,t) input another multi-head self-attention module to extract inter-frame attention of the feature map and obtain global spatiotemporal features from the global perspective; (3) from F i Local and global spatiotemporal features extracted from sequences and F i The sequence undergoes residual connection processing, then is input into a feedforward network. The input and output of the feedforward network are then processed through residuals to obtain the aggregated spatiotemporal feature map H extracted from the information collected by the main robot. ego Similarly, in the second feature extraction structure, the continuous feature map C... i (i = t - T + 1, ..., t) are integrated into a spatiotemporal feature map S. c In the spatial dimension, it is voxelized into a sequence of feature blocks s (c,j) The features, j = 1, 2, ..., H*W, are aggregated using 1-D convolution operations to aggregate local spatial features in the depth dimension. Then, a multi-head self-attention module is input to extract local spatiotemporal features for intra-frame attention. C... i(i = t - T + 1, ..., t) is input into a multi-head self-attention module, which extracts the global spatiotemporal features of the fused inter-frame attention. Then C i The extracted global and local spatiotemporal features are fused into an aggregated spatiotemporal feature map H of the neighboring robots. neighbor .

[0060] In this embodiment of the invention, S f or S c Divided feature blocks The sequence is input into a 1-D convolutional layer with a kernel size of k*D for local feature aggregation, resulting in the following local features:

[0061]

[0062] Among them, o h,w,t K is the local feature extracted by the convolution kernel from the feature block s at position (h,w,t), where s(h,w,d,i) is the feature of depth d at position (h,w) at time i. d ∈R k*D The kernel used in the 1-D convolution is represented by k, where k represents the kernel size in the depth dimension. This embodiment of the invention transforms H×W×D×T dimensional data composed of feature block sequences into H×W×T dimensional data through 1-D convolution. Each aggregated feature block... The local temporal information carried in consecutive frames is aggregated into the feature map to enhance the model's ability to capture local temporal changes in features.

[0063] Then, the sequenced feature maps (o) are concatenated and input into a multi-head self-attention mechanism to further aggregate the interrelationships between local feature spaces in the sequence, obtaining local spatiotemporal features with local temporal-global spatial correlation. Global spatiotemporal features from the global spatial perspective are directly extracted from the feature map sequence. The local and global spatiotemporal features are then concatenated and fused to obtain an aggregated spatiotemporal feature map from local to global perspective. Let H be the aggregated spatiotemporal feature maps extracted from the main robot. ego The aggregated spatiotemporal feature map extracted from the neighboring robot is H. neighbor Because the neighboring robot returned C i The importance of voxelization was previously incorporated. i H neighbor This will naturally carry information about the importance of voxel space from the perspective of the main robot, facilitating subsequent fusion of feature maps from the main robot and neighboring robots. Spatiotemporal feature map H ego and H neighbor The data dimensions are all H×W×T.

[0064] The feature fusion processor obtains the temporal feature maps H of the main robot and neighboring robots. ego With Hneighbor Then, cross-robot feature fusion is performed through a cross-attention mechanism to obtain the fused feature A at time t. t as follows:

[0065]

[0066] Among them, CrossAttn Wx It is a cross-attention layer parameterized by Wx.

[0067] Due to H neighbor It inherently possesses the characteristic of voxel space importance, which can effectively align cross-robot features to the same spatial dimension. Furthermore, since the input feature map sequence already carries temporal features, H is fused through cross-attention. ego With H neighbor The output A is obtained t This allows for the simultaneous alignment of temporal features across robots and coordinate systems into the same feature space, both in time and space dimensions. The output A of the feature fusion unit... t The data dimension is H×W.

[0068] Finally, the localization output network obtains the predicted relative positional relationship between point cloud pairs input at the current time t through dual-head output. Predicted value of rotation angle The output matrix P is composed of t And the confidence level c of the current prediction result after softmax normalization. t To address the issues of significant inter-frame errors and unreliable result confidence, this invention implements a confidence-weighted time-decrease output aggregation scheme for a fixed time window T. Specifically, for the current time t, it combines the historical output matrix P within the time window T. i and confidence level c i Let i = t - T + 1, ..., t - 1. Before using the model's output, calculate O. t The final output is as follows:

[0069]

[0070] Where α i It combines the confidence level and the decay factor weight obtained from the time step, and is defined as:

[0071]

[0072] Where γ∈[0,1] is a predefined attenuation factor that reduces the influence of historical time steps on the output as the time step increases.

[0073] The temporal relative positioning module of this invention outputs matrix P for the current time t. t Confidence level c t And the final output result O t According to O t Determine the relative positions of the machines at time t and output matrix P. t and confidence level c t This will be used as historical data for predictions at subsequent times.

[0074] The temporal relative localization module of this invention, based on continuous sensing data and continuous communication frames, performs spatiotemporal feature aggregation on sequential single-source feature maps from both a global perspective and a local time-global space dimension to fully utilize the temporal correlations implicit in continuous frames. Then, sparse feature map sequences from neighboring robots are aggregated through a cross-attention mechanism, thereby completing the fusion of multi-source, cross-device temporal feature maps. To further improve the reliability of the prediction output, a time-decaying weighted output aggregation strategy is implemented using a sliding window based on the confidence level and time step of historical prediction outputs.

[0075] The method of this invention was verified and compared using perception data collected by standard robotic devices. During real-time operation, the average bandwidth usage for transmitting continuous raw point cloud perception information between devices was approximately 120 Mbps. However, under the condition that the top-k value is half the size of the feature map during mid-term feature interaction, the average bandwidth usage for feature sharing between devices decreased to 4.6 Mbps, which makes large-scale robot swarm collaboration possible. Furthermore, in actual verification, after training with the same time-step data (300,000 point cloud perception data pairs), the existing optimal point cloud registration scheme BUFFER had an average localization error of 5.73 cm and a single prediction time of 0.2 seconds on the validation set. The method of this invention had an average localization error of 2.67 cm and a single prediction time including perception data interaction of 0.37 seconds. This demonstrates that the present invention can effectively avoid the huge communication overhead caused by the transmission of raw perception data while efficiently utilizing perception data to achieve high-precision collaborative localization with an acceptable amount of additional computation.

[0076] In summary, the method of this invention can adjust the amount of interactive feature data according to objective communication conditions to meet the requirements of real-time operation. It can also integrate multi-source time-series data across devices, taking into account the temporal correlation of relative positioning data. This can effectively improve the accuracy and information utilization of relative positioning, and enhance the temporal continuity and reliability of the relative positioning algorithm.

[0077] Except for the technical features described in the specification, all other technologies are known to those skilled in the art. Descriptions of well-known components and technologies are omitted in this invention to avoid redundancy and unnecessary limitation. The embodiments described above do not represent all embodiments consistent with this application. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this invention are still within the protection scope of this invention.

Claims

1. A method for communication and positioning integration of a swarm of intelligent machines, applied in a scenario of interconnection of the swarm of intelligent machines, characterized in that, The method comprises the following steps: Step 1: the intelligent robot perceives through laser radar, extracts mid-term features and interacts with the module, constructs a query vector and broadcasts, extracts mid-term features for interaction; The mid-term feature extraction and interaction module of each intelligent robot constructs a multi-resolution feature map from the radar point cloud data perceived by itself, then constructs a query vector for broadcasting; after robot A and robot B interact with the query vector, the mid-term feature extraction and interaction module of robot A calculates the spatial importance of the multi-resolution feature map of itself to robot B based on the query vector of robot B, and then generates mid-term features with spatial importance; robot A splices the mid-term features and the spatial importance features as interaction features, and returns them to robot B; Step 2: the base station constructs a module for optimal communication link, first allocates optimal communication link to intelligent robots within the cluster, and then completes the communication link between intelligent robots located at the boundary of the cluster; Each intelligent robot initially attempts to connect all base stations in a broadcast manner, then only retains the communication link with the highest signal strength, and groups intelligent robots connected to the same base station into a cluster; the optimal communication link construction module calculates the distance score between the robots within the cluster based on signal strength and spatial importance, and then allocates cooperative positioning objects to the robots within the cluster according to the principle of maximizing the sum of bidirectional distance scores between the intelligent robots within the cluster; for robots located at the boundary of the cluster, the distance score between the robots located at the boundary of the adjacent cluster is calculated, and the communication link of the robot located at the boundary of the cluster is optimized according to the distance score; the base station feeds back the allocation of cooperative positioning to the robots, and establishes a direct communication link between the two robots for cooperative positioning; Step 3: the robots with established communication links communicate, and the robots use a time-sequenced relative positioning module to perform point cloud registration based on the multi-resolution feature map sequence collected by themselves and the mid-term feature sequence transmitted by the cooperative positioning object, to obtain the relative position relationship between the robots; The time-sequenced relative positioning module comprises two feature extraction structures, a feature fusioner and a positioning output network; assuming that robot A obtains a multi-resolution feature map sequence of continuous sampling time at the current time, and obtains a mid-term feature sequence corresponding to the continuous sampling time from robot B; the multi-resolution feature map sequence and the mid-term feature sequence are input into a feature extraction structure to extract aggregated spatio-temporal feature maps; the feature fusioner fuses the aggregated spatio-temporal feature maps output by the two feature extraction structures, and inputs the fused features into the positioning output network; the positioning output network outputs the relative position relationship between the two robots at the current time.

2. The method of claim 1, wherein, The step 1, the robot A's medium-term feature extraction and interaction module realizes the function including: first using the PointPillars pre-training network to represent the radar point cloud data as a nine-dimensional feature tensor, extracting the multi-resolution feature map F of the i-th point cloud i ; then using the request encoder G q , compressing the feature map F i into a communication-friendly query vector Q i and broadcasting it out; after receiving the query vector Q' sent by the neighbor robot B i , using the voxelization importance generator Φ v to calculate the importance of each voxelized spatial feature in the feature map F i , and setting the generated spatial importance feature map V i ; using the top-k strategy to select the k values with the largest weights in V i , and setting the corresponding positions to 1 and the other positions to 0 to obtain a binary mask matrix M; element-wise point multiplying M and F i , retaining only the features of F i corresponding to the positions with the value 1 in the mask matrix, discarding the features of F i corresponding to the positions with the value 0 in the mask matrix, and obtaining the feature map F M i ; element-wise point multiplying F M and V i , obtaining a spatial importance weighted sparse matrix graph, that is, a medium-term feature C i ; the robot A concatenates the medium-term feature C i and the spatial importance feature map V i as the feature for interacting with the robot B.

3. The method according to claim 1 or 2, characterized in that, In step 1, the spatial importance of the multi-resolution feature map of robot A to robot B is calculated as follows: The query vector Q' of robot B is i The multi-resolution feature map F of robot A is i After splicing, the voxelized importance generator is used to calculate F i The spatial importance V of robot B is i ; wherein the voxelized importance generator Φ v It is implemented in the form of a target detection head using a single-layer convolutional layer.

4. The method of claim 2, wherein, In step 1, the value of k is adjusted according to the actual communication bandwidth to change the sparse feature map participating in feature interaction.

5. The method of claim 1, wherein, In step 2, the optimal communication link construction module calculates the distance score between the robots within the cluster as follows: Let the signal strength of two robots a, b in the cluster connected with the base station be S a , S b , the spatial importance of the multi-resolution feature map of robot b to robot a be V ab , the element of V ab and D ab be calculated, then the distance score R ab from robot a to b is as follows: Wherein, α is a weight factor, and the value range is (0, 1).

6. The method according to claim 1 or 5, characterized in that, In step 2, the optimal communication link construction module calculates the distance score between all robots in the cluster, and according to the maximum number of connections m supported by a single robot, the bidirectional score between devices in the cluster is maximized R sum The robots in the cluster are assigned cooperative positioning objects, and each robot is assigned m cooperative positioning objects; wherein R sum =∑(R ab +R ba ),a,b∈[1,n];n is the total number of robots in the cluster, R ab is the distance score from robot a to b, and R ba is the distance score from robot b to a; When the robot leaves the current cluster or the signal strength difference between the two robots in the allocated communication link and the base station connection is less than the set threshold, the optimal communication link construction module recalculates the distance score between the affected robot devices in the cluster and reallocates the cooperative positioning objects.

7. The method according to claim 1 or 5, characterized in that, In step 2, if the signal strength S of the robot connected with the base station and the distance score with the allocated cooperative object in the cluster are both lower than the preset threshold, the robot is considered to be located at the cluster boundary; for the two robots located at the cluster boundary of the adjacent base stations, the distance score between the two robots is calculated based on the signal strength and the spatial importance, and the cooperative positioning object of the two robots is updated according to the distance score.

8. The method of claim 1, wherein, In step 3, both feature extraction structures comprise one one-dimensional convolution module and N encoding modules, and the encoding module is obtained by changing the multi-head self-attention module of the encoding module of the Transformer structure into two multi-head self-attention modules; the robot A obtains the multi-resolution feature map sequence F i (i=t-T+1,…,t) of the robot A and the intermediate feature sequence C i (i=t-T+1,…,t) of the robot B; on one hand, the multi-resolution feature map sequence of the robot A or the intermediate feature sequence of the robot B is integrated and cut into a voxelized feature block sequence in the spatial dimension, the one-dimensional convolution module convolves the voxelized feature block sequence, aggregates the local spatial feature sequence in the depth dimension, inputs a multi-head self-attention module, calculates the mutual relationship between the voxelized feature blocks, and obtains the local space-time features fused with the intra-frame attention; on the other hand, the multi-resolution feature map sequence of the robot A or the intermediate feature sequence of the robot B is input into another multi-head self-attention module to extract the global space-time features fused with the inter-frame attention; The local spatio-temporal features and the global spatio-temporal features are aggregated in the encoding module, and the aggregated spatio-temporal feature map H is obtained from the multi-resolution feature map sequence of the robot A ego The aggregated spatio-temporal feature map H is obtained from the mid-term feature sequence of the robot B neighbor ; wherein T is a continuous sampling time length, and N is a positive integer.

9. The method according to claim 1 or 8, characterized in that, In step 3, the feature fusioner is implemented using N decoding modules of the Transformer structure, N being a positive integer; the feature fusioner obtains the aggregated spatiotemporal feature map H extracted from the multi-resolution feature map sequence of the robot A at the current t time ego And the aggregated spatiotemporal feature map H extracted from the current intermediate feature sequence of the robot B neighbor Cross-attention mechanism is used for cross-robot feature fusion to obtain the fusion feature A at the current t time t As follows: A t = CrossAttn Wx (H ego , H neighbor , H neighbor ) where CrossAttn Wx is a cross-attention layer parameterized by Wx.

10. The method of claim 1 or 8, wherein, In step 3, the positioning output network predicts the relative position relationship between the two robots A and B at the current time t according to the fusion features at the current time t and the rotation angle and the confidence c of the prediction result t ; the current relative position relationship and the rotation angle form an output matrix P t , combined with the historical output matrix P i and the confidence c within the time window T i , i = t-T+1,…, t-1; calculate the final output matrix O at the current time t t : where α i is the time decay factor based on the confidence weighting at time i, calculated as: Wherein, γ is a predefined attenuation factor, γ ∈ [0, 1].

Images