Multi-sensing-node collaborative mapping method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a multi-sensing-node collaborative mapping method and system, electronic equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: constructing a local map taking a local node as a reference according to multi-frame point cloud data locally acquired by the node, discretizing the local map to obtain a plurality of subspaces, and performing geometric feature coding and intensity feature coding on each subspace to correspondingly obtain a geometric structure code and an intensity structure code; and feature description is carried out on the local map according to the strength structure codes and the geometric structure codes, a bimodal feature descriptor of the local map is obtained, the bimodal feature descriptor and node poses of the local map are interacted by multiple nodes, and a global map is constructed based on interaction information. According to the invention, the efficiency and accuracy of multi-node collaborative mapping can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, system, electronic device and storage medium for collaborative mapping of multiple sensing nodes. Background Art

[0002] In the field of unmanned driving, its operating scenarios often involve complex terrain, narrow passages, and dynamic obstacles, and are highly dependent on real-time high-precision maps. For example, due to limited perception coverage and data update delays, the traditional single-vehicle mapping model is unable to meet the stringent requirements of multi-vehicle formations for global path collaborative planning and obstacle avoidance. Multi-vehicle collaborative mapping uses distributed perception and data fusion of multiple vehicles to build a unified high-precision map of the operating area in a short period of time, significantly improving the safety and efficiency of transportation. Multiple vehicles synchronously scan the road geometry and obstacle distribution, share environmental feature data in real time, and can quickly generate centimeter-level precision maps, providing an accurate basis for complex operations such as multi-vehicle coordinated steering. Multi-vehicle collaborative mapping technology can reduce the risk of transportation interruptions caused by delayed map updates in freight, improve the speed of global mapping, and provide key technical support for large-scale development in energy infrastructure, high-end manufacturing and other fields.

[0003] Multi-vehicle collaborative mapping technology can be categorized by architecture: centralized and distributed. Centralized systems rely on a central server for global data fusion, but suffer from high communication bandwidth requirements and bottlenecks at central nodes. Distributed architectures, on the other hand, achieve data collaboration through local communication between vehicles, offering advantages such as network load balancing and strong system fault tolerance, and are more aligned with the development trend of edge computing in the Internet of Vehicles. However, current mapping methods struggle to balance feature matching accuracy with communication efficiency. Overcompression of feature dimensions can easily lead to mismatches, while transmitting complete point cloud data increases network load. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a multi-sensing node collaborative mapping method, system, electronic device and storage medium, aiming to improve the efficiency and accuracy of multi-node collaborative mapping.

[0005] To achieve the above objectives, an embodiment of the present application provides a method for collaborative mapping of multiple sensing nodes, including the following steps:

[0006] Construct a local map based on the local node based on the multi-frame point cloud data collected locally by the node;

[0007] Discretizing the local map to obtain multiple subspaces;

[0008] Performing geometric feature coding and intensity feature coding on each of the subspaces to obtain geometric structure coding and intensity structure coding respectively;

[0009] Performing feature description on the local map according to the intensity structure coding and the geometric structure coding to obtain a bimodal feature descriptor of the local map;

[0010] A global map is constructed based on the bimodal feature descriptors of the local maps of multiple nodes and the node poses.

[0011] In some embodiments, constructing a local map based on the local node based on the multi-frame point cloud data collected locally by the node includes the following steps:

[0012] Dividing each frame of point cloud data into plane point cloud and edge point cloud;

[0013] Determine a node motion transformation relationship between two frames of point cloud data according to the plane point cloud and the edge point cloud;

[0014] A local map with the local node as the coordinate origin is constructed according to the point cloud data of each frame and the corresponding node motion transformation relationship.

[0015] In some embodiments, discretizing the local map to obtain multiple subspaces includes the following steps:

[0016] Taking the coordinate origin of the local map as the center, dividing the local map into a plurality of annular areas and a plurality of sectors in radial and circumferential directions respectively;

[0017] A plurality of subspaces are determined according to the plurality of the ring areas and the plurality of the sectors.

[0018] In some embodiments, performing geometric feature coding and intensity feature coding on each of the subspaces to obtain geometric structure coding and intensity structure coding respectively includes the following steps:

[0019] Binary-encode the perception state of each point cloud in the subspace to obtain a geometric encoding value of the subspace;

[0020] forming a geometric structure code according to the geometric code values ​​of the plurality of subspaces;

[0021] Extracting the maximum reflection intensity of multiple point clouds in the subspace to obtain an intensity encoding value of the subspace;

[0022] An intensity structure code is formed according to the intensity code values ​​of the plurality of subspaces.

[0023] In some embodiments, the bimodal feature descriptor includes an intensity distribution descriptor and a geometric distribution descriptor, and the feature description of the local map according to the intensity structure coding and the geometric structure coding to obtain the bimodal feature descriptor of the local map includes the following steps:

[0024] Using a discrete indicator function to perform row-mode-based intensity distribution calculation on the intensity structure code to obtain an intensity distribution descriptor;

[0025] A Fourier hash algorithm is used to calculate the annular grayscale distribution of the geometric structure code to obtain a geometric distribution descriptor.

[0026] In some embodiments, constructing a global map based on bimodal feature descriptors of local maps of multiple nodes and node poses comprises the following steps:

[0027] Perform loop closure detection based on the bimodal feature descriptor of the local map of each node and the node pose to obtain loop closure information;

[0028] Optimizing the global coordinate system of the node pose according to the loop information to obtain a pose transformation relationship between each node coordinate system and the global coordinate system;

[0029] A global map is constructed based on the posture transformation relationship and the local map of each node.

[0030] In some embodiments, performing global coordinate system optimization on the node pose according to the loopback information to obtain a pose transformation relationship between each node coordinate system and the global coordinate system includes the following steps:

[0031] Determine a first transformation matrix of all cross-node closed-loop constraints according to the loop closure information;

[0032] Constructing a nonlinear least squares objective function according to the first transformation matrix and a second transformation matrix for representing the posture transformation relationship between the node coordinate system and the global coordinate system;

[0033] The second transformation matrix in the nonlinear least squares objective function is optimized and iterated until a preset condition is met, thereby obtaining a posture transformation relationship between the node coordinate system and the global coordinate system.

[0034] To achieve the above objectives, another aspect of the present application provides a multi-sensing node collaborative mapping system, including:

[0035] The first module is used to construct a local map based on the local node based on the multi-frame point cloud data collected locally by the node;

[0036] The second module is used to discretize the local map to obtain multiple subspaces;

[0037] The third module is used to perform geometric feature coding and intensity feature coding on each of the subspaces, and obtain geometric structure coding and intensity structure coding respectively;

[0038] a fourth module, configured to perform feature description on the local map according to the intensity structure coding and the geometric structure coding to obtain a bimodal feature descriptor of the local map;

[0039] The fifth module is used to construct a global map based on the bimodal feature descriptors of the local maps of multiple nodes and the node poses.

[0040] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the method described in the above embodiment is implemented.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the above embodiment.

[0042] The multi-sensory node collaborative mapping method, system, electronic device and storage medium proposed in this application, which constructs a local map based on the local node based on the multi-frame point cloud data collected locally by the node, and then discretizes the local map to obtain multiple subspaces, and then performs geometric feature encoding and intensity feature encoding on each subspace respectively, and obtains geometric structure encoding and intensity structure encoding accordingly. The local map is characterized according to the intensity structure encoding and the geometric structure encoding to obtain a bimodal feature descriptor of the local map. Multiple nodes interact with the bimodal feature descriptors and node poses of the local maps, and a global map is constructed based on the interactive information. This application constructs a lightweight bimodal feature descriptor with strong discrimination by jointly optimizing the spatial distribution characteristics of the point cloud and the statistical characteristics of the reflection intensity. Compared with the feature interaction methods such as point cloud curvature and normal vector in related technologies, this application adopts a lightweight bimodal feature descriptor for interaction and map construction, which can reduce communication overhead. At the same time, it compensates for the expression defects of geometric features in dynamic objects and weak texture areas through intensity information, improves matching accuracy, and thus improves the efficiency and accuracy of multi-node collaborative mapping. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flowchart of the multi-sensing node collaborative mapping method provided in an embodiment of the present application;

[0044] Figure 2 This is a schematic diagram of discretization and subspace encoding of local map point cloud data provided by an embodiment of the present application;

[0045] Figure 3Schematic diagram of the cross-vehicle closed-loop constraint detection process provided by an embodiment of the present application;

[0046] Figure 4 Schematic diagram of the global coordinate optimization and local coordinate optimization process provided by the embodiment of the present application;

[0047] Figure 5 This is an overall flow chart of the multi-sensing node collaborative mapping method provided in an embodiment of the present application;

[0048] Figure 6 This is a schematic diagram of the overall process of the multi-sensing node collaborative mapping method provided in an embodiment of the present application;

[0049] Figure 7 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0051] It should be noted that although the system is divided into functional modules and the flowcharts illustrate a logical sequence, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowcharts. The terms "first," "second," and so on in the specification, claims, and drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0053] First, let’s analyze some of the terms used in this application:

[0054] The core goal of SLAM (Simultaneous Localization and Mapping) algorithms is to enable mobile devices (such as robots, drones, and autonomous vehicles) to achieve real-time positioning, environmental mapping, and autonomous navigation in unknown environments. The implementation steps of SLAM algorithms include sensor data acquisition, feature extraction and matching, pose transformation estimation, and map optimization.

[0055] The incremental PCM (Pairwise Consistency Maximization) algorithm is an improved version of the traditional pairwise consistency maximization (PCM). The core idea is to reduce computational complexity by incrementally processing candidate pairs rather than performing a one-time global optimization, while maintaining and updating the global consistency set in real time. In dynamic or large-scale scenarios, it can efficiently screen out mutually compatible candidate pairs (such as closed-loop matching in SLAM and association in multi-target tracking). Traditional PCM requires traversing all combinations of candidate pairs, while incremental PCM only verifies new candidate pairs (such as newly detected closed-loop hypotheses) against the current consistency set when they arrive, rather than recalculating all historical pairs. If the new candidate is compatible with all members of the existing set (satisfying geometric / semantic constraints), it is added to the set. If there is a conflict, the priority (such as the matching score) is used to decide whether to remove the conflicting member or replace it with the new candidate.

[0056] Current multi-vehicle collaborative mapping technology faces multiple technical bottlenecks in complex scenarios. First, traditional point cloud data transmission requires extremely high communication bandwidth (about 1MB per frame), and it is difficult to support real-time interaction among multiple vehicles in low-bandwidth scenarios, resulting in a sharp drop in mapping efficiency and even communication interruption. Secondly, related methods rely on a single geometric feature descriptor (such as point cloud curvature or normal vector), and the feature discrimination is significantly reduced in weak texture areas (such as flat roads) or under the interference of dynamic obstacles. The cross-vehicle closed-loop detection mismatch rate is as high as 20%, causing map misalignment and path planning failure. In addition, the cumulative error of single-vehicle SLAM in multi-vehicle collaboration will propagate through the closed-loop constraint chain, causing the global consistency of cross-vehicle map splicing to deteriorate, seriously threatening the collaborative operation safety of heavy-duty transport vehicle groups. These technical defects have seriously restricted the large-scale application of multi-vehicle collaborative mapping technology in complex industrial scenarios.

[0057] Based on this, the embodiments of the present application provide a multi-sensing node collaborative mapping method, system, electronic device and storage medium, aiming to improve the efficiency and accuracy of multi-node collaborative mapping.

[0058] The multi-sensing node collaborative mapping method, system, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the multi-sensing node collaborative mapping method in the embodiments of the present application is described.

[0059] The multi-sensory node collaborative mapping method provided in the embodiment of the present application relates to the field of computer technology. The multi-sensory node collaborative mapping method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a sensing node in a distributed network, such as a drone, robot, or vehicle in a formation; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the multi-sensory node collaborative mapping method, but is not limited to the above forms.

[0060] Figure 1 This is an optional flowchart of the multi-sensing node collaborative mapping method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0061] Step S101, constructing a local map based on the local node based on the multi-frame point cloud data collected locally by the node;

[0062] Step S102, discretizing the local map to obtain multiple subspaces;

[0063] Step S103, performing geometric feature coding and intensity feature coding on each subspace respectively, and obtaining geometric structure coding and intensity structure coding accordingly;

[0064] Step S104, performing feature description on the local map according to the intensity structure coding and the geometric structure coding to obtain a bimodal feature descriptor of the local map;

[0065] Step S105 : constructing a global map based on the bimodal feature descriptors of the local maps of multiple nodes and the node poses.

[0066] In step S101 of some embodiments, for the scenario of vehicle formation, the node may refer to a vehicle in the formation. The vehicle is equipped with a perception sensor, and point cloud data can be collected in different postures to obtain multi-frame point cloud data. Point cloud data is a manifestation of three-dimensional spatial information, which is composed of a large number of discrete three-dimensional coordinate points. Each point contains the following three-dimensional coordinates, color, reflection intensity, normal vector and timestamp information. For example, the laser radar installed on the roof perceives and collects data at a frame rate of 10HZ and records it in the memory in the form of a laser radar point cloud. The combined navigation installed on both sides of the front of the vehicle perceives real-time vehicle positioning data at a frequency of 100HZ and records it in the memory. After the vehicle-mounted terminal collects multi-frame point cloud data locally, it can construct a local map based on the local node according to the point cloud data and the collected posture.

[0067] According to some embodiments of the present application, step S101 may include but is not limited to the following steps:

[0068] Step S201, dividing each frame of point cloud data into plane point cloud and edge point cloud;

[0069] Step S202, determining a node motion transformation relationship between two frames of point cloud data based on the plane point cloud and the edge point cloud;

[0070] Step S203 : constructing a local map with the local node as the coordinate origin according to the point cloud data of each frame and the corresponding node motion transformation relationship.

[0071] Specifically, the collected point cloud data is used as input, and the point cloud is divided into a plane point cloud containing plane features and an edge point cloud containing contour features according to the geometric distribution characteristics, and the node motion transformation relationship of the two frames of point clouds is estimated. The node motion transformation relationship is the angle change and displacement change when the vehicle collects the two frames of point clouds before and after. According to the node motion change relationship between each frame of point cloud data and the two frames of point clouds, the vehicle locally executes the SLAM algorithm to construct a local map with the vehicle itself as the coordinate origin. The local map is the point cloud data obtained by combining multiple frames of point cloud data according to the node motion change relationship. The SLAM algorithm can be an algorithm such as Lego-LOAM, L IO-SAM, etc., and the embodiments of the present application are not specifically limited. Each sensing node in the distribution network (such as each vehicle in the transport formation) locally executes the SLAM algorithm, and the corresponding local map under the node's own coordinate system is obtained.

[0072] In step S102 of some embodiments, discretizing the local map involves segmenting the local map according to a specific grid size, with each resulting grid being a subspace. By discretizing the local map and then performing feature encoding based on the point cloud data within the subspace, encoding efficiency can be improved while enhancing the representation of map content.

[0073] According to some embodiments of the present application, step S102 may include but is not limited to the following steps:

[0074] Step S301: Divide the local map into a plurality of annular areas and a plurality of sectors in radial and circumferential directions, respectively, with the coordinate origin of the local map as the center;

[0075] Step S302: Determine multiple subspaces according to multiple ring areas and multiple sectors.

[0076] In this embodiment, please refer to Figure 2 , divide each frame point cloud in the local map into N evenly according to the radial and circumferential directions r For ring area and N s sectors, N r ×N s The small areas are highly discretized into several subspaces, for example, divided into 8 subspaces.

[0077] Specifically, each frame point cloud is divided into N points evenly in radial and circumferential directions with the laser radar as the coordinate origin. r ×N s subspaces, each subspace is defined as:

[0078]

[0079] Among them, p k represents the point cloud set of the kth frame, L max Indicates the maximum radial length, Represents the radial distance of the kth point in the point cloud; Represents the circumferential angle of the kth point in the point cloud.

[0080] In step S103 of some embodiments, geometric feature coding and intensity feature coding are performed on each subspace respectively, and geometric structure coding and intensity structure coding are obtained accordingly. Geometric feature coding refers to coding the geometric parameters of the subspace according to certain coding rules, and intensity feature coding refers to coding the intensity parameters of the subspace according to certain coding rules. Please continue to refer to Figure 2 , binary encoding is performed according to whether there is a point cloud, and converted into grayscale values ​​as the geometric encoding values ​​of the subspace. The geometric structure encoding is a matrix formed by the geometric encoding values ​​of each subspace as elements, which is N r ×N s Similarly, the maximum reflection intensity after calibration of each subspace is taken as the intensity encoding value of the subspace, and the intensity structure is encoded as a matrix formed by the intensity encoding values ​​of each subspace as elements, which is N r ×N s dimensional matrix.

[0081] According to some embodiments of the present application, step S103 may include but is not limited to the following steps:

[0082] Step S401, binary encoding is performed on the perception state of each point cloud in the subspace to obtain a geometric code value of the subspace;

[0083] Step S402, forming a geometric structure code according to the geometric code values ​​of the multiple subspaces;

[0084] Step S403, extracting the maximum reflection intensity of multiple point clouds in the subspace to obtain the intensity coding value of the subspace;

[0085] Step S404: forming an intensity structure code according to the intensity code values ​​of the multiple subspaces.

[0086] Specifically, in geometric structure coding, the subspace point cloud is highly discretized and then vertically encoded. The binary encoding method is as follows:

[0087]

[0088] Where, H h is the maximum effective altitude, H l Is the lowest valid height, bit k Indicates whether there is a point cloud in the subspace that falls within the k-th height interval.

[0089] The binary coding method can be used to obtain the binary coding identifier of the subspace, and then the binary coding representation is converted into a decimal representation to obtain the geometric coding value of the subspace, which can be considered as the grayscale value of the subspace. The geometric coding value of each subspace is used as an element to form N r ×N s dimensional matrix to obtain the geometric structure encoding.

[0090] In intensity structure encoding, the lidar reflection intensity is calibrated before encoding. The calibration function used is as follows:

[0091] η r =η cal ·f(R)·g(S)+∈;

[0092] Among them, η r is the original reflectivity, η cal is the reflectivity after calibration, f(R) is the distance attenuation function, g(S) is the device gain parameter, and ∈ is the measurement noise.

[0093] The maximum reflection intensity is extracted from the calibrated reflection intensity of multiple point clouds in the subspace, and the maximum reflection intensity is used as the intensity encoding value of the subspace. Then, the intensity encoding values ​​of each subspace are used as elements to form Nr ×N s dimensional matrix to obtain the intensity structure encoding.

[0094] In step S104 of some embodiments, a feature description is performed on the local map based on intensity structure coding and geometric structure coding to obtain a bimodal feature descriptor for the local map. The bimodal feature descriptor is a lightweight feature descriptor based on geometric-intensity bimodal structure coding, which includes an intensity distribution descriptor and a geometric distribution descriptor.

[0095] According to some embodiments of the present application, S104 may include but is not limited to the following steps:

[0096] Step S501, using a discrete indicator function to perform row-mode-based intensity distribution calculation on the intensity structure code to obtain an intensity distribution descriptor;

[0097] Step S502: Fourier hashing algorithm is used to calculate the ring-shaped grayscale distribution of the geometric structure code to obtain a geometric distribution descriptor.

[0098] Specifically, when extracting feature descriptors, the row modal intensity distribution of the intensity structure code is calculated, and the circular Fourier perceptual hash (CFPH) calculation is performed on the geometric structure code. Furthermore, the circular BRIEF descriptor of the geometric structure code can also be extracted, and the circular BRIEF descriptor can be used for closed-loop detection later.

[0099] The process of extracting the intensity coding row modal intensity distribution from the intensity structure coding is expressed as follows:

[0100]

[0101] Among them, v mode It is the intensity value that appears most frequently in the intensity structure encoding of a frame of the local map. δ(a, b) is the Kronecker delta (discrete indicator) function, and its value is 1 when a=b.

[0102] The feature extraction process for geometric structure coding is divided into circular Fourier perceptual hashing and circular BRIEF descriptor extraction. The circular Fourier hashing calculation process is as follows:

[0103]

[0104] Among them, I g (x, y) is the grayscale value of the geometric structure encoding, F(u, v) is the complex spectrum after two-dimensional discrete Fourier transform, A(u, v) is the spectrum amplitude, Re(·) represents the real part, Im(·) represents the imaginary part, which is used to generate a 64-bit hash code, and μ is the binarization threshold.

[0105] In step S105 of some embodiments, a global map is constructed based on the bimodal feature descriptors and node poses of the local maps of multiple nodes. Specifically, by communicating and exchanging feature descriptors and node poses between vehicles, feature matching and pose alignment are performed based on the feature descriptors to align the acquisition pose of each frame of point cloud to the global coordinate system, thereby realizing global map construction and local coordinate optimization. The embodiment of the present application proposes a geometry-intensity bimodal encoding mechanism, which constructs a lightweight descriptor with strong discrimination by jointly optimizing the spatial distribution characteristics of the point cloud and the statistical characteristics of the reflection intensity. This design allows a single vehicle (i.e., a sensing node) to complete cross-vehicle data association by only broadcasting the compressed bimodal feature package, which greatly reduces the communication overhead compared to traditional methods. At the same time, it compensates for the expression defects of geometric features in dynamic objects and weak texture areas through intensity information, achieves centimeter-level matching accuracy, and constructs a globally consistent environment map under limited bandwidth, ultimately achieving a coordinated improvement in mapping efficiency and system robustness, thereby improving path planning reliability and transportation safety in complex scenarios.

[0106] According to some embodiments of the present application, step S105 may include but is not limited to the following steps:

[0107] Step S601: performing loop closure detection based on the bimodal feature descriptor of the local map of each node and the node pose to obtain loop closure information;

[0108] Step S602: Optimize the global coordinate system of the node pose according to the loop information to obtain the pose transformation relationship between each node coordinate system and the global coordinate system;

[0109] Step S603: construct a global map based on the posture transformation relationship and the local map of each node.

[0110] In step S601 of some embodiments, feature loop closure detection is performed based on the bimodal feature descriptors of each frame of the local map across nodes and the corresponding node poses, and false loop closures are removed through incremental PCM. For example, incremental PCM is performed when more than five cross-vehicle loop closure constraints are detected. The cross-vehicle loop closure constraints must meet the following conditions:

[0111]

[0112] in, is the closed-loop constraint of vehicle β at time j and vehicle α at time i, and ∈ is the constraint tolerance threshold, which is used to eliminate false closed loops.

[0113] If the cross-vehicle closed-loop constraint does not meet the above conditions, it is considered a false closed loop and is eliminated. The above closed-loop detection can obtain loop information, which is used to characterize the feature matching of two frames of point clouds between nodes, that is, the existence of a closed-loop constraint.

[0114] In step S602 of some embodiments, step S602 may include but is not limited to the following steps:

[0115] Step S701, determining a first transformation matrix of all cross-node closed-loop constraints according to loop closure information;

[0116] Step S702: constructing a nonlinear least squares objective function based on the first transformation matrix and the second transformation matrix used to represent the posture transformation relationship between the node coordinate system and the global coordinate system;

[0117] Step S703: Optimize the second transformation matrix in the iterative nonlinear least squares objective function until a preset condition is met, thereby obtaining a posture transformation relationship between the node coordinate system and the global coordinate system.

[0118] For details, please refer to Figure 4 , when the i-th cross-vehicle closed-loop constraint is detected between vehicles β and α, the global optimization problem is constructed by the following relationship:

[0119]

[0120] in, is the j-th pose in the local coordinate system of vehicle β For the relative pose measurement obtained by point cloud matching, the transformation matrix T of all cross-vehicle closed-loop constraints is βα (i.e. the first transformation matrix) and the global coordinate system calculated value (i.e., the second transformation matrix) constructs a nonlinear least squares objective function as follows:

[0121]

[0122] Among them, the error term Ω βα The covariance matrix is ​​the Levenberg-Marquardt algorithm, which is used to iteratively optimize the transformation matrix T from the local coordinate system (i.e., node coordinate system) of each vehicle to the global coordinate system. gα , effectively overcoming the initial pose uncertainty problem in cross-vehicle closed-loop detection.

[0123] In step S603 of some embodiments, please continue to refer to Figure 4 Based on the alignment of the global coordinate system, the local pose graph is fine-tuned (i.e., local coordinate system optimization). The cross-car closed-loop constraint is converted into a virtual observation in the local coordinate system, which is expressed as follows:

[0124]

[0125] Get adjacent constraint points by radius search α x k, construct a virtual closed-loop constraint chain as follows:

[0126]

[0127] Among them, Chain(·) represents the chain transfer of closed-loop constraints across vehicles for local optimization, z αβ 、z βγ 、z γα represents the closed-loop constraint between vehicles.

[0128] The optimization objective function integrates three types of constraints, as follows:

[0129]

[0130] Among them, e vir =Chain(z αβ , z βγ , z γα )-Pred(x α ), Pred(x α )∈SE(3) is the local pose predicted by the vehicle α’s own odometry. The vehicle’s local coordinate system is optimized using the Levenberg-Marquardt algorithm and implemented through the g2o framework.

[0131] Each vehicle maintains a globally consistent local map, which is then stitched together into a global map through coordinate transformation.

[0132] According to some embodiments of this application, please refer to Figure 5 and Figure 6 In the embodiment of the present application, the local point cloud data is preprocessed and the local SLAM algorithm is executed to construct a local map. Based on the geometric distribution characteristics and reflection intensity physical characteristics of the lidar point cloud, a radial-circumferential joint discretization space partitioning model is constructed, and a binary vertical discretization rule of geometric structure encoding and a reflectivity dynamic calibration function of intensity structure encoding are proposed. On this basis, each frame of the point cloud of the local map is fused with a circular Fourier perceptual hash (CFPH) and a row mode intensity distribution algorithm for feature encoding to construct a lightweight bimodal feature descriptor for low-bandwidth communication scenarios. After the vehicle-to-vehicle interaction bimodal feature descriptor, in order to solve the mismatch accumulation in cross-vehicle closed-loop detection, an incremental PCM algorithm is designed to construct a multi-constraint chain verification mechanism based on the cross-vehicle closed-loop constraint to remove false closed loops. According to the global consistency requirements of distributed collaborative mapping, a global coordinate system optimization model based on the SE(3) transformation matrix is ​​established, and the fine adjustment of the local pose graph is achieved through virtual observation chain transformation. Finally, the global coordinate transformation matrix and the local constraint error function are synchronously iterated through a hierarchical optimization architecture to achieve efficient splicing and dynamic error suppression of multi-vehicle maps.

[0133] The embodiments of the present invention address the problems existing in related multi-vehicle collaborative mapping technologies, such as low feature transmission efficiency in scenarios with limited communication bandwidth, high mismatch rate in cross-vehicle closed-loop detection, and global Figure 1 In order to overcome defects such as poor consistency, a distributed collaborative mapping method based on geometric-intensity dual-modal coding is provided. Through the complementary fusion of the annular space discretization coding of the geometric mode and the reflectivity statistical characteristics of the intensity mode, combined with the hierarchical optimization architecture, global correction and local fine adjustment of cross-vehicle posture constraints are achieved, thereby achieving the technical effects of reducing communication load, lowering closed-loop mismatching rate, and reducing multi-vehicle map splicing errors.

[0134] The present application also proposes a multi-sensing node collaborative mapping system, including:

[0135] The first module is used to construct a local map based on the local node based on the multi-frame point cloud data collected locally by the node;

[0136] The second module is used to discretize the local map to obtain multiple subspaces;

[0137] The third module is used to perform geometric feature coding and intensity feature coding on each subspace, and obtain geometric structure coding and intensity structure coding respectively;

[0138] The fourth module is used to describe the features of the local map based on the intensity structure coding and the geometric structure coding to obtain a bimodal feature descriptor of the local map;

[0139] The fifth module is used to construct a global map based on the bimodal feature descriptors of the local maps of multiple nodes and the node poses.

[0140] It can be understood that the contents of the above-mentioned multi-sensing node collaborative mapping method embodiment are all applicable to the present system embodiment. The functions specifically implemented by the present system embodiment are the same as those in the above-mentioned multi-sensing node collaborative mapping method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned multi-sensing node collaborative mapping method embodiment.

[0141] The present application also provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the multi-sensing node collaborative mapping method described above is implemented. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0142] See also Figure 7 , Figure 7 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0143] The processor 901 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0144] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called by the processor 901 to execute the multi-sensing node collaborative mapping method of the embodiments of this application;

[0145] Input / output interface 903, used to implement information input and output;

[0146] Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0147] Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 );

[0148] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0149] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned multi-sensing node collaborative mapping method.

[0150] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0151] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0152] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0153] The system embodiment described above is merely illustrative. The modules described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0154] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0155] The terms "first," "second," "third," "fourth," etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential sequence. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or apparatus.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules described above is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0157] The modules described above as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0159] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store programs.

[0160] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A multi-sensing node collaborative mapping method, characterized in that: The following steps are involved: Construct a local map based on the local node based on the multi-frame point cloud data collected locally by the node; Discretizing the local map to obtain multiple subspaces; Performing geometric feature coding and intensity feature coding on each of the subspaces respectively, and obtaining geometric structure coding and intensity structure coding accordingly; Performing feature description on the local map according to the intensity structure coding and the geometric structure coding to obtain a bimodal feature descriptor of the local map; A global map is constructed based on the bimodal feature descriptors of the local maps of multiple nodes and the node poses.

2. The multi-sensing node collaborative mapping method according to claim 1, characterized in that: The method of constructing a local map based on the local node based on the multi-frame point cloud data collected locally by the node includes the following steps: Dividing each frame of point cloud data into plane point cloud and edge point cloud; Determine a node motion transformation relationship between two frames of point cloud data according to the plane point cloud and the edge point cloud; A local map with the local node as the coordinate origin is constructed according to the point cloud data of each frame and the corresponding node motion transformation relationship.

3. The multi-sensing node collaborative mapping method according to claim 1, characterized in that: Discretizing the local map to obtain multiple subspaces includes the following steps: Taking the coordinate origin of the local map as the center, dividing the local map into a plurality of annular areas and a plurality of sectors in radial and circumferential directions respectively; A plurality of subspaces are determined according to the plurality of the ring areas and the plurality of the sectors.

4. The multi-sensing node collaborative mapping method according to claim 1, characterized in that: The step of performing geometric feature coding and intensity feature coding on each of the subspaces to obtain geometric structure coding and intensity structure coding respectively includes the following steps: Binary-encode the perception state of each point cloud in the subspace to obtain a geometric encoding value of the subspace; forming a geometric structure code according to the geometric code values ​​of the plurality of subspaces; Extracting the maximum reflection intensity of multiple point clouds in the subspace to obtain an intensity encoding value of the subspace; An intensity structure code is formed according to the intensity code values ​​of the plurality of subspaces.

5. The multi-sensing node collaborative mapping method according to claim 4, characterized in that: The bimodal feature descriptor includes an intensity distribution descriptor and a geometric distribution descriptor. The feature description of the local map is performed according to the intensity structure coding and the geometric structure coding to obtain the bimodal feature descriptor of the local map, including the following steps: Using a discrete indicator function to perform row-mode-based intensity distribution calculation on the intensity structure code to obtain an intensity distribution descriptor; A Fourier hash algorithm is used to calculate the annular grayscale distribution of the geometric structure code to obtain a geometric distribution descriptor.

6. The multi-sensing node collaborative mapping method according to claim 1, characterized in that: The method of constructing a global map based on the bimodal feature descriptors of the local maps of multiple nodes and the node poses includes the following steps: Perform loop closure detection based on the bimodal feature descriptor of the local map of each node and the node pose to obtain loop closure information; Optimizing the global coordinate system of the node pose according to the loop information to obtain a pose transformation relationship between each node coordinate system and the global coordinate system; A global map is constructed based on the posture transformation relationship and the local map of each node.

7. The multi-sensing node collaborative mapping method according to claim 6, characterized in that: The global coordinate system optimization is performed on the node pose according to the loop information to obtain the pose transformation relationship between each node coordinate system and the global coordinate system, including the following steps: Determine a first transformation matrix of all cross-node closed-loop constraints according to the loop closure information; Constructing a nonlinear least squares objective function according to the first transformation matrix and a second transformation matrix for representing the posture transformation relationship between the node coordinate system and the global coordinate system; The second transformation matrix in the nonlinear least squares objective function is optimized and iterated until a preset condition is met, thereby obtaining a posture transformation relationship between the node coordinate system and the global coordinate system.

8. A multi-sensing node collaborative mapping system, characterized in that: include: The first module is used to construct a local map based on the local node based on the multi-frame point cloud data collected locally by the node; The second module is used to discretize the local map to obtain multiple subspaces; The third module is used to perform geometric feature coding and intensity feature coding on each of the subspaces, and obtain geometric structure coding and intensity structure coding respectively; a fourth module, configured to perform feature description on the local map according to the intensity structure coding and the geometric structure coding to obtain a bimodal feature descriptor of the local map; The fifth module is used to construct a global map based on the bimodal feature descriptors of the local maps of multiple nodes and the node poses.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for implementing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A storage medium, which is a computer-readable storage medium and is used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of claims 1 to 7.

Images