A multi-millimeter wave radar point cloud fusion method suitable for modeling a smoke environment

Abstract

A multi-millimeter wave radar point cloud fusion method suitable for smoke environment modeling relates to the field of building interior simultaneous localization and map building, and comprises the following steps: step S1, deploying a plurality of millimeter wave radars on unmanned detection equipment; step S2, performing main direction estimation on collected radar point cloud data; step S3, extracting point cloud data set key points; step S4, constructing a candidate edge set; and step S5, solving a minimum Manhattan network. The present application improves the point cloud density of the collected data by deploying 4D millimeter wave radars on unmanned equipment, obtains a candidate edge set extending along the Manhattan direction based on the point cloud data set key point set, solves the minimum Manhattan network after sorting the candidate edge set, obtains the candidate edge set conforming to the minimum Manhattan network rule, and further realizes the fusion of a plurality of point cloud data on the premise of eliminating redundant data generated after the superposition of the multi-millimeter wave radar point cloud data.

Description

Technical Field

[0001] This invention relates to the field of synchronous positioning and mapping technology inside buildings, and in particular to a multi-millimeter-wave radar point cloud fusion method suitable for modeling dense smoke environments. Background Technology

[0002] In recent years, with the acceleration of urbanization, the number of high-rise and super high-rise buildings exceeding 100 meters in height in my country has continued to grow. High-rise buildings are characterized by dense populations, enclosed spaces, and difficulties in evacuation and smoke extraction. Once a fire occurs, it can easily lead to significant casualties and pose a severe challenge to fire rescue operations.

[0003] In building fires, the combustion process often produces large amounts of high-temperature, toxic smoke that fills enclosed or semi-enclosed spaces, severely impacting the visibility and operational capabilities of firefighters. Traditionally, upon arrival, fire brigades first activate the building's existing smoke extraction systems or install external smoke extraction equipment, only proceeding with interior search and rescue once the fire is under control or the smoke has thinned. However, if the internal temperature is too high or the smoke is heavy, individual firefighters cannot conduct interior attacks to prevent injury or death to rescue personnel.

[0004] In recent years, with the rapid development of unmanned detection equipment technology, platforms such as drones, robots, and robotic dogs, equipped with high-performance multispectral sensors, lidar, and thermal imagers, have achieved a leap from single visible light detection to multi-dimensional perception. Through AI edge computing and real-time data transmission, unmanned detection equipment has the ability to autonomously plan paths and intelligently identify fire points and the location of trapped personnel, significantly improving the accuracy and efficiency of perception in complex fire environments.

[0005] The combination of unmanned detection equipment and millimeter-wave radar provides technical support for solving Simultaneous Localization and Mapping (SLAM) in dense smoke environments. Millimeter-wave radar, with electromagnetic waves of wavelengths between 1 and 10 millimeters, possesses strong penetrating power, effectively penetrating smoke, dust, and even some non-metallic obstacles. It is unaffected by ambient light and visual obstruction caused by dense smoke, thus compensating for the performance degradation or failure of optical sensors such as cameras and lidar under dense smoke conditions. In SLAM, millimeter-wave radar, through Doppler effect and time-of-flight (TOF) measurements, can stably acquire environmental point cloud data, supporting basic applications in pose estimation and map building.

[0006] However, limited by the finite number of virtual antenna channels and wavelength, the sparsity of the sampling point cloud of millimeter-wave radar is an inherent physical characteristic. The resolution of radar in the angular dimension (including azimuth and elevation) is far lower than that of lidar, making it impossible to perform fine sampling of object contours. A target (such as a vehicle) may be clearly outlined by tens of thousands of points in a lidar point cloud, while in a millimeter-wave radar point cloud it is often represented by only a few dozen or even a few sparse scattering points. This sparsity makes point clouds difficult to use directly for high-precision object classification and shape recognition, fundamentally limiting the application of millimeter-wave radar in environments requiring detailed modeling. Summary of the Invention

[0007] In view of the above-mentioned shortcomings and deficiencies of the existing technology, the present invention provides a multi-millimeter-wave radar point cloud fusion method suitable for modeling dense smoke environments. By deploying several 4D millimeter-wave radars on unmanned equipment, the density of the collected point cloud is increased. Based on the key point set of the point cloud dataset, a set of candidate edges extending along the Manhattan direction is obtained. After sorting the candidate edge set, the minimum Manhattan network is solved to obtain a set of candidate edges that conform to the minimum Manhattan network rules. Thus, the fusion of multiple point cloud data is achieved under the premise of eliminating redundant data generated after the superposition of multi-millimeter-wave radar point cloud data. At the same time, the minimum point cloud description for multi-millimeter-wave radar to detect the same target is realized, reducing the amount of point cloud data and facilitating subsequent data transmission requirements.

[0008] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0009] A multi-millimeter-wave radar point cloud fusion method suitable for modeling dense smoke environments includes the following steps:

[0010] Step S1: Deploy several millimeter-wave radars on unmanned detection equipment;

[0011] Step S2: Estimate the principal direction of the collected radar point cloud data:

[0012] Step S201: Using the coordinates of the unmanned detection equipment as the reference Manhattan coordinate system, and based on the positional relationship between the millimeter-wave radar and the center point of the unmanned detection equipment, project several radar point cloud data onto the reference Manhattan coordinate system to obtain a multi-radar point cloud set based on the Manhattan coordinate system. Where: C is the radar point cloud, k is a positive integer greater than 2, C k This represents the k-th radar point cloud.

[0013] Step S202: Use the Random Sample Consensus Algorithm (RANSAC) fitting plane method to fit each point cloud C. i Extract planar features to obtain the planar normal vector n of the point cloud. i ;

[0014] Step S203: Collect all plane normal vectors to obtain the plane normal vector matrix. Where: k is a positive integer greater than 2, n k This is the normal vector of the k-th plane;

[0015] Step S204: Perform singular value decomposition on the plane normal vector matrix N. Where: U is an m×m orthogonal matrix, and I m It is an m×m identity matrix. Let V be an m×3 diagonal matrix, and let V be a 3×3 orthogonal matrix. I3 is a 3×3 identity matrix, where m is a positive integer greater than 0. The first three columns d1, d2, and d3 of V are calculated, which are the basis vectors of the Manhattan coordinate system. Finally, the projection basis vector matrix based on the Manhattan coordinate system is obtained. ;

[0016] Step S3: Extract key points from the point cloud dataset;

[0017] Step S301: Extract the multi-radar point cloud set Find the corner point and obtain its coordinates. ;where x i Let be the X-coordinate of the i-th corner point, where y i Let z be the y-coordinate of the i-th corner point, where z i Let Z be the Z coordinate of the i-th corner point;

[0018] Step S302: Set the coordinates of the corner point P i Projecting onto the nearest Manhattan direction yields the keypoint set of the point cloud dataset. Where p1 is the first keypoint, p2 is the second keypoint, and p... n This is the nth key point;

[0019] Step S4: Construct a candidate edge set:

[0020] Key point set of point cloud dataset For each keypoint within the boundary, extend a line segment along the Manhattan direction R until it intersects with other keypoints or obstacles, thus obtaining a set of candidate edges. Where e1 is the first candidate edge, e2 is the second candidate edge, and e... l Let this be the l-th candidate edge;

[0021] Step S5: Solve for the minimum Manhattan network:

[0022] Step S501: Define E as the set of candidate edges that conform to the minimum Manhattan network. For the candidate edge set... Sort the candidate edges in ascending order of their lengths to obtain the sorted set of edges. ,in, This is the first candidate edge after sorting. This is the second candidate edge after sorting. Let be the l-th candidate edge after sorting, and ;

[0023] Step S502: Traverse the sorted candidate edge set Each inner edge , For the sorted i-th candidate edge, calculate Able to cover The quantity, if the quantity is greater than 0, then Joined to E, and from Remove relevant key points until the process is complete. Completed or If the result is empty, the final E is the set of candidate edges that conform to the minimum Manhattan network rule.

[0024] Furthermore, in step S1, the detection ranges of several millimeter-wave radars deployed on the unmanned detection equipment overlap by more than 30%.

[0025] Furthermore, in step S301, corner points are extracted using Harris3D or the planar intersection method.

[0026] Furthermore, in step S302, the corner coordinates P are... i The method for projecting to the nearest Manhattan direction is, if and Then the projection onto the X direction, if and Then it is projected onto the Y direction, if and Then, projecting onto the Z direction, we get... ;

[0027] Where, round is the projection of the projection basis vector matrix R based on the Manhattan coordinate system.

[0028] The beneficial effects of this invention are as follows: By deploying several millimeter-wave radars on unmanned detection equipment, the superposition of multiple radar signals in the same detection area at the same time increases the density of the collected point cloud. Based on the key point set of the point cloud dataset, a set of candidate edges extending along the Manhattan direction is obtained. After sorting the candidate edge set, the minimum Manhattan network is solved to obtain a set of candidate edges that conforms to the minimum Manhattan network rules. This is the minimum point cloud target description dataset after fusing all millimeter-wave radar point clouds. Thus, the fusion of multiple point cloud data is achieved under the premise of eliminating redundant data generated after the superposition of multiple millimeter-wave radar point cloud data. At the same time, the minimum point cloud description of multiple millimeter-wave radars detecting the same target is achieved, reducing the amount of point cloud data and facilitating subsequent data transmission requirements. Attached Figure Description

[0029] Figure 1 A schematic diagram illustrating the deployment of millimeter-wave radar on unmanned detection equipment. Detailed Implementation

[0030] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] This invention provides a multi-millimeter-wave radar point cloud fusion method suitable for modeling dense smoke environments, comprising the following steps:

[0032] Step S1: Deploy several millimeter-wave radars on the unmanned detection equipment. Specifically, the detection ranges of adjacent millimeter-wave radars deployed on the unmanned detection equipment overlap by more than 30%. More specifically, such as... Figure 1 As shown, a 4D millimeter-wave radar can be deployed on each of the following sides of unmanned detection equipment: top, bottom, front, back, left, and right. One of these radars needs to be designated as the master radar, and the other radars connect to it via a synchronization interface. The master radar is responsible for sending synchronization signals to ensure that multiple radars sample data simultaneously, thus eliminating sampling time errors between the radars.

[0033] Step S2: Estimate the principal direction of the collected radar point cloud data:

[0034] Step S201: Using the coordinates of the unmanned detection equipment as the reference Manhattan coordinate system, and based on the positional relationship between the millimeter-wave radar and the center point of the unmanned detection equipment, project several radar point cloud data onto the reference Manhattan coordinate system to obtain a multi-radar point cloud set based on the Manhattan coordinate system. Where: C is the radar point cloud, k is a positive integer greater than 2, C k This represents the k-th radar point cloud.

[0035] Step S202: Use the Random Sample Consensus Algorithm (RANSAC) fitting plane method to fit each point cloud C. i Extract planar features to obtain the planar normal vector n of the point cloud. i ;

[0036] Step S203: Collect all plane normal vectors to obtain the plane normal vector matrix. Where: k is a positive integer greater than 2, n k This is the normal vector of the k-th plane;

[0037] Step S204: Perform Singular Value Decomposition (SVD) on the plane normal vector matrix N. Where: U is an m×m orthogonal matrix, and I m It is an m×m identity matrix. Let V be an m×3 diagonal matrix, and let V be a 3×3 orthogonal matrix. I3 is a 3×3 identity matrix, V T The first three columns of V, d1, d2, and d3, are calculated and represent the basis vectors of the Manhattan coordinate system. Finally, the projection basis vector matrix based on the Manhattan coordinate system is obtained. .

[0038] Step S3: Extract key points from the point cloud dataset;

[0039] Step S301: Extract the multi-radar point cloud set Find the corner point and obtain its coordinates. ;where x i Let be the X-coordinate of the i-th corner point, where y i Let z be the y-coordinate of the i-th corner point, where z i Let Z be the Z coordinate of the i-th corner point. Specifically, Harris3D or the planar intersection method can be used to analyze the multi-radar point cloud set. Process and extract corner points.

[0040] Step S302: Set the coordinates of the corner point P i Projecting onto the nearest Manhattan direction yields the keypoint set of the point cloud dataset. Where p1 is the first keypoint, p2 is the second keypoint, and p... n This is the nth key point.

[0041] Specifically, the corner coordinates P i The method for projecting to the nearest Manhattan direction is, if and Then the projection onto the X direction, if and Then it is projected onto the Y direction, if and Then, projecting onto the Z direction, we get... ;

[0042] Where, round is the projection of the projection basis vector matrix R based on the Manhattan coordinate system.

[0043] Step S4: Construct a candidate edge set:

[0044] Key point set of point cloud dataset For each keypoint within the area, extend a line segment along the Manhattan direction R until it intersects with other keypoints or obstacles, for example, from... Generate edges along the X direction ,in The coordinates of the next key point are used to obtain the candidate edge set. Where e1 is the first candidate edge, e2 is the second candidate edge, and e... l This is the l-th candidate edge.

[0045] Step S5: Solve for the minimum Manhattan network:

[0046] Step S501: Define E as the set of candidate edges that conform to the minimum Manhattan network. For the candidate edge set... Sort the candidate edges in ascending order of their lengths to obtain the sorted set of edges. ,in, This is the first candidate edge after sorting. This is the second candidate edge after sorting. Let be the l-th candidate edge after sorting, and ;

[0047] Step S502: Traverse the sorted candidate edge set Each inner edge , For the sorted i-th candidate edge, calculate Able to cover The quantity, if the quantity is greater than 0, then Joined to E, and from Remove relevant key points until the process is complete. Completed or If empty, the final E is the set of candidate edges that conforms to the minimum Manhattan network rule, which is the minimum point cloud target description dataset after fusing all millimeter-wave radar point clouds.

[0048] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Any modifications, alterations, substitutions, and variations made by those skilled in the art to the above embodiments are within the scope of the present invention.

Claims

1. A multi-millimeter wave radar point cloud fusion method suitable for modeling smoke environment, characterized in that, The method comprises the following steps: Step S1, deploying a plurality of millimeter wave radars on unmanned detection equipment; Step S2, performing main direction estimation on collected radar point cloud data; Step S201, taking the unmanned detection equipment coordinates as the reference Manhattan coordinate system, projecting a plurality of radar point cloud data to the reference Manhattan coordinate system according to the position relationship between the millimeter wave radar and the center point of the unmanned detection equipment, and obtaining a plurality of radar point cloud sets based on the Manhattan coordinate system ; wherein: C is a radar point cloud, k is a positive integer greater than 2, C k is the kth radar point cloud; Step S202, a fitting plane method of random sample consensus algorithm RANSAC is used to each point cloud C i extracting a plane feature to obtain a plane normal vector n of the point cloud i ; Step S203, collect all plane normal vectors to obtain a plane normal vector matrix ; wherein: k is a positive integer greater than 2, n k is the kth plane normal vector; Step S204, singular value decomposition is performed on the plane normal vector matrix N, wherein: U is an m x m orthogonal matrix, and , I m is an m x m identity matrix, is an m x 3 diagonal matrix, V is a 3 x 3 orthogonal matrix, and , I3 is a 3 x 3 identity matrix, wherein m is a positive integer greater than 0, the first three columns d1, d2, d3 of V are calculated, which are the base vectors of the Manhattan coordinate system; finally, the projection base vector matrix based on the Manhattan coordinate system is obtained. Step S3, extracting key points of the point cloud data set; Step S301, extracting a multi-radar point cloud set , to obtain corner point coordinates ; wherein x i is the X coordinate of the i-th corner point, wherein y i is the Y coordinate of the i-th corner point, wherein z i is the Z coordinate of the i-th corner point; Step S302, the corner point coordinates P i Project to the nearest Manhattan direction, get the point cloud data set key point set ; wherein p1 is the first key point, p2 is the second key point, p n n is the nth key point; Step S4, constructing a candidate edge set; Key point set in point cloud dataset For each key point in the key point set, a line segment is extended along the Manhattan direction R until it intersects with other key points or obstacles, obtaining a candidate edge set ; wherein, wherein e1 is the first candidate edge, e2 is the second candidate edge, and e l is the lth candidate edge Step S5, solving a minimum Manhattan network; Step S501, defining E as a candidate edge set meeting a minimum Manhattan network, and performing the following steps on the candidate edge set E arranging the candidate edges in ascending order according to the edge length to obtain a sorted candidate edge set wherein, is the first candidate edge in the sorted candidate edge set, is the second candidate edge in the sorted candidate edge set, is the lth candidate edge in the sorted candidate edge set, and ; Step S502: Traverse the sorted candidate edge set Each inner edge , For the sorted i-th candidate edge, calculate Able to cover The quantity, if the quantity is greater than 0, then Joined to E, and from Remove relevant key points until the process is complete. Completed or If the result is empty, the final E is the set of candidate edges that conform to the minimum Manhattan network rule.

2. The multi-millimeter wave radar point cloud fusion method suitable for modeling a dense smoke environment according to claim 1, characterized in that: In the step S1, the detection ranges of adjacent millimeter wave radars are overlapped by more than 30%.

3. The multi-millimeter wave radar point cloud fusion method for modeling dense smoke environment according to claim 1, characterized in that: In the step S301, the corner points are extracted by using Harris3D or plane intersection method.

4. The multi-millimeter wave radar point cloud fusion method for modeling dense smoke environment according to claim 1, characterized in that, In the step S302, the corner point coordinate P i The method of projecting to the nearest Manhattan direction is that if and , then projecting to the X direction, if and , then projecting to the Y direction, if and , then projecting to the Z direction, obtaining: ； Wherein, round is the projection based on the projection basis vector matrix R of the Manhattan coordinate system.

Images