Dual radar posture construction method, device, electronic device and storage medium

Abstract

The present application provides a dual radar posture construction method, device, electronic device and storage medium, wherein the method includes: obtaining first radar data of laser radar and second radar data of millimeter wave radar; preprocessing the first radar data and the second radar data respectively to obtain a laser point cloud map and a millimeter wave point cloud map; generating a dual-range point cloud map according to the laser point cloud map and the millimeter wave point cloud map; performing posture construction according to the dual-range point cloud map to obtain a first posture; performing motion compensation on the first posture to obtain a target posture. The implementation of the present application can improve the accuracy and efficiency of posture construction, realize rapid detection in complex weather and complex environments, improve information interaction between multiple radars, enable different radars to complement each other's advantages, and reduce local errors.

Description

Technical Field

[0001] The present application relates to the field of map processing technology, and in particular to a dual radar posture construction method, device, electronic device and storage medium. Background Art

[0002] Real-time positioning and mapping technologies have been applied in many fields such as ports, logistics, and warehousing. However, traditional posture construction technologies are usually unable to handle complex weather conditions and have poor processing capabilities in complex environments such as rain, snow, and dynamic conditions.

[0003] Traditional pose construction cannot quickly detect complex weather conditions, and pose construction works well for static scenes. However, in real environments, there are many dynamic scenes and the environment is complex and changeable, which seriously interferes with pose calculation and has low accuracy. Existing technologies require high time and hardware costs to achieve high-precision pose construction in complex environments, and there is a lack of information exchange between multiple radars, which makes it impossible to complement each other's advantages. The computational cost is high, and if the local error is large, it is difficult to eliminate the error in a timely manner. Summary of the invention

[0004] The purpose of this application is to provide a dual-radar posture construction method, device, electronic device and storage medium, which can improve the accuracy and efficiency of posture construction, realize rapid detection in complex weather and complex environments, improve information interaction between multiple radars, enable different radars to complement each other's advantages, and reduce local errors.

[0005] In a first aspect, an embodiment of the present application provides a dual radar pose construction method, the method comprising:

[0006] Acquire first radar data of the laser radar and second radar data of the millimeter wave radar;

[0007] Preprocessing the first radar data and the second radar data respectively to obtain a laser point cloud image and a millimeter wave point cloud image;

[0008] Generate a dual-range point cloud image according to the laser point cloud image and the millimeter wave point cloud image;

[0009] Perform pose construction according to the dual-distance point cloud image to obtain the first pose;

[0010] Perform motion compensation on the first posture to obtain a target posture.

[0011] In the above implementation process, the posture construction is carried out through the interaction of dual radars, and the moving objects are detected and compensated, which can improve the accuracy and efficiency of posture construction, realize rapid detection in complex weather and complex environments, improve the information interaction between multiple radars, and enable different radars to complement each other's advantages and reduce local errors.

[0012] Furthermore, the step of generating a dual-range point cloud image according to the laser point cloud image and the millimeter wave point cloud image includes:

[0013] Normalizing the laser point cloud image and the millimeter wave point cloud image respectively to obtain a laser point cloud image and a millimeter wave point cloud image with consistent pixel formats;

[0014] The pixel points between the laser point cloud image and the millimeter wave point cloud image with the same pixel format are correlated to obtain the dual-range point cloud image.

[0015] In the above implementation process, the laser point cloud map and the millimeter wave point cloud map are normalized and then correlated with each other, which can improve the matching degree between the laser point cloud map and the millimeter wave point cloud map, and provide a data basis for improving the accuracy of posture construction.

[0016] Furthermore, the step of correlating the pixel points between the laser point cloud image and the millimeter wave point cloud image with the same pixel format to obtain the dual-range point cloud image includes:

[0017] The pixel points between the laser point cloud image and the millimeter wave point cloud image with the same pixel format are matched one by one to obtain an initial dual-range point cloud image;

[0018] Traversing each pixel point in the initial dual-distance point cloud image;

[0019] If a pixel point is a pixel point in both the laser point cloud map and the millimeter wave point cloud map, the pixel point is marked as the first flag position; if the pixel point only belongs to a pixel point in the laser point cloud map or the millimeter wave point cloud map, the pixel point is marked as the second flag position to obtain the dual-distance point cloud map.

[0020] In the above implementation process, the pixels of the laser point cloud image and the millimeter wave point cloud image are matched and the pixels that exist at the same time are marked to facilitate the subsequent detection of dynamic objects and improve the data processing efficiency.

[0021] Furthermore, the step of constructing a pose according to the dual-distance point cloud image to obtain the first pose includes:

[0022] Performing complex weather detection on the dual-range point cloud image to obtain complex weather estimation;

[0023] Determining whether the complex weather estimate is a preset threshold;

[0024] If so, the millimeter wave point cloud image is constructed to obtain the first pose.

[0025] In the above implementation process, complex weather detection is performed on the dual-range point cloud image to obtain complex weather estimation, and then pose construction is performed. The impact of complex weather on accuracy can be taken into account at the beginning of pose construction, thereby improving the accuracy of subsequent pose construction.

[0026] Furthermore, the step of constructing the pose of the millimeter wave point cloud image to obtain the first pose includes:

[0027] Performing dynamic object removal on the millimeter wave point cloud image to obtain a millimeter wave point cloud image after the dynamic object removal;

[0028] The millimeter-wave point cloud image after the dynamic object is removed is subjected to posture construction to obtain the first posture.

[0029] In the above implementation process, dynamic objects are removed from the millimeter wave point cloud image before posture construction, which can solve the influence of dynamic objects in the positioning process and improve the positioning accuracy.

[0030] Furthermore, the step of removing dynamic objects from the millimeter wave point cloud image to obtain the millimeter wave point cloud image after the dynamic objects are removed includes:

[0031] Obtaining the speed of the pixel point of the moving object in the millimeter wave point cloud image;

[0032] Determining whether the speed is greater than zero;

[0033] If yes, determine that the pixel is a moving pixel, and mark the moving pixel as a motion flag;

[0034] All motion flags in the millimeter wave point cloud image are replaced with blank pixels to obtain the millimeter wave point cloud image after the dynamic objects are removed.

[0035] In the above implementation process, the motion flag is replaced by a blank pixel, so that the position of the dynamic object can be quickly and accurately obtained, and it can be reasonably replaced to reduce the positioning error caused by the dynamic object.

[0036] Furthermore, the step of constructing the pose of the millimeter wave point cloud image after the dynamic object is removed to obtain the first pose includes:

[0037] Obtaining a current frame pixel point set and a previous frame pixel point set of the millimeter wave point cloud image;

[0038] Inter-frame matching is performed on the current frame pixel point set and the previous frame pixel point set to obtain the first posture.

[0039] In the above implementation process, the millimeter wave point cloud map is matched between frames to obtain the first pose, which can improve the real-time nature of map construction, increase the construction speed, and avoid excessive inter-frame errors that affect accuracy.

[0040] Further, the step of performing motion compensation on the first posture to obtain a target posture comprises:

[0041] Obtaining the carrier velocity of the first posture at the current moment and the carrier velocity at the previous moment;

[0042] Obtaining a compensation displacement according to the carrier speed at the current moment and the carrier speed at the previous moment;

[0043] Compensating the compensation displacement to the blank pixel points of the millimeter wave point cloud image after the dynamic object is removed, to obtain a compensated millimeter wave point cloud image;

[0044] Performing pose construction on the compensated millimeter wave point cloud image to obtain a second pose;

[0045] Obtaining an error value between the first posture and the second posture;

[0046] The first posture and the second posture are corrected according to the error value to obtain the target posture.

[0047] In the above implementation process, after removing and compensating the dynamic objects, a secondary pose construction is performed, and then the errors of the two pose constructions are corrected to obtain the target pose, which can improve the map accuracy and make the pose construction simpler and more efficient.

[0048] In a second aspect, an embodiment of the present application further provides a dual radar posture construction device, the device comprising:

[0049] An acquisition module, used to acquire first radar data of the laser radar and second radar data of the millimeter wave radar;

[0050] A preprocessing module, used to preprocess the first radar data and the second radar data respectively to obtain a laser point cloud image and a millimeter wave point cloud image;

[0051] A generating module, used for generating a dual-range point cloud image according to the laser point cloud image and the millimeter wave point cloud image;

[0052] A posture construction module, used to construct a posture according to the dual-distance point cloud image to obtain a first posture;

[0053] The motion compensation module is used to perform motion compensation on the first posture to obtain a target posture.

[0054] In the above implementation process, the posture construction is carried out through the interaction of dual radars, and the moving objects are detected and compensated, which can improve the accuracy and efficiency of posture construction, realize rapid detection in complex weather and complex environments, improve the information interaction between multiple radars, and enable different radars to complement each other's advantages and reduce local errors.

[0055] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0056] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects.

[0057] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method as described in any one of the first aspects.

[0058] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0059] And it can be implemented according to the contents of the specification. The following is a detailed description of the preferred embodiments of the present application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the range values. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 A schematic diagram of a dual radar pose construction method provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of the structural composition of a dual radar posture construction device provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the structural composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0065] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0066] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0067] Embodiment 1

[0068] Figure 1 is a flow chart of a dual radar posture construction method provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0069] S1, obtaining first radar data of the laser radar and second radar data of the millimeter wave radar;

[0070] S2, preprocessing the first radar data and the second radar data respectively to obtain a laser point cloud image and a millimeter wave point cloud image;

[0071] S3, generating a dual-range point cloud image according to the laser point cloud image and the millimeter wave point cloud image;

[0072] S4, constructing the pose according to the double-distance point cloud image to obtain the first pose;

[0073] S5, performing motion compensation on the first pose to obtain the target pose.

[0074] In the above implementation process, the posture construction is carried out through the interaction of dual radars, and the moving objects are detected and compensated, which can improve the accuracy and efficiency of posture construction, realize rapid detection in complex weather and complex environments, improve the information interaction between multiple radars, and enable different radars to complement each other's advantages and reduce local errors.

[0075] The laser radar used for data collection is a 3D laser radar with 64 scanning lines and a frequency of 10HZ. It collects 10 frames of point cloud data per second. The point cloud data contains the distance information from the drone to the environment.

[0076] In S1, the millimeter wave radar used in the embodiment of the present application is a 4D millimeter wave radar, which can emit 76-81GHz electromagnetic waves to the outside world. After being reflected by the surrounding environment, the antenna receives the radar echo signal. By processing the echo signal, the second radar data containing the relative distance and relative speed information between various objects in the environment and the millimeter wave radar can be obtained.

[0077] In S2, the first radar data and the second radar data are preprocessed to obtain a laser point cloud image and a millimeter wave point cloud image.

[0078] Each beam of the laser radar in the embodiment of the present application collects 360 laser points, for a total of 64 beams. Each laser point is therefore depth projected to generate a two-dimensional image, namely a laser point cloud map, in which each pixel contains the distance information from the carrier to the reflection point.

[0079] The millimeter-wave radar scanning in the embodiment of the present application has a horizontal field of view of 120°, an angular resolution of 1°, a vertical range of 45°, and a vertical resolution of 1°. A two-dimensional depth map is made for the point cloud obtained by the millimeter-wave radar scanning. To ensure subsequent alignment with the laser point cloud map, the horizontal field of view is set to [-60°, 60°] to generate a two-dimensional image, namely a millimeter-wave point cloud map. Each pixel of the image contains the distance information and relative speed information from the carrier to the reflection point.

[0080] Furthermore, S3 includes:

[0081] Normalizing the laser point cloud image and the millimeter wave point cloud image respectively to obtain the laser point cloud image and the millimeter wave point cloud image with the same pixel format;

[0082] The pixel points between the laser point cloud image and the millimeter wave point cloud image with the same pixel format are correlated to obtain a dual-range point cloud image.

[0083] In the above implementation process, the laser point cloud map and the millimeter wave point cloud map are normalized and then correlated with each other, which can improve the matching degree between the laser point cloud map and the millimeter wave point cloud map, and provide a data basis for improving the accuracy of posture construction.

[0084] A dual-range point cloud image is generated from a laser point cloud image and a millimeter-wave point cloud image. The pixel format of the laser point cloud image is 512*64, and the pixel format of the millimeter-wave point cloud image is 120*45. The laser point cloud image and the millimeter-wave point cloud image are normalized to generate a 120*45 image with the same pixel format.

[0085] Furthermore, the step of correlating the pixel points between the laser point cloud image and the millimeter wave point cloud image with the same pixel format to obtain the dual-range point cloud image includes:

[0086] The pixels of the laser point cloud image and the millimeter wave point cloud image with the same pixel format are matched one by one to obtain the initial dual-range point cloud image;

[0087] Traverse each pixel in the initial double-distance point cloud image;

[0088] If the pixel point is a pixel point in both the laser point cloud image and the millimeter wave point cloud image, the pixel point is marked as the first flag position. If the pixel point only belongs to a pixel point in the laser point cloud image or the millimeter wave point cloud image, the pixel point is marked as the second flag position to obtain a dual-distance point cloud image.

[0089] In the above implementation process, the pixels of the laser point cloud image and the millimeter wave point cloud image are matched and the pixels that exist at the same time are marked to facilitate the subsequent detection of dynamic objects and improve the data processing efficiency.

[0090] In the embodiment of the present application, since the laser radar scans 360°, while the scanning angle of the millimeter-wave radar is limited, in order to align the millimeter-wave point cloud map of the millimeter-wave radar with the laser point cloud map, the embodiment of the present application needs to take a partial laser point cloud image in the positive direction [-60°, 60°] of the laser point cloud map. In addition, since the number of rows of the laser point cloud map and the millimeter-wave point cloud map is inconsistent, and there is a height change, in order to reduce the interference caused by the above errors, it is necessary to remove the laser point cloud line bundles in the top 5 rows and the bottom 10 rows again to obtain a 120*45 partial laser point cloud map. Then the pixels in the 120*45 partial laser point cloud map and the millimeter-wave point cloud map are matched one by one.

[0091] If a pixel is located in both the laser point cloud map and the millimeter wave point cloud map, the pixel is marked. The specific formula is as follows:

[0092] ;

[0093] Among them, p represents the pixel point in the double distance point cloud image, and represents the millimeter wave point cloud image, Represents a laser point cloud image, FLAG represents a flag bit. If the pixel belongs to both a millimeter wave point cloud image and a laser point cloud image, the flag bit is set to 1 (the first flag bit in the embodiment of the present application is 1), otherwise it is set to 0. A pixel point with a flag bit of 1 indicates that the pixel point has a dual distance.

[0094] Furthermore, S4 includes:

[0095] Perform complex weather detection on the dual-range point cloud image to obtain complex weather estimation;

[0096] Determine whether the complex weather estimate is a preset threshold;

[0097] If so, construct the pose of the millimeter wave point cloud image and obtain the first pose.

[0098] In the above implementation process, complex weather detection is performed on the dual-range point cloud image to obtain complex weather estimation, and then pose construction is performed. The impact of complex weather on accuracy can be taken into account at the beginning of pose construction, thereby improving the accuracy of subsequent pose construction.

[0099] Calculate the sum of the distance differences of the double distances of the pixels whose flag bit is 1. The specific calculation formula is as follows:

[0100] ;

[0101] in, It represents the sum of the distance differences between the double distances of the pixels whose flag bit is 1. i represents an integer, and n represents the number of pixels whose flag bit is 1. represents the x-axis coordinate value of the i-th pixel point obtained by the millimeter-wave radar, represents the y-axis coordinate value of the i-th pixel point obtained by the millimeter-wave radar, represents the z-axis coordinate value of the i-th pixel point obtained by the millimeter-wave radar, Indicates the distance from the center point of the millimeter wave radar to the center point of the laser radar. Represents the x-axis coordinate value of the i-th pixel point obtained by the laser radar, Represents the y-axis coordinate value of the i-th pixel point obtained by the laser radar, Represents the z-axis coordinate value of the i-th pixel point obtained by the laser radar.

[0102] The complex weather estimate is obtained based on the sum of the distance differences of the double distances of the pixels whose flag bit is 1, and the rain and fog threshold is initialized. , the calculation formula for complex weather estimation is as follows:

[0103] ;

[0104] in, It represents the sum of the distance differences between the double distances of the pixel points whose flag bit is 1. F is the complex weather estimation. If the complex weather estimation is 1 (preset threshold), it indicates that it is rainy and foggy at this time. In this case, the position and posture of the carrier needs to be calculated using the millimeter wave radar first.

[0105] Furthermore, the millimeter wave point cloud image is constructed to obtain the first pose, including:

[0106] Performing dynamic object removal on the millimeter wave point cloud image to obtain a millimeter wave point cloud image after the dynamic object removal;

[0107] The pose of the millimeter-wave point cloud image after the dynamic objects are removed is constructed to obtain the first pose.

[0108] In the above implementation process, dynamic objects are removed from the millimeter wave point cloud image before posture construction, which can solve the influence of dynamic objects in the positioning process and improve the positioning accuracy.

[0109] Furthermore, the step of removing dynamic objects from the millimeter wave point cloud image to obtain the millimeter wave point cloud image after the dynamic objects are removed includes:

[0110] Obtain the speed of the pixel points of the moving object in the millimeter wave point cloud image;

[0111] Determine whether the speed is greater than zero;

[0112] If yes, determine that the pixel is a moving pixel, and mark the moving pixel as a motion flag;

[0113] All motion flags in the millimeter-wave point cloud image are replaced with blank pixels to obtain the millimeter-wave point cloud image after dynamic objects are removed.

[0114] In the above implementation process, the motion flag is replaced by a blank pixel, so that the position of the dynamic object can be quickly and accurately obtained, and it can be reasonably replaced to reduce the positioning error caused by the dynamic object.

[0115] The speed of the moving object is detected, and the formula is as follows:

[0116] ;

[0117] in, Indicates the speed of the pixel point of the moving object in the millimeter wave point cloud image, represents the wavelength, Indicates the relative phase difference between the two signals in the received echo. Indicates a time interval.

[0118] The embodiment of the present application uses a deep search algorithm to traverse the millimeter wave point cloud image and mark the pixels of dynamic objects. The formula is as follows:

[0119] ;

[0120] in, Indicates the motion flag. If the motion flag is 1, it means that the pixel is a pixel of a moving object (motion pixel).

[0121] The pixels of the motion flag are replaced with blank pixels.

[0122] Furthermore, the step of constructing the pose of the millimeter wave point cloud image after the dynamic objects are removed to obtain the first pose includes:

[0123] Get the current frame pixel point set and the previous frame pixel point set of the millimeter wave point cloud image;

[0124] The pixel point set of the current frame is matched with the pixel point set of the previous frame to obtain the first pose.

[0125] In the above implementation process, the millimeter wave point cloud map is matched between frames to obtain the first pose, which can improve the real-time nature of map construction, increase the construction speed, and avoid excessive inter-frame errors that affect accuracy.

[0126] The current frame's millimeter wave point cloud image pixel set B and the previous frame's millimeter wave point cloud image pixel set , the ICP algorithm is used to calculate the inter-frame translation position. The inter-frame matching algorithm is a classic algorithm for calculating the position in the lidar positioning and mapping algorithm.

[0127] The centroids of all pixels in the current frame pixel set and the previous frame pixel set are:

[0128] ;

[0129] ;

[0130] in, Represents the centroid of the pixel points in the current frame pixel set. represents the centroid of the pixel in the previous frame, N represents the total number of pixels, j represents the sequence number, pB represents the current frame pixel set, and q Represents the pixel set of the previous frame.

[0131] Remove the centroid:

[0132] ;

[0133] in, Represents the position information of all pixels in the current frame after removing the centroid. Represents the position information of all pixels in the previous frame after removing the centroid. Indicates the position information of all pixels in the current frame pixel set without removing the centroid. It represents the position information of all pixels in the previous frame without removing the centroid, and i represents an integer.

[0134] Construct the Hessian matrix H based on the above position information after removing the center of mass:

[0135]

[0136] Perform SVD decomposition on the Hessian matrix to obtain the first pose of the carrier.

[0137] Further, S5 includes:

[0138] Get the carrier speed of the first pose at the current moment and the carrier speed at the previous moment;

[0139] Obtaining a compensation displacement according to the carrier speed at the current moment and the carrier speed at the previous moment;

[0140] Compensating the compensation displacement to the blank pixel points of the millimeter wave point cloud image after the dynamic object is removed, so as to obtain a compensated millimeter wave point cloud image;

[0141] The pose of the compensated millimeter wave point cloud image is constructed to obtain a second pose;

[0142] Get the error value between the first pose and the second pose;

[0143] The first pose and the second pose are corrected according to the error value to obtain the target pose.

[0144] In the above implementation process, after removing and compensating the dynamic objects, a secondary pose construction is performed, and then the errors of the two pose constructions are corrected to obtain the target pose, which can improve the map accuracy and make the pose construction simpler and more efficient.

[0145] The carrier velocity at the current moment and the carrier velocity at the previous moment are calculated from the carrier's first position. , and then by the time interval , calculate the compensation displacement , the compensation displacement is applied to each pixel of the moving object, and then the second pose is calculated by the ICP algorithm .

[0146] The adjacent posture is corrected by the error value between the first posture and the second posture. The specific formula is shown as follows:

[0147] ;

[0148] in, For the first position, is the second posture, W represents the number of postures that need to be corrected in the correction window (ie, the error value in the embodiment of the present application), and if W=3, it indicates that the current posture needs to be corrected using the previous three postures.

[0149] The current posture correction formula is as follows:

[0150] ;

[0151] ;

[0152] in, represents the corrected target pose, represents the i-th first pose before the current frame, represents the i-th second pose before the current frame;

[0153] Finally, the corrected target pose is output.

[0154] The embodiment of the present application adopts a dual radar combination of laser radar and millimeter wave radar to solve the interference problem of rainy and foggy weather and dynamic objects on posture construction. In order to achieve the matching of laser radar and millimeter wave radar information, the point cloud images generated by both are normalized, which can improve the matching accuracy and lay a good foundation for improving the positioning accuracy in the later stage.

[0155] A dual-range point cloud map is generated from the laser point cloud map and the millimeter-wave point cloud map, and then the dual-range point cloud map is used for complex weather detection. The difference in rainy and foggy weather distance detection between the laser radar and millimeter-wave radar is used to judge rainy and foggy weather. The calculation is simple and convenient, and the detection efficiency is high.

[0156] The pixels of dynamic objects in the millimeter-wave point cloud map are removed and replaced with blank pixels. At the same time, the carrier speed is calculated based on the carrier's primary posture, and motion compensation is performed on the dynamic objects. The carrier's secondary posture is calculated, and a correction window is set. The current posture is corrected using the previous primary and secondary postures. Local correction can improve positioning accuracy and real-time performance.

[0157] Embodiment 2

[0158] In order to execute the method corresponding to the above embodiment 1 to achieve the corresponding functions and technical effects, a dual radar posture construction device is provided below, such as Figure 2 As shown, the device comprises:

[0159] An acquisition module 1 is used to acquire first radar data of the laser radar and second radar data of the millimeter wave radar;

[0160] A preprocessing module 2, used to preprocess the first radar data and the second radar data respectively to obtain a laser point cloud image and a millimeter wave point cloud image;

[0161] A generating module 3 is used to generate a dual-range point cloud image according to the laser point cloud image and the millimeter wave point cloud image;

[0162] A posture construction module 4 is used to construct a posture according to the double-distance point cloud image to obtain a first posture;

[0163] The motion compensation module 5 is used to perform motion compensation on the first pose to obtain the target pose.

[0164] In the above implementation process, the posture construction is carried out through the interaction of dual radars, and the moving objects are detected and compensated, which can improve the accuracy and efficiency of posture construction, realize rapid detection in complex weather and complex environments, improve the information interaction between multiple radars, and enable different radars to complement each other's advantages and reduce local errors.

[0165] Furthermore, the generating module 3 is also used for:

[0166] Normalizing the laser point cloud image and the millimeter wave point cloud image respectively to obtain the laser point cloud image and the millimeter wave point cloud image with the same pixel format;

[0167] The pixel points between the laser point cloud image and the millimeter wave point cloud image with the same pixel format are correlated to obtain a dual-range point cloud image.

[0168] In the above implementation process, the laser point cloud map and the millimeter wave point cloud map are normalized and then correlated with each other, which can improve the matching degree between the laser point cloud map and the millimeter wave point cloud map, and provide a data basis for improving the accuracy of posture construction.

[0169] Furthermore, the generating module 3 is also used for:

[0170] The pixels of the laser point cloud image and the millimeter wave point cloud image with the same pixel format are matched one by one to obtain the initial dual-range point cloud image;

[0171] Traverse each pixel in the initial double-distance point cloud image;

[0172] If the pixel point is a pixel point in both the laser point cloud image and the millimeter wave point cloud image, the pixel point is marked as the first flag position. If the pixel point only belongs to a pixel point in the laser point cloud image or the millimeter wave point cloud image, the pixel point is marked as the second flag position to obtain a dual-distance point cloud image.

[0173] In the above implementation process, the pixels of the laser point cloud image and the millimeter wave point cloud image are matched and the pixels that exist at the same time are marked to facilitate the subsequent detection of dynamic objects and improve the data processing efficiency.

[0174] Furthermore, the posture construction module 4 is also used for:

[0175] Perform complex weather detection on the dual-range point cloud image to obtain complex weather estimation;

[0176] Determine whether the complex weather estimate is a preset threshold;

[0177] If so, construct the pose of the millimeter wave point cloud image and obtain the first pose.

[0178] In the above implementation process, complex weather detection is performed on the dual-range point cloud image to obtain complex weather estimation, and then pose construction is performed. The impact of complex weather on accuracy can be taken into account at the beginning of pose construction, thereby improving the accuracy of subsequent pose construction.

[0179] Furthermore, the posture construction module 4 is also used for:

[0180] Performing dynamic object removal on the millimeter wave point cloud image to obtain a millimeter wave point cloud image after the dynamic object removal;

[0181] The pose of the millimeter-wave point cloud image after the dynamic objects are removed is constructed to obtain the first pose.

[0182] In the above implementation process, dynamic objects are removed from the millimeter wave point cloud image before posture construction, which can solve the influence of dynamic objects in the positioning process and improve the positioning accuracy.

[0183] Furthermore, the posture construction module 4 is also used for:

[0184] Obtain the speed of the pixel points of the moving object in the millimeter wave point cloud image;

[0185] Determine whether the speed is greater than zero;

[0186] If yes, determine that the pixel is a moving pixel, and mark the moving pixel as a motion flag;

[0187] All motion flags in the millimeter-wave point cloud image are replaced with blank pixels to obtain the millimeter-wave point cloud image after dynamic objects are removed.

[0188] In the above implementation process, the motion flag is replaced by a blank pixel, so that the position of the dynamic object can be quickly and accurately obtained, and it can be reasonably replaced to reduce the positioning error caused by the dynamic object.

[0189] Furthermore, the posture construction module 4 is also used for:

[0190] Get the current frame pixel point set and the previous frame pixel point set of the millimeter wave point cloud image;

[0191] The pixel point set of the current frame is matched with the pixel point set of the previous frame to obtain the first pose.

[0192] In the above implementation process, the millimeter wave point cloud map is matched between frames to obtain the first pose, which can improve the real-time nature of map construction, increase the construction speed, and avoid excessive inter-frame errors that affect accuracy.

[0193] Furthermore, the motion compensation module 5 is also used for:

[0194] Get the carrier speed of the first pose at the current moment and the carrier speed at the previous moment;

[0195] Obtaining a compensation displacement according to the carrier speed at the current moment and the carrier speed at the previous moment;

[0196] Compensating the compensation displacement to the blank pixel points of the millimeter wave point cloud image after the dynamic object is removed, so as to obtain a compensated millimeter wave point cloud image;

[0197] The pose of the compensated millimeter wave point cloud image is constructed to obtain a second pose;

[0198] Get the error value between the first pose and the second pose;

[0199] The first pose and the second pose are corrected according to the error value to obtain the target pose.

[0200] In the above implementation process, after removing and compensating the dynamic objects, a secondary pose construction is performed, and then the errors of the two pose constructions are corrected to obtain the target pose, which can improve the map accuracy and make the pose construction simpler and more efficient.

[0201] The above dual radar posture construction device can implement the method of the above embodiment 1. The options in the above embodiment 1 are also applicable to this embodiment and will not be described in detail here.

[0202] The rest of the contents of the embodiments of the present application can refer to the contents of the above-mentioned embodiment 1, and will not be repeated in this embodiment.

[0203] Embodiment 3

[0204] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the dual radar posture construction method of embodiment one.

[0205] Optionally, the above-mentioned electronic device may be a server.

[0206] See also Figure 3 , Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include a processor 31, a communication interface 32, a memory 33 and at least one communication bus 34. The communication bus 34 is used to realize direct connection and communication between these components. The communication interface 32 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The processor 31 may be an integrated circuit chip with signal processing capabilities.

[0207] The processor 31 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor 31 can also be any conventional processor, etc.

[0208] The memory 33 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory 33 stores computer readable instructions. When the computer readable instructions are executed by the processor 31, the device can execute the above-mentioned Figure 1 The method embodiment involves various steps.

[0209] Optionally, the electronic device may further include a storage controller and an input / output unit. The memory 33, the storage controller, the processor 31, the peripheral interface, and the input / output unit are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components may be electrically connected to each other via a communication bus 34. The processor 31 is used to execute executable modules stored in the memory 33, such as software function modules or computer programs included in the device.

[0210] The input and output unit is used to provide users with the task creation and to create a start optional time period or preset execution time for the task to realize the interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse and a keyboard.

[0211] Understandably, Figure 3 The structure shown is for illustration only. The electronic device may also include Figure 3 More or fewer components as shown, or with Figure 3 Different configurations are shown. Figure 3 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0212] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the dual radar posture construction method of embodiment one is implemented.

[0213] The embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the method described in the method embodiment.

[0214] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart, can be implemented with a dedicated hardware-based device that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0215] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0216] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0217] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0218] The above is only a specific implementation of the present application, but the protection range of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical range disclosed in the present application, which should be included in the protection range of the present application. Therefore, the protection range of the present application should be based on the protection range of the claims.

[0219] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A dual radar pose construction method, characterized in that: The method comprises: Acquire first radar data of the laser radar and second radar data of the millimeter wave radar; Preprocessing the first radar data and the second radar data respectively to obtain a laser point cloud image and a millimeter wave point cloud image; Generate a dual-range point cloud image according to the laser point cloud image and the millimeter wave point cloud image; Perform pose construction according to the dual-distance point cloud image to obtain the first pose; Performing motion compensation on the first posture to obtain a target posture; The step of generating a dual-range point cloud image according to the laser point cloud image and the millimeter wave point cloud image comprises: Normalizing the laser point cloud image and the millimeter wave point cloud image respectively to obtain a laser point cloud image and a millimeter wave point cloud image with consistent pixel formats; Correlating the pixel points of the laser point cloud image and the millimeter wave point cloud image with the same pixel format to obtain the dual-range point cloud image; The step of correlating the pixel points between the laser point cloud image and the millimeter wave point cloud image with the same pixel format to obtain the dual-range point cloud image comprises: The pixel points between the laser point cloud image and the millimeter wave point cloud image with the same pixel format are matched one by one to obtain an initial dual-range point cloud image; Traversing each pixel point in the initial dual-distance point cloud image; If a pixel point is a pixel point in both the laser point cloud image and the millimeter wave point cloud image, the pixel point is marked as a first flag position; if a pixel point only belongs to a pixel point in the laser point cloud image or the millimeter wave point cloud image, the pixel point is marked as a second flag position to obtain the dual-distance point cloud image; The step of constructing a pose according to the dual-distance point cloud image to obtain the first pose comprises: Performing complex weather detection on the dual-range point cloud image to obtain complex weather estimation; Determining whether the complex weather estimate is a preset threshold; If yes, construct the pose of the millimeter wave point cloud image to obtain the first pose; If a pixel is located in both the laser point cloud map and the millimeter wave point cloud map, the pixel is marked. The specific formula is as follows: ； Among them, p represents the pixel point in the double distance point cloud image, and represents the millimeter wave point cloud image, Indicates the laser point cloud image, FLAG indicates the flag bit. If the pixel belongs to both the millimeter wave point cloud image and the laser point cloud image, the flag bit is set to 1, otherwise it is set to 0. A pixel with a flag bit of 1 indicates that the pixel has dual distances. The complex weather estimate is obtained based on the sum of the distance differences of the double distances of the pixels whose flag bit is 1, and the rain and fog threshold is initialized. , the calculation formula for complex weather estimation is as follows: ； in, It represents the sum of the distance differences between the double distances of the pixels whose flag bit is 1, and F is the complex weather estimation.

2. The dual radar pose construction method according to claim 1, characterized in that: The step of constructing the pose of the millimeter wave point cloud image to obtain the first pose includes: Performing dynamic object removal on the millimeter wave point cloud image to obtain a millimeter wave point cloud image after the dynamic object removal; The millimeter-wave point cloud image after the dynamic object is removed is subjected to posture construction to obtain the first posture.

3. The dual radar pose construction method according to claim 2, characterized in that: The step of removing dynamic objects from the millimeter wave point cloud image to obtain the millimeter wave point cloud image after the dynamic objects are removed includes: Obtaining the speed of the pixel point of the moving object in the millimeter wave point cloud image; Determining whether the speed is greater than zero; If yes, determine that the pixel is a moving pixel, and mark the moving pixel as a motion flag; All motion flags in the millimeter wave point cloud image are replaced with blank pixels to obtain the millimeter wave point cloud image after the dynamic objects are removed.

4. The dual radar pose construction method according to claim 3, characterized in that: The step of constructing the pose of the millimeter wave point cloud image after the dynamic object is removed to obtain the first pose includes: Obtaining a current frame pixel point set and a previous frame pixel point set of the millimeter wave point cloud image; Inter-frame matching is performed on the current frame pixel point set and the previous frame pixel point set to obtain the first posture.

5. The dual radar pose construction method according to claim 1, characterized in that: The step of performing motion compensation on the first posture to obtain a target posture comprises: Obtaining the carrier velocity of the first posture at the current moment and the carrier velocity at the previous moment; Obtaining a compensation displacement according to the carrier speed at the current moment and the carrier speed at the previous moment; Compensating the compensation displacement to the blank pixel points of the millimeter wave point cloud image after the dynamic object is removed, to obtain a compensated millimeter wave point cloud image; Performing pose construction on the compensated millimeter wave point cloud image to obtain a second pose; Obtaining an error value between the first posture and the second posture; The first posture and the second posture are corrected according to the error value to obtain the target posture.

6. A dual radar posture construction device, characterized in that: The device comprises: An acquisition module, used to acquire first radar data of the laser radar and second radar data of the millimeter wave radar; A preprocessing module, used to preprocess the first radar data and the second radar data respectively to obtain a laser point cloud image and a millimeter wave point cloud image; A generating module, used for generating a dual-range point cloud image according to the laser point cloud image and the millimeter wave point cloud image; A posture construction module, used to construct a posture according to the dual-distance point cloud image to obtain a first posture; A motion compensation module, used for performing motion compensation on the first posture to obtain a target posture; The generation module is also used for: Normalizing the laser point cloud image and the millimeter wave point cloud image respectively to obtain a laser point cloud image and a millimeter wave point cloud image with consistent pixel formats; Correlating the pixel points of the laser point cloud image and the millimeter wave point cloud image with the same pixel format to obtain the dual-range point cloud image; The pixel points between the laser point cloud image and the millimeter wave point cloud image with the same pixel format are matched one by one to obtain an initial dual-range point cloud image; Traversing each pixel point in the initial dual-distance point cloud image; If a pixel point is a pixel point in both the laser point cloud image and the millimeter wave point cloud image, the pixel point is marked as a first flag position; if a pixel point only belongs to a pixel point in the laser point cloud image or the millimeter wave point cloud image, the pixel point is marked as a second flag position to obtain the dual-distance point cloud image; The pose construction module is also used for: Performing complex weather detection on the dual-range point cloud image to obtain complex weather estimation; Determining whether the complex weather estimate is a preset threshold; If yes, construct the pose of the millimeter wave point cloud image to obtain the first pose; If a pixel is located in both the laser point cloud map and the millimeter wave point cloud map, the pixel is marked. The specific formula is as follows: ； Among them, p represents the pixel point in the double distance point cloud image, and represents the millimeter wave point cloud image, Indicates the laser point cloud image, FLAG indicates the flag bit. If the pixel belongs to both the millimeter wave point cloud image and the laser point cloud image, the flag bit is set to 1, otherwise it is set to 0. A pixel with a flag bit of 1 indicates that the pixel has dual distances. The complex weather estimate is obtained based on the sum of the distance differences of the double distances of the pixels whose flag bit is 1, and the rain and fog threshold is initialized. , the calculation formula for complex weather estimation is as follows: ； in, It represents the sum of the distance differences between the double distances of the pixels whose flag bit is 1, and F is the complex weather estimation.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the dual radar pose construction method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the dual radar pose construction method as described in any one of claims 1 to 5.

Images