Laser radar external parameter calibration method and device, storage medium, program product and computer equipment

Abstract

The invention discloses a laser radar external parameter calibration method and device, a storage medium, a program product and computer equipment, and the method comprises the steps: obtaining the point cloud data of a to-be-calibrated laser radar, and determining a scanning scene corresponding to the point cloud data; determining a point cloud intensity value of each point in the scanning scene according to the point cloud data; determining a dynamic parameter based on the point cloud data and the point cloud intensity value under the condition of determining that an object meeting a preset condition exists in the scanning scene; and based on the dynamic parameters and the point cloud intensity value, adjusting the point cloud data by using a normal distribution model to obtain first target point cloud data, and performing external parameter calibration on the to-be-calibrated laser radar according to the first target point cloud data, thereby adjusting and improving the display effect of the point cloud data during visualization, and improving the accuracy of the calibration of the to-be-calibrated laser radar. Therefore, the external parameter calibration effect is improved.

Description

Technical Field

[0001] This application relates to the technical field of lidar, and in particular, to a lidar extrinsic parameter calibration method, device, storage medium, program product, and computer device. Background Art

[0002] During the extrinsic parameter calibration process, selecting corresponding points and / or corner points is the key to ensuring accurate alignment of data from different sensors (such as lidar and camera). Taking the point cloud data set generated by the lidar and the image data set captured by the camera as an example, it is usually necessary to first select a sufficient number of and reasonably distributed corresponding points and / or corner points in these two data sets, and then estimate the conversion relationship between the coordinate systems of the lidar and the camera based on this, so as to complete the alignment work between the coordinate systems and thus complete the extrinsic parameter calibration.

[0003] In related technologies, usually the point cloud data is first visualized, and then the corresponding points and / or corner points are selected from the visualized point cloud using a calibration tool. However, the visualization display effect of the point cloud data is easily interfered by the objects in the corresponding scene, thereby affecting the effect of extrinsic parameter calibration. Summary of the Invention

[0004] To solve the above technical problems, embodiments of this application propose a lidar extrinsic parameter calibration method, device, storage medium, program product, and computer device, which can improve the visualization display effect of point cloud data to enhance the effect of extrinsic parameter calibration.

[0005] In a first aspect, embodiments of this application provide a lidar extrinsic parameter calibration method, including:

[0006] Obtain the point cloud data of the lidar to be calibrated, and determine the scanning scene corresponding to the point cloud data;

[0007] Based on the point cloud data, determine the point cloud intensity value of each point in the scanning scene;

[0008] When it is determined that there is an object in the scanning scene that meets the preset conditions, based on the point cloud data and the point cloud intensity value, determine the dynamic parameters; and,

[0009] Based on the dynamic parameters and the point cloud intensity value, use the normal distribution model to adjust the point cloud data to obtain the first target point cloud data, and perform extrinsic parameter calibration on the lidar to be calibrated according to the first target point cloud data.

[0010] Optionally, the step of using the normal distribution model to adjust the point cloud data based on the dynamic parameters and the point cloud intensity value to obtain the first target point cloud data includes:

[0011] Determine a dynamic threshold interval using a normal distribution model based on the dynamic parameter and the point cloud intensity value;

[0012] Adjust the point cloud data based on the dynamic threshold interval and the point cloud intensity value to obtain first target point cloud data.

[0013] Optionally, the determining a dynamic threshold interval using a normal distribution model based on the dynamic parameter and the point cloud intensity value includes:

[0014] Determine the standard deviation and mean of the point cloud intensity value;

[0015] Calculate the product of the dynamic parameter and the standard deviation of the point cloud intensity value, and determine the dynamic threshold interval using a normal distribution model based on the product and the mean of the point cloud intensity value.

[0016] Optionally, the adjusting the point cloud data based on the dynamic threshold interval and the point cloud intensity value to obtain first target point cloud data includes:

[0017] Traverse each point among the respective points. If the point cloud intensity value of the point is greater than the upper limit value of the dynamic threshold interval, adjust the point cloud intensity value of the point to the upper limit value of the dynamic threshold interval in the point cloud data. If the point cloud intensity value of the point is less than the lower limit value of the dynamic threshold interval, adjust the point cloud intensity value of the point to the lower limit value of the dynamic threshold interval in the point cloud data;

[0018] After traversing the respective points, use the adjusted point cloud data as the first target point cloud data.

[0019] Optionally, the determining a dynamic parameter based on the point cloud data and the point cloud intensity value includes:

[0020] Determine the mean of the marker point cloud intensity of a specific marker in the scanning scene based at least on the point cloud data;

[0021] Determine a dynamic parameter based on the mean of the marker point cloud intensity and the point cloud intensity value.

[0022] Optionally, the determining a dynamic parameter based on the mean of the marker point cloud intensity and the point cloud intensity value includes:

[0023] In the respective points, determine the proportion of points whose point cloud intensity value is less than the mean of the marker point cloud intensity;

[0024] Obtain a dynamic parameter matching the determined proportion based on a preset mapping relationship between the proportion and the parameter.

[0025] Optionally, the specific marker includes a ground marker. Based on at least the point cloud data, determining the average marker point cloud intensity of the specific marker within the scanned scene includes:

[0026] Extracting the ground point cloud data corresponding to the ground plane from the point cloud data;

[0027] Obtaining an image of the scanned scene and detecting a detection box corresponding to the ground marker in the image;

[0028] Based on the detection box and the ground point cloud data, determining the average marker point cloud intensity of the ground marker.

[0029] Optionally, based on the detection box and the ground point cloud data, determining the average marker point cloud intensity of the ground marker includes:

[0030] Projecting the detection box onto the ground point cloud data to obtain a projection point range;

[0031] Using the point cloud intensity value as a clustering basis, clustering the point cloud within the projection point range in the ground point cloud data to obtain a point cloud clustering cluster;

[0032] Based on the point cloud clustering cluster, determining the average marker point cloud intensity of the ground marker.

[0033] Optionally, the preset condition includes that the reflectivity of an object is greater than a reflectivity threshold;

[0034] In the case where it is determined that there is no object with a reflectivity greater than the reflectivity threshold within the scanned scene, the method further includes:

[0035] Adjusting the point cloud data according to the point cloud intensity value to obtain second target point cloud data;

[0036] Performing external parameter calibration on the lidar to be calibrated according to the second target point cloud data.

[0037] Optionally, adjusting the point cloud data according to the point cloud intensity value to obtain second target point cloud data includes:

[0038] Normalizing the point cloud intensity value;

[0039] Adjusting the point cloud data according to the normalized point cloud intensity value to obtain second target point cloud data, where when displaying the second target point cloud data, the color of each point is determined by the magnitude of the normalized point cloud intensity value.

[0040] In a second aspect, an embodiment of the present application provides a lidar external parameter calibration device, including:

[0041] A data acquisition module, configured to acquire point cloud data of a lidar to be calibrated and determine a scanning scene corresponding to the point cloud data;

[0042] A point cloud intensity value determination module, configured to determine point cloud intensity values of each point in the scanning scene according to the point cloud data;

[0043] A first calibration module, configured to:

[0044] When it is determined that there is an object satisfying a preset condition in the scanning scene, determine dynamic parameters based on the point cloud data and the point cloud intensity values; and

[0045] Based on the dynamic parameters and the point cloud intensity values, adjust the point cloud data by using a normal distribution model to obtain first target point cloud data, and perform external parameter calibration on the lidar to be calibrated according to the first target point cloud data.

[0046] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in any one of the above are implemented.

[0047] In a fourth aspect, an embodiment of the present application provides a computer program product, including computer instructions, and when the computer instructions are executed by a processor, the steps of the method described in any one of the above are implemented.

[0048] In a fifth aspect, an embodiment of the present application provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the steps of the method described in any one of the above are implemented.

[0049] In summary, the embodiments of the present application have at least the following beneficial effects:

[0050] By adopting the embodiments of the present application, by acquiring the point cloud data of the lidar to be calibrated and determining the scanning scene corresponding to the point cloud data; determining the point cloud intensity values of each point in the scanning scene according to the point cloud data; when it is determined that there is an object satisfying a preset condition in the scanning scene, determining dynamic parameters based on the point cloud data and the point cloud intensity values; and based on the dynamic parameters and the point cloud intensity values, adjusting the point cloud data by using a normal distribution model to obtain first target point cloud data, and performing external parameter calibration on the lidar to be calibrated according to the first target point cloud data, the display effect of the point cloud data during visualization can be adjusted and improved, so as to improve the effect of external parameter calibration. Description of the Drawings

[0051] Figure 1 It is a schematic flowchart of the method for calibrating the external parameters of a lidar provided by an embodiment of the present application;

[0052] Figure 2 It is a schematic diagram of the display effect of the second target point cloud data provided by an embodiment of the present application;

[0053] Figure 3 It is another schematic diagram of the display effect of the second target point cloud data provided by an embodiment of the present application;

[0054] Figure 4 It is a schematic diagram of the display effect of the first target point cloud data provided by an embodiment of the present application;

[0055] Figure 5 It is a schematic structural diagram of the lidar external parameter calibration device provided by an embodiment of the present application;

[0056] Figure 6 It is a schematic structural diagram of the computer device provided by an embodiment of the present application. Detailed implementation manners

[0057] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0058] In the description of the present application, the terms "first", "second", "third", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", "third", etc. may explicitly or implicitly include one or more of such features. In the description of the present application, unless otherwise specified, the meaning of "a plurality" is two or more. In the description of the present application, the term "including" and its variants are open-ended inclusions, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "according to" is "at least partially according to". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments".

[0059] In the description of this application, it should be noted that unless otherwise clearly specified and defined, the terms "installation", "connection", and "linkage" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0060] In the description of this application, it should be noted that unless otherwise defined, all technical and scientific terms used in this application have the same meanings as those commonly understood by those skilled in the technical field to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0061] In the first aspect, referring to Figure 1 , a flowchart of a method for calibrating the external parameters of a lidar provided by an embodiment of this application is shown. The method includes steps S101 - S103, which are specifically as follows:

[0062] S101, obtain the point cloud data of the lidar to be calibrated, and determine the scanning scene corresponding to the point cloud data;

[0063] In one example, the lidar to be calibrated can be a roadside lidar. For example, it can be set in an intersection area, and the intersection area can include crossroads, etc.

[0064] In one example, the point cloud data of the lidar to be calibrated can be obtained by scanning the scanning scene with the lidar to be calibrated.

[0065] S102, according to the point cloud data, determine the point cloud intensity values of each point in the scanning scene;

[0066] It should be noted that the point cloud intensity value in this embodiment can refer to the intensity of the signal returned by each point (such as the coordinate point in the three - dimensional world) in the scanning scene during the lidar scanning process. This intensity can be related to factors such as the reflectivity, distance, and incident angle of the target surface existing at this point.

[0067] In one example, the point cloud data scanned by the lidar to be calibrated can contain the intensity values of each point. Thus, the point cloud intensity values of each point in the scanning scene can be directly obtained by reading the corresponding file of the point cloud data.

[0068] In one example, a general method for calculating point cloud intensity values can be adopted to calculate the point cloud data to obtain the point cloud intensity values of each point in the scanning scene. For example, the echo signals in the point cloud data can be parsed, and the intensity value of each point can be determined based on the parsed signal intensity.

[0069] S103. When it is determined that there is an object in the scanning scene that meets the preset conditions, based on the point cloud data and the point cloud intensity values, determine the dynamic parameters; and, based on the dynamic parameters and the point cloud intensity values, use the normal distribution model to adjust the point cloud data to obtain the first target point cloud data, and perform external parameter calibration on the lidar to be calibrated according to the first target point cloud data.

[0070] In this embodiment, corresponding dynamic parameters will be obtained for the acquired point cloud data, so as to realize the dynamic adjustment of the point cloud data, so that the adjusted point cloud data can better adapt to the actual situation in the scanning scene, and can adaptively reduce the interference of the objects in the scanning scene on the display effect of the point cloud data, thereby being able to adaptively improve the visualization display effect of the point cloud data to improve the effect of external parameter calibration. In addition, since the normal distribution model is used, this embodiment can be adaptively applied to different range intensity values of different models of lidars to be calibrated.

[0071] In one example, the above-mentioned object that meets the preset conditions may include an interference object that can form a strong echo.

[0072] In one example, the above-mentioned object that meets the preset conditions may include an object that causes specific interference to the visualization display effect of the point cloud data, where the specific interference may include interference caused by at least one of the following factors: the reflectivity of the object surface, the distance of the object relative to the lidar to be calibrated (for example, the distance is too small), the incident angle of the object relative to the lidar to be calibrated, etc.

[0073] In one example, the visualization display effect of the point cloud data can be determined by the point cloud intensity values of each point in the scanning scene (for example, setting corresponding colors for each point according to the point cloud intensity value of each point and displaying them). Thus, the interference described in the above example may refer to the above factors affecting the point cloud intensity values of each point in the scanning scene determined by the point cloud data, thereby affecting the visualization display effect of the point cloud data.

[0074] In one example, determining the dynamic parameters based on the point cloud data and the point cloud intensity values may include: inputting the point cloud data and the point cloud intensity values into a pre-trained dynamic parameter model to obtain the dynamic parameters output by the dynamic parameter model. Here, it is only necessary to pre-train the dynamic parameter model so that the dynamic parameter model has the ability to take the point cloud data and the point cloud intensity values as inputs and the dynamic parameters as outputs. Specifically, historical point cloud data and their corresponding historical point cloud intensity values may be used as sample data, and a general training method may be used to train the artificial intelligence model to obtain the dynamic parameter model.

[0075] In one example, since the normal distribution model is determined according to the standard deviation and the mean, it is possible to consider using the dynamic parameters to determine the standard deviation and / or the mean (for example, using the dynamic parameter as a coefficient and multiplying it by a preset standard deviation to determine the standard deviation, and / or using the dynamic parameter as a coefficient and multiplying it by a preset mean to determine the mean), so as to use the dynamic parameters to determine the normal distribution model. In this way, based on the dynamic parameters and the point cloud intensity values, adjusting the point cloud data using the normal distribution model to obtain the first target point cloud data may include: according to the point cloud intensity values, using the probability density function corresponding to the normal distribution model determined by the dynamic parameters to calculate the probability density of each point, and using this probability density as a weight to adjust the point cloud data to obtain the first target point cloud data. Here, the implementation manner of this adjustment may also be diverse. For example, directly scaling the information corresponding to each point by the weight, or only retaining the points with a probability density higher than the density threshold to reduce noise, so that the first target point cloud data obtained after adjustment has a better display effect when displayed.

[0076] In one example, performing external parameter calibration on the lidar to be calibrated according to the first target point cloud data may include: performing visualization processing on the first target point cloud data to display the first target point cloud data, and selecting corresponding points and / or corner points in the displayed first target point cloud data, and performing external parameter calibration on the lidar to be calibrated according to the selected corresponding points and / or corner points. Among them, the corresponding points and / or corner points can be manually and / or automatically selected using a calibration tool.

[0077] In an alternative embodiment, the adjusting the point cloud data using the normal distribution model based on the dynamic parameters and the point cloud intensity values to obtain the first target point cloud data includes:

[0078] Determining a dynamic threshold interval based on the dynamic parameters and the point cloud intensity values;

[0079] Adjusting the point cloud data based on the dynamic threshold interval and the point cloud intensity values to obtain the first target point cloud data.

[0080] In some cases, in addition to the interference of objects in the scanning scene, the point cloud data usually also contains a large amount of noise, which may come from measurement errors of sensors, environmental factors (such as weather conditions), etc. In this embodiment, the determined dynamic threshold interval can be used to effectively identify and filter out as much as possible the interference of objects and the above-mentioned noise, so as to improve the display effect of the first target point cloud data. For example, regions with typical reflection characteristics, such as road surfaces, building walls, etc., can be highlighted.

[0081] In one example, as described above, since the normal distribution model is determined according to the standard deviation and the mean, it can be considered that one of the standard deviation and the mean is determined by a dynamic parameter and the other is determined by the point cloud intensity value, or alternatively, one of the standard deviation and the mean is determined by the dynamic parameter and the point cloud intensity value and the other is pre-configured, so as to determine the normal distribution model by using the dynamic parameter and the point cloud intensity value, and use a certain confidence interval corresponding to the determined normal distribution model as the dynamic threshold interval. Among them, the specific way to determine the standard deviation and / or the mean in this embodiment can be to scale the preset standard deviation and / or the preset mean by using the basis for determination (dynamic parameter and / or point cloud intensity value) as a weight coefficient, or to calculate it as the input of some designed functional expressions, which is not strictly limited here.

[0082] In one example, based on the dynamic threshold interval and the point cloud intensity value, adjusting the point cloud data to obtain the first target point cloud data may include: for the points in the point cloud data whose point cloud intensity values are not within the dynamic threshold interval, performing adjustment processing, and taking the adjusted point cloud data as the first target point cloud data, where the adjustment processing may include: performing scaling processing on the points whose point cloud intensity values are not within the dynamic threshold interval, and the scaling processing may include scaling by a set ratio and / or making the point cloud intensity value of the scaled point within the dynamic threshold interval, which is not strictly limited here.

[0083] In an alternative embodiment, the determining the dynamic threshold interval by using the normal distribution model based on the dynamic parameter and the point cloud intensity value includes:

[0084] Determining the standard deviation and the mean of the point cloud intensity value; exemplarily, the standard deviation and the mean of the point cloud intensity values of each point can be directly calculated.

[0085] Calculating the product of the dynamic parameter and the standard deviation of the point cloud intensity value, and determining the dynamic threshold interval by using the normal distribution model based on the product and the mean of the point cloud intensity value.

[0086] In this embodiment, the threshold interval dynamically calculated based on statistics (mean and standard deviation) in combination with dynamic parameters can automatically adapt to the actual scanning scenario without manual parameter adjustment, improving the generality and efficiency of the processing flow.

[0087] In one example, the calculation of the upper limit value of the dynamic threshold interval may include the following formula: int_mean + x * int_std.

[0088] In one example, the calculation of the lower limit value of the dynamic threshold interval may include the following formula: int_mean - x * int_std.

[0089] Wherein, int_mean is the mean of the point cloud intensity values, x is the dynamic parameter, and int_std is the standard deviation of the point cloud intensity values.

[0090] In an alternative embodiment, adjusting the point cloud data based on the dynamic threshold interval and the point cloud intensity value to obtain first target point cloud data includes:

[0091] Traverse each point among the respective points. If the point cloud intensity value of the point is greater than the upper limit value of the dynamic threshold interval, then adjust the point cloud intensity value of the point to the upper limit value of the dynamic threshold interval in the point cloud data. If the point cloud intensity value of the point is less than the lower limit value of the dynamic threshold interval, then adjust the point cloud intensity value of the point to the lower limit value of the dynamic threshold interval in the point cloud data;

[0092] After traversing the respective points, use the adjusted point cloud data as the first target point cloud data.

[0093] In this embodiment, the point cloud intensity values of each point in the first target point cloud data can be mapped into the dynamic threshold interval, so as to avoid the occurrence of points with too large or too small point cloud intensity values in the point cloud data to form noise interference while retaining (without deleting) the information of each point, thereby improving the display effect of the first target point cloud data. In addition, it should be understood that this embodiment uses the normal distribution model to adjust the overall point cloud intensity value of the point cloud data at one time (that is, traverse once), so as to increase the visual display of specific markers in the point cloud rather than process in blocks. The overall solution is more concise, has good applicability, can be parallel computed, has a fast computing speed, and the application direction is that it can make the selection of the same-name calibration points more accurate, and it is convenient to select the corner points in some cases where only the corner points of the markers can be used and the conventional color attachment method cannot select the corner points.

[0094] In an alternative embodiment, determining the dynamic parameter based on the point cloud data and the point cloud intensity value includes:

[0095] Determine the mean value of the marker point cloud intensity of a specific marker within the scanning scene based at least on the point cloud data;

[0096] Determine a dynamic parameter based on the mean value of the marker point cloud intensity and the point cloud intensity value.

[0097] In this embodiment, since the dynamic parameter is determined by the mean value of the marker point cloud intensity of a specific marker, the determined dynamic parameter can be particularly associated with the influence of the specific marker, such that the display effect of the finally adjusted first target point cloud data can particularly highlight the specific marker, so as to facilitate the selection of homologous points and / or corner points based on this as a reference, thereby improving the external parameter calibration effect.

[0098] It should be noted that the number of specific markers in this embodiment can be one or more, and the specific marker can be some markers that need to be prominently displayed set by the user. For example, traffic signs (ground traffic marking lines) painted on the ground.

[0099] In one example, determining the mean value of the marker point cloud intensity of a specific marker within the scanning scene based at least on the point cloud data may include: identifying the point cloud that matches the specific marker in the point cloud data, determining the intensity values of the identified point cloud, and calculating the mean value of the intensity values as the mean value of the marker point cloud intensity.

[0100] In one example, determining a dynamic parameter based on the mean value of the marker point cloud intensity and the point cloud intensity value may include: inputting the mean value of the marker point cloud intensity and the point cloud intensity value into a pre-trained dynamic parameter model to obtain the dynamic parameter output by the dynamic parameter model. Here, it is only necessary to pre-train the dynamic parameter model so that the dynamic parameter model has the ability to take the mean value of the marker point cloud intensity and the point cloud intensity value as inputs and the dynamic parameter as the output. Specifically, the historical mean value of the marker point cloud intensity and its corresponding historical point cloud intensity value can be used as sample data, and the artificial intelligence model can be trained using a general training method to obtain the dynamic parameter model. Among them, the historical mean value of the marker point cloud intensity can be determined from the data in the historical point cloud data that matches the specific marker.

[0101] In an alternative embodiment, the determining a dynamic parameter based on the mean value of the marker point cloud intensity and the point cloud intensity value includes:

[0102] Determine the proportion of points in which the point cloud intensity value is less than the mean value of the marker point cloud intensity among the respective points;

[0103] Obtain a dynamic parameter that matches the determined proportion based on a preset mapping relationship between the proportion and the parameter.

[0104] In some cases, different scanning scenarios usually have different reflection characteristics (for example, the influence caused by the number and / or position of objects that meet the above preset conditions), which will lead to significant differences in the intensity distribution in the point cloud data. In this embodiment, by calculating the proportion of points below the mean value and using this proportion to adjust the dynamic parameters, the above differences can be adaptively processed, so that even in the case of uneven or large changes in the intensity distribution, a suitable threshold interval can be found, thus ensuring the robustness and adaptability of the algorithm. Moreover, by presetting the mapping relationship, the corresponding dynamic parameters can also be efficiently obtained.

[0105] In one example, the mapping relationship can be recorded in a mapping table. In this way, according to the determined proportion, the mapping table can be looked up to query the dynamic parameter that matches the determined proportion in the mapping table.

[0106] During specific implementation, the point cloud intensity value is denoted as intensity_set, the marker point cloud intensity mean value is denoted as int_mean_marking, and the dynamic parameter is denoted as x. Thus, the above proportion can be used to represent the intensity value sorting position of the marker point cloud intensity mean value int_mean_marking in the point cloud intensity value intensity_set. Exemplarily, the above mapping relationship can be configured as follows: when the marker point cloud intensity mean value int_mean_marking is in the top 98% of the point cloud intensity value intensity_set, the dynamic parameter x takes 3 (it is not difficult to understand that x can also take other values, which are not strictly limited here); when the marker point cloud intensity mean value int_mean_marking is in the top 98% of the point cloud intensity value intensity_set, the dynamic parameter x takes 2 (it is not difficult to understand that x can also take other values, which are not strictly limited here), etc.

[0107] In an alternative embodiment, the specific marker includes a ground marker. Determining the marker point cloud intensity mean value of the specific marker in the scanning scene based at least on the point cloud data includes:

[0108] Extracting the ground point cloud data corresponding to the ground plane in the point cloud data;

[0109] Obtaining an image of the scanning scene and detecting a detection frame corresponding to the ground marker in the image;

[0110] Based on the detection frame and the ground point cloud data, determining the marker point cloud intensity mean value of the ground marker.

[0111] In one example, extracting the ground point cloud data corresponding to the ground plane from the point cloud data may include: using the RANSAC (RANdom SAmple Consensus) algorithm to extract the ground plane parameters indicating the ground plane of the point cloud pc in the point cloud coordinate system of the point cloud data, and extracting the partial point cloud with a distance less than the threshold τ from the ground plane according to the ground plane parameters to obtain the ground point cloud data pc_ground. It is not difficult to understand that the implementation method of extracting a certain specific part of the point cloud data from the point cloud data should be diverse and is not strictly limited here.

[0112] In one example, the image can be obtained by the camera photographing the above scanning scene. The number of the images can be one or more, and the target detection algorithm can be used to detect the images to detect the detection frames corresponding to the ground markers. Exemplarily, the target detection algorithm may include a visual target detection model.

[0113] In one example, based on the detection frame and the ground point cloud data, determining the mean value of the marker point cloud intensity of the ground marker may include: extracting the point cloud corresponding to the detection frame from the ground point cloud data and calculating the mean value of the intensity values of the extracted point cloud as the mean value of the marker point cloud intensity.

[0114] In one example, the number of ground markers can be one or more. Correspondingly, each ground marker can correspond to one or more detection frames.

[0115] In one example, the ground marker may include traffic signs (ground traffic marking lines) painted on the ground.

[0116] In an alternative implementation manner, the determining the mean value of the marker point cloud intensity of the ground marker based on the detection frame and the ground point cloud data includes:

[0117] Projecting the detection frame onto the ground point cloud data to obtain a projection point range;

[0118] Using the point cloud intensity value as the clustering basis, clustering the point cloud within the projection point range in the ground point cloud data to obtain a point cloud clustering cluster;

[0119] Determining the mean value of the marker point cloud intensity of the ground marker according to the point cloud clustering cluster.

[0120] In this embodiment, in a complex environment, ground markers may be partially occluded by other objects or mixed in a background with a similar reflectivity. Therefore, by obtaining an accurate detection box first and projecting the detection box onto the ground point cloud, the interference of irrelevant point clouds can be effectively reduced, so that the focus can be on the area of interest (i.e., the point cloud within the projection point range in the ground point cloud data), thereby improving the detection accuracy. Moreover, clustering based on the point cloud intensity value can help identify point groups with similar reflection characteristics, even if the point groups are not completely continuous in terms of spatial position, which helps to better capture the true boundary and shape of the marker, especially in low light or complex backgrounds.

[0121] In one example, taking the point cloud intensity value as the clustering basis, clustering the point cloud within the projection point range in the ground point cloud data to obtain a point cloud clustering cluster may include: taking the point cloud intensity value as the clustering basis and using the k-means clustering algorithm to cluster the point cloud within the projection point range in the ground point cloud data to obtain a point cloud clustering cluster.

[0122] In one example, the number of point cloud clustering clusters is one or more. According to the point cloud clustering cluster, determining the marker point cloud intensity mean value of the ground marker may include: determining the mean value of the clustering cluster point cloud intensity values corresponding to each point cloud clustering cluster, and obtaining a target point cloud clustering cluster from one or more point cloud clustering clusters, where the target point cloud clustering cluster includes: a point cloud clustering cluster whose corresponding clustering cluster point cloud intensity value mean is greater than a set mean threshold value, and / or, the top N point cloud clustering clusters with the largest corresponding clustering cluster point cloud intensity value mean; determining the marker point cloud intensity mean value int_mean_marking of the ground marker according to the mean value of the clustering cluster point cloud intensity values of the target point cloud clustering cluster.

[0123] In an alternative implementation manner, the preset condition includes that the reflectivity of the object is greater than a reflectivity threshold;

[0124] In the case where it is determined that there is no object with a reflectivity greater than the reflectivity threshold in the scanned scene, the method further includes:

[0125] Adjusting the point cloud data according to the point cloud intensity value to obtain second target point cloud data;

[0126] Performing external parameter calibration on the lidar to be calibrated according to the second target point cloud data.

[0127] It can be understood that at this time, since there are already objects with strong reflectivity in the scene (such as markers with strong reflectivity), the distinction between the point cloud colors of other objects displayed based on intensity and the objects around them is not significant, or the distinction between the reflectivity of the object with strong reflectivity itself and the reflectivity of the surrounding scenery is not significant. As a result, it is difficult to select target homologous points in the lidar point cloud or the corner points selected have large errors. In this case, this embodiment can minimize the influence of the objects with strong reflectivity that exist when the first target point cloud data is displayed. However, if there are no objects with strong reflectivity in the scene, the point cloud data can be directly adjusted according to the point cloud intensity value to obtain the second target point cloud data, and the above problems will not exist either.

[0128] In one example, adjusting the point cloud data according to the point cloud intensity value to obtain the second target point cloud data may include: directly adjusting the point cloud data according to the magnitude of the point cloud intensity value, so that when the adjusted second target point cloud data is displayed, the color of each point is determined by the magnitude of the point cloud intensity value.

[0129] In an alternative embodiment, adjusting the point cloud data according to the point cloud intensity value to obtain the second target point cloud data includes:

[0130] Normalize the point cloud intensity value;

[0131] Adjust the point cloud data according to the normalized point cloud intensity value to obtain the second target point cloud data, where when the second target point cloud data is displayed, the color of each point is determined by the magnitude of the normalized point cloud intensity value.

[0132] In this embodiment, the range difference between the normalized point cloud intensity values is not too large. Therefore, the colors assigned according to the magnitude of the normalized point cloud intensity value can achieve multi-level adaptive gradient color conversion (such as multi-level adaptive RGB gradient color conversion).

[0133] Specifically, when implemented, the gradient color display order of the normalized point cloud intensity values from weak to strong can be blue, green, yellow, and red in sequence (the same color can also have light and dark distinctions). Its principle is to divide the point cloud intensity value into n levels for display, and the implementation steps are as follows:

[0134] a. Normalize the point cloud intensity values in the scene to between 0 and n - 1;

[0135] b. Since the intensity values are displayed in a gradient order of colors from weak to strong as blue, green, yellow, and red, the RGB value change order is (0, 0, n), (0, n, 0), (n, n, 0), (n, 0, 0), and each color change process includes round(n / 3) colors.

[0136] It should be understood that for the lidar to be calibrated with a longer wavelength and when there is no interference from objects with a strong reflectivity in the scene, the overall display effect will be better. For example, when visualizing the second target point cloud data, the display effect of the second target point cloud data can be referred to Figure 2 the picture shown, where the ground marking lines can be clearly seen, so it is easier to select corner points.

[0137] However, in some cases, when there are interfering objects that can form strong echoes / objects with a reflectivity greater than the reflectivity threshold in the scanning scene, because the interfering objects that can form strong echoes / objects with a reflectivity greater than the reflectivity threshold existing at this time will have an intensity value significantly stronger than other objects in the scene. At this time, if the point cloud data is processed into the above-mentioned second target point cloud data and visualized, the display effect will be as Figure 3 shown. It can be seen that when there are interfering objects that can form strong echoes / objects with a reflectivity greater than the reflectivity threshold in the scene at this time, after normalization, it will cause the colors assigned to markers with stronger reflection intensity values such as marking lines to be close to those of the ground with lower reflection intensity values, making it difficult to distinguish and difficult to select corner points from them.

[0138] Correspondingly, when there are interfering objects that can form strong echoes / objects with a reflectivity greater than the reflectivity threshold in the scanning scene, when visualizing the first target point cloud data obtained by the embodiments of the present application, its display effect will be as Figure 4 shown. It can be seen that at this time, it can be adaptively optimized. For example, according to the intensity value of specific markers in the extracted point cloud, the dynamic parameter x is optimized to achieve an optimized display effect. At this time, the interference of objects in the scanning scene on the display effect of the point cloud data can be adaptively reduced, so as to adaptively improve the visualization display effect of the point cloud data (for example, it can make the corner points of the marking lines in the point cloud scene more obvious after adaptive optimization). When selecting homologous points for external parameter calibration, it can be more accurate. When it is inconvenient to perform world coordinate calibration on-site in the early stage, the homologous point splicing can be performed remotely using the corner points of the marking lines in the scene first, and then the calibration of converting the lidar coordinates to world coordinates can be performed later.

[0139] In a second aspect, correspondingly, the embodiments of the present application further provide a lidar external parameter calibration device, which can implement all the processes of the lidar external parameter calibration method provided in the above embodiments.

[0140] SeeFigure 5 , showing a schematic structural diagram of the lidar extrinsic parameter calibration device provided by an embodiment of the present application. The lidar extrinsic parameter calibration device includes:

[0141] A data acquisition module 501, configured to acquire point cloud data of a lidar to be calibrated and determine a scanning scene corresponding to the point cloud data;

[0142] A point cloud intensity value determination module 502, configured to determine point cloud intensity values of each point in the scanning scene according to the point cloud data;

[0143] A first calibration module 503, configured to:

[0144] When it is determined that there is an object satisfying a preset condition in the scanning scene, determine dynamic parameters based on the point cloud data and the point cloud intensity values; and,

[0145] Based on the dynamic parameters and the point cloud intensity values, use a normal distribution model to adjust the point cloud data to obtain first target point cloud data, and perform extrinsic parameter calibration on the lidar to be calibrated according to the first target point cloud data.

[0146] In an optional implementation manner, the adjusting the point cloud data to obtain first target point cloud data by using a normal distribution model based on the dynamic parameters and the point cloud intensity values includes:

[0147] Determine a dynamic threshold interval by using a normal distribution model based on the dynamic parameters and the point cloud intensity values;

[0148] Adjust the point cloud data based on the dynamic threshold interval and the point cloud intensity values to obtain first target point cloud data.

[0149] In an optional implementation manner, the determining a dynamic threshold interval by using a normal distribution model based on the dynamic parameters and the point cloud intensity values includes:

[0150] Determine the standard deviation and mean value of the point cloud intensity values;

[0151] Calculate the product of the dynamic parameter and the standard deviation of the point cloud intensity values, and determine the dynamic threshold interval by using a normal distribution model based on the product and the mean value of the point cloud intensity values.

[0152] In an optional implementation manner, the adjusting the point cloud data based on the dynamic threshold interval and the point cloud intensity values to obtain first target point cloud data includes:

[0153] Traverse each of the points. If the point cloud intensity value of a point is greater than the upper limit value of the dynamic threshold range, adjust the point cloud intensity value of this point to the upper limit value of the dynamic threshold range in the point cloud data. If the point cloud intensity value of this point is less than the lower limit value of the dynamic threshold range, adjust the point cloud intensity value of this point to the lower limit value of the dynamic threshold range in the point cloud data;

[0154] After traversing each of the points, use the adjusted point cloud data as the first target point cloud data.

[0155] In an alternative embodiment, the determining of the dynamic parameter based on the point cloud data and the point cloud intensity value includes:

[0156] Determine the mean value of the marker point cloud intensity of a specific marker in the scanning scene based at least on the point cloud data;

[0157] Determine the dynamic parameter based on the mean value of the marker point cloud intensity and the point cloud intensity value.

[0158] In an alternative embodiment, the determining of the dynamic parameter based on the mean value of the marker point cloud intensity and the point cloud intensity value includes:

[0159] In each of the points, determine the proportion of the points whose point cloud intensity value is less than the mean value of the marker point cloud intensity;

[0160] Based on the mapping relationship between the preset proportion and the parameter, obtain the dynamic parameter matching the determined proportion.

[0161] In an alternative embodiment, the specific marker includes a ground marker. The determining of the mean value of the marker point cloud intensity of the specific marker in the scanning scene based at least on the point cloud data includes:

[0162] Extract the ground point cloud data corresponding to the ground plane in the point cloud data;

[0163] Obtain an image of the scanning scene and detect a detection frame corresponding to the ground marker in the image;

[0164] Determine the mean value of the marker point cloud intensity of the ground marker based on the detection frame and the ground point cloud data.

[0165] In an alternative embodiment, the determining of the mean value of the marker point cloud intensity of the ground marker based on the detection frame and the ground point cloud data includes:

[0166] Project the detection frame onto the ground point cloud data to obtain a projection point range;

[0167] Using the point cloud intensity value as the clustering basis, cluster the point cloud within the projection point range in the ground point cloud data to obtain a point cloud clustering cluster;

[0168] Determine the mean value of the marker point cloud intensity of the ground marker according to the point cloud clustering cluster.

[0169] In an alternative embodiment, the preset condition includes that the reflectivity of the object is greater than the reflectivity threshold;

[0170] The device further includes a second calibration module for:

[0171] In the case where it is determined that there is no object with a reflectivity greater than the reflectivity threshold in the scanning scene, adjust the point cloud data according to the point cloud intensity value to obtain second target point cloud data;

[0172] Perform external parameter calibration on the lidar to be calibrated according to the second target point cloud data.

[0173] In an alternative embodiment, the adjusting the point cloud data according to the point cloud intensity value to obtain second target point cloud data includes:

[0174] Normalize the point cloud intensity value;

[0175] Adjust the point cloud data according to the normalized point cloud intensity value to obtain second target point cloud data, wherein when displaying the second target point cloud data, the color of each point is determined by the magnitude of the normalized point cloud intensity value.

[0176] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in any one of the above are implemented.

[0177] In a fourth aspect, an embodiment of the present application provides a computer program product, including computer instructions, and when the computer instructions are executed by a processor, the steps of the method described in any one of the above are implemented.

[0178] In a fifth aspect, an embodiment of the present application provides a computer device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the steps of the method described in any one of the above are implemented.

[0179] See Figure 6, the computer device of this embodiment includes: a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601, such as a lidar extrinsic parameter calibration program. When the processor 601 executes the computer program, it implements the steps in each of the above-described lidar extrinsic parameter calibration method embodiments, such as Figure 1 the steps S101 - S103 shown.

[0180] Exemplarily, the computer program can be divided into one or more modules / units. The one or more modules / units are stored in the memory 602 and executed by the processor 601 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the computer device.

[0181] The computer device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device may include, but is not limited to, a processor 601 and a memory 602. Those skilled in the art can understand that the schematic diagram is only an example of the computer device and does not constitute a limitation on the computer device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may further include input / output devices, network access devices, a bus, etc.

[0182] The processor 601 can be a central processing unit (CPU), or can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor 601 can also be any conventional processor, etc. The processor 601 is the control center of the computer device and connects various parts of the entire computer device through various interfaces and lines.

[0183] The memory 602 can be used to store the computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory 602, and invoking the data stored in the memory 602, the processor 601 realizes various functions of the computer device. The memory 602 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.); the data storage area can store the data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory 602 can include high-speed random access memory, and can also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0184] Among them, if the modules / units integrated in the computer device are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of this application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 601, the steps of the above-mentioned various method embodiments can be realized. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0185] In summary, the embodiments of this application have at least the following beneficial effects:

[0186] By adopting the embodiments of the present application, point cloud data of a lidar to be calibrated is obtained, and a scanning scene corresponding to the point cloud data is determined; according to the point cloud data, point cloud intensity values of each point in the scanning scene are determined; when it is determined that there is an object satisfying a preset condition in the scanning scene, dynamic parameters are determined based on the point cloud data and the point cloud intensity values; and, based on the dynamic parameters and the point cloud intensity values, the point cloud data is adjusted by using a normal distribution model to obtain first target point cloud data, and the lidar to be calibrated is externally calibrated according to the first target point cloud data, so that the display effect of the point cloud data during visualization can be adjusted and improved, thereby improving the effect of external calibration.

[0187] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary hardware platform, and of course, it can also be implemented entirely by hardware. Based on such an understanding, all or part of the technical solution of the present application that contributes to the background technology can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM (Read-Only Memory), RAM (Random Access Memory), magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments of the present application.

[0188] The above is the preferred embodiment of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present application.

Claims

1. A method for calibrating the extrinsic parameters of a lidar, characterized in that, Including: Obtain the point cloud data of the lidar to be calibrated, and determine the scanning scene corresponding to the point cloud data; Determine the point cloud intensity value of each point in the scanning scene according to the point cloud data; When it is determined that there is an object in the scanning scene that meets the preset conditions, determine the dynamic parameters based on the point cloud data and the point cloud intensity value; And, Based on the dynamic parameters and the point cloud intensity value, use the normal distribution model to adjust the point cloud data to obtain the first target point cloud data, and perform external parameter calibration on the lidar to be calibrated according to the first target point cloud data.

2. The method according to claim 1, wherein The step of using the normal distribution model to adjust the point cloud data based on the dynamic parameters and the point cloud intensity value to obtain the first target point cloud data includes: Determine the dynamic threshold interval using the normal distribution model based on the dynamic parameters and the point cloud intensity value; Adjust the point cloud data based on the dynamic threshold interval and the point cloud intensity value to obtain the first target point cloud data.

3. The method according to claim 2, wherein The step of determining the dynamic threshold interval using the normal distribution model based on the dynamic parameters and the point cloud intensity value includes: Determine the standard deviation and mean of the point cloud intensity value; Calculate the product of the dynamic parameter and the standard deviation of the point cloud intensity value, and determine the dynamic threshold interval using the normal distribution model based on the product and the mean of the point cloud intensity value.

4. The method according to claim 2, wherein The step of adjusting the point cloud data based on the dynamic threshold interval and the point cloud intensity value to obtain the first target point cloud data includes: Traverse each point among the various points. If the point cloud intensity value of this point is greater than the upper limit value of the dynamic threshold interval, adjust the point cloud intensity value of this point to the upper limit value of the dynamic threshold interval in the point cloud data. If the point cloud intensity value of this point is less than the lower limit value of the dynamic threshold interval, adjust the point cloud intensity value of this point to the lower limit value of the dynamic threshold interval in the point cloud data; After traversing the various points, use the adjusted point cloud data as the first target point cloud data.

5. The method according to claim 1, characterized in that The step of determining the dynamic parameters based on the point cloud data and the point cloud intensity value includes: Determine at least based on the point cloud data the mean value of the marker point cloud intensity of the specific marker in the scanning scene; Determine the dynamic parameters based on the mean value of the marker point cloud intensity and the point cloud intensity value.

6. The method according to claim 5, characterized in that The step of determining the dynamic parameters based on the mean value of the marker point cloud intensity and the point cloud intensity value includes: In the various points, determine the proportion of the points whose point cloud intensity value is less than the mean value of the marker point cloud intensity; Obtain the dynamic parameter matching the determined proportion based on the mapping relationship between the preset proportion and the parameter.

7. The method according to claim 5, wherein The specific marker includes a ground marker. The step of determining at least based on the point cloud data the mean value of the marker point cloud intensity of the specific marker in the scanning scene includes: Extract the ground point cloud data corresponding to the ground plane in the point cloud data; Obtain an image of the scanning scene, and detect a detection frame corresponding to the ground marker in the image; Based on the detection box and the ground point cloud data, determine the mean value of the point cloud intensity of the ground marker.

8. The method according to claim 7, wherein The determining the mean value of the point cloud intensity of the ground marker based on the detection box and the ground point cloud data includes: Project the detection box onto the ground point cloud data to obtain a projection point range; Using the point cloud intensity value as a clustering basis, cluster the point cloud within the projection point range in the ground point cloud data to obtain point cloud clustering clusters; Based on the point cloud clustering clusters, determine the mean value of the point cloud intensity of the ground marker.

9. The method according to any one of claims 1-8, characterized in that The preset condition includes that the reflectivity of the object is greater than the reflectivity threshold; In the case where it is determined that there is no object with a reflectivity greater than the reflectivity threshold in the scanning scene, the method further includes: Adjust the point cloud data according to the point cloud intensity value to obtain second target point cloud data; Perform external parameter calibration on the lidar to be calibrated according to the second target point cloud data.

10. The method according to claim 9, wherein The adjusting the point cloud data according to the point cloud intensity value to obtain second target point cloud data includes: Normalize the point cloud intensity value; Adjust the point cloud data according to the normalized point cloud intensity value to obtain second target point cloud data, wherein when the second target point cloud data is displayed, the color of each point is determined by the magnitude of the normalized point cloud intensity value.

11. A calibration device for the extrinsic parameters of a lidar, characterized in that, Includes: A data acquisition module, configured to acquire the point cloud data of the lidar to be calibrated and determine the scanning scene corresponding to the point cloud data; A point cloud intensity value determination module, configured to determine the point cloud intensity value of each point in the scanning scene according to the point cloud data; A first calibration module, configured to: In the case where it is determined that there is an object satisfying the preset condition in the scanning scene, determine dynamic parameters based on the point cloud data and the point cloud intensity value; and, Based on the dynamic parameters and the point cloud intensity value, use a normal distribution model to adjust the point cloud data to obtain first target point cloud data, and perform external parameter calibration on the lidar to be calibrated according to the first target point cloud data.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1-10.

13. A computer program product, comprising computer instructions, characterized in that, When the computer instruction is executed by a processor, it implements the method according to any one of claims 1-10.

14. A computer device, characterized in that, Includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the method according to any one of claims 1-10.

Images