Parameter calibration method, device, computer equipment and storage medium

Abstract

The present application relates to a parameter calibration method, device, computer equipment and storage medium. After obtaining the first position information between the first sensor and the faulty sensor, and the second position information between the first sensor and the second sensor that replaces the faulty sensor, the change value between the first position information and the second position information is obtained, and the external parameter information is adjusted based on the change value, and then the adjusted external parameter information is calibrated based on the mapping relationship between the point cloud and the image. This method avoids re-calibrating the entire process, saves a lot of working time, and thus improves the calibration efficiency of parameters between sensor devices.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a parameter calibration method, device, computer equipment and storage medium. Background Art

[0002] With the development of sensor technology and computer technology, simultaneous positioning and map construction solutions based on various sensors have been widely used in fields such as robot autonomous navigation, unmanned driving, mobile measurement and battlefield environment construction.

[0003] Among the many sensor options, LiDAR and cameras have great complementary advantages. For example, cameras can obtain more complete texture and color information of the target, as well as more comprehensive category determination and color definition of the target; LiDAR can obtain more complete target position and size information, and has more advantages in coordinate positioning. Therefore, there is a solution to integrate LiDAR and cameras to improve the visual perception ability of vehicles, but the integration of LiDAR and cameras needs to consider the calibration of internal and external parameters between the two, and in the long-term use process, after the LiDAR or camera is replaced due to failure or other objective reasons, the internal and external parameters between the LiDAR and the camera need to be recalibrated for the entire process.

[0004] However, the above recalibration workload is large and time-consuming, resulting in inefficiency in the calibration of internal and external parameters between the lidar and the camera. Summary of the invention

[0005] Based on this, it is necessary to provide a parameter calibration method, device, computer equipment and storage medium that can improve the efficiency of parameter calibration between lidar and camera in order to solve the above technical problems.

[0006] In a first aspect, an embodiment of the present application provides a parameter calibration method, the method comprising:

[0007] Acquire first position information between the first sensor and the faulty sensor, and second position information between the first sensor and the second sensor; the second sensor is a sensor that replaces the faulty sensor;

[0008] Adjusting the external reference information between the first sensor and the second sensor according to the position information change value between the first position information and the second position information;

[0009] Based on the mapping relationship between the point cloud and the image, the adjusted extrinsic parameter information is calibrated.

[0010] In one embodiment, the above-mentioned position information change value includes: a relative angle change value and a relative position change value;

[0011] According to the change value between the first position information and the second position information, the external parameter information between the first sensor and the second sensor is adjusted, including:

[0012] The relative angle change value is added to the rotation matrix in the external parameter information, and the relative position change value is added to the translation matrix in the external parameter information.

[0013] In one embodiment, the first sensor is a camera device, and the fault sensor and the second sensor are both laser radars;

[0014] Then, according to the mapping relationship between the point cloud and the image, the adjusted external parameter information is calibrated, including:

[0015] According to the mapping relationship between the point cloud and the image and the adjusted external parameter information, the point cloud data collected by the second sensor is mapped onto the image to obtain a point cloud plane image;

[0016] Based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel points in the point cloud plane image, a loss function related to the external parameter information is constructed;

[0017] Optimize the loss function until the loss function meets the preset optimization termination condition, and obtain the external parameter information corresponding to the termination of the loss function optimization.

[0018] In one embodiment, the point cloud data collected by the second sensor is mapped onto the image according to the mapping relationship between the point cloud and the image and the adjusted external parameter information to obtain the point cloud plane image, including:

[0019] Obtain the coordinates of the point cloud points in the point cloud data in the world coordinate system and the internal reference information of the camera device;

[0020] Substitute the adjusted external parameter information, the internal parameter information of the camera device, and the coordinates of the point cloud points in the point cloud data in the world coordinate system into the mapping relationship between the point cloud and the image, and calculate the coordinates of the point cloud points in the point cloud data in the pixel coordinate system;

[0021] According to the coordinates of the point cloud points in the point cloud data in the pixel coordinate system, a point cloud plane image is obtained.

[0022] In one embodiment, the loss function related to the external parameter information is constructed based on the reflectivity of the point cloud in the point cloud data and the gray value of the pixel point in the point cloud plane image, including:

[0023] A reflectivity histogram is constructed according to the reflectivity of the point cloud in the point cloud data, and a gray value histogram is constructed according to the gray values ​​of the pixels in the point cloud plane image;

[0024] Constructing a joint histogram based on the reflectance histogram and the gray value histogram;

[0025] The loss function related to the external parameter information is constructed based on the reflectance histogram, gray value histogram and joint histogram.

[0026] In one embodiment, the loss function related to the external parameter information is constructed based on the reflectivity histogram, the gray value histogram and the joint histogram, including:

[0027] The reflectivity edge probability distribution is calculated according to the reflectivity histogram, the gray value edge probability distribution is calculated according to the gray value histogram, and the joint probability distribution is calculated according to the joint histogram;

[0028] The loss function related to the external parameter information is constructed according to the reflectivity edge probability distribution, gray value edge probability distribution and joint probability distribution.

[0029] In one embodiment, the above-mentioned optimization of the loss function until the loss function satisfies a preset optimization termination condition, and the external parameter information corresponding to the termination of the loss function optimization is obtained, including:

[0030] The preset gradient descent algorithm is used to adjust the value of the loss function until the value of the loss function meets the preset optimization termination condition, and the corresponding external parameter information when the loss function optimization is terminated is obtained.

[0031] In one embodiment, the above optimization termination condition includes: the value of the loss function is less than a preset threshold or the continuous change rate of the value of the loss function is within a preset range.

[0032] In a second aspect, an embodiment of the present application provides a parameter calibration device, the device comprising:

[0033] A position information acquisition module, used to acquire first position information between the first sensor and the faulty sensor, and second position information between the first sensor and the second sensor; the second sensor is a sensor that replaces the faulty sensor;

[0034] A change value acquisition module, used to adjust the external reference information between the first sensor and the second sensor according to the position information change value between the first position information and the second position information;

[0035] The calibration module is used to calibrate the adjusted external parameter information based on the mapping relationship between the point cloud and the image.

[0036] In one embodiment, the above-mentioned position information change value includes: a relative angle change value and a relative position change value; then the above-mentioned change value acquisition module is specifically used to add the relative angle change value to the rotation matrix in the external reference information, and to add the relative position change value to the translation matrix in the external reference information.

[0037] In one embodiment, the first sensor is a camera device, and the fault sensor and the second sensor are both laser radars;

[0038] The above calibration module includes:

[0039] A mapping unit, used to map the point cloud data collected by the second sensor onto the image according to the mapping relationship between the point cloud and the image and the adjusted external parameter information, so as to obtain a point cloud plane image;

[0040] A construction unit, used to construct a loss function related to the external parameter information based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel point in the point cloud plane image;

[0041] The optimization unit is used to optimize the loss function until the loss function meets the preset optimization termination condition and obtain the external parameter information corresponding to the termination of the loss function optimization.

[0042] In one embodiment, the mapping unit includes:

[0043] An acquisition subunit, used to acquire the coordinates of the point cloud points in the point cloud data in the world coordinate system and the internal reference information of the camera device;

[0044] A calculation subunit is used to substitute the adjusted external parameter information, the internal parameter information of the camera device, and the coordinates of the point cloud points in the point cloud data in the world coordinate system into the mapping relationship between the point cloud and the image, and calculate the coordinates of the point cloud points in the point cloud data in the pixel coordinate system;

[0045] The subunit is determined to obtain a point cloud plane image according to the coordinates of the point cloud points in the point cloud data in the pixel coordinate system.

[0046] In one embodiment, the building block comprises:

[0047] A histogram subunit, used to construct a reflectivity histogram according to the reflectivity of the point cloud in the point cloud data, and to construct a gray value histogram according to the gray values ​​of the pixels in the point cloud plane image;

[0048] A joint subunit, used for constructing a joint histogram according to the reflectance histogram and the gray value histogram;

[0049] A subunit is constructed to construct a loss function related to external parameter information based on reflectance histogram, gray value histogram and joint histogram.

[0050] In one of the embodiments, the above-mentioned construction subunit is specifically used to calculate the reflectivity edge probability distribution based on the reflectivity histogram, calculate the grayscale value edge probability distribution based on the grayscale value histogram, and calculate the joint probability distribution based on the joint histogram; and construct a loss function related to the external parameter information based on the reflectivity edge probability distribution, the grayscale value edge probability distribution, and the joint probability distribution.

[0051] In one of the embodiments, the above-mentioned optimization unit is specifically used to adjust the value of the loss function using a preset gradient descent algorithm until the value of the loss function meets a preset optimization termination condition, and obtain the external parameter information corresponding to the termination of the loss function optimization.

[0052] In one embodiment, the above optimization termination condition includes: the value of the loss function is less than a preset threshold or the continuous change rate of the value of the loss function is within a preset range.

[0053] In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any method provided in the embodiment of the first aspect are implemented.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any method provided in the embodiment of the first aspect above are implemented.

[0055] The embodiment of the present application provides a parameter calibration method, device, computer equipment and storage medium, which obtains the first position information between the first sensor and the faulty sensor, and the second position information between the first sensor and the second sensor that replaces the faulty sensor, obtains the change value between the first position information and the second position information, and adjusts the external parameter information based on the change value, and then calibrates the adjusted external parameter information based on the mapping relationship between the point cloud and the image. In this method, after obtaining the change value determined by the position relationship between the two sensors before the replacement and the position relationship between the two sensors after the replacement (i.e., the prior information of the sensor device), the external parameter information is first adjusted based on the change value, and then the adjusted external parameter information is calibrated. In this way, the external parameter information is first adjusted using the prior information of the sensor device directly, without the need to re-measure the relative position and angle and then adjust the process, avoiding the need to re-calibrate the entire process, saving a lot of working time, and thus improving the calibration efficiency of parameters between sensor devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 An application environment diagram of a parameter calibration method in an embodiment;

[0057] Figure 1aThe positional relationship between the laser radar and the camera device in one embodiment;

[0058] Figure 1b is an internal structure diagram of a computer device in one embodiment;

[0059] Figure 2 A schematic diagram of a flow chart of a parameter calibration method in an embodiment;

[0060] Figure 3 is a flow chart of a parameter calibration method in another embodiment;

[0061] Figure 4 is a flow chart of a parameter calibration method in another embodiment;

[0062] Figure 5 is a flow chart of a parameter calibration method in another embodiment;

[0063] Figure 6 is a flow chart of a parameter calibration method in another embodiment;

[0064] Figure 7 is a schematic diagram of a parameter calibration method in another embodiment;

[0065] Figure 8 4 is a structural block diagram of a parameter calibration device in an embodiment. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] The parameter calibration method provided in this application can be applied to Figure 1 In the application environment shown, the application environment includes laser radar 01, camera device 02 and computer device 03. Among them, laser radar 01, camera device 02 and computer device can communicate with each other; laser radar 01 includes but is not limited to pulse radar, continuous wave radar, meter wave radar, decimeter wave radar, centimeter wave radar, etc.; camera device 02 includes but is not limited to professional camera, CCD camera, network camera, portable camera, black and white camera, color camera, infrared camera, X-ray camera, undercover camera, etc.; computer device 03 includes but is not limited to server, various terminals: personal computer, laptop computer, smart phone, tablet computer and portable wearable device, etc.

[0068] The positions of the laser radar 01 and the camera device 02 are relatively fixed. Figure 1a, which is a schematic diagram of the installation of a laser radar 01 and a camera device 02; wherein, the internal structure diagram of the computer device 03 can be found in Figure 1b . The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data for parameter calibration. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a parameter calibration method is implemented.

[0069] The embodiments of the present application provide a parameter calibration method, device, computer equipment and storage medium, which can improve the efficiency of parameter calibration between laser radar and camera. The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems will be described in detail below through embodiments and in combination with the accompanying drawings. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. It should be noted that a parameter calibration method provided by the present application, Figure 2-Figure 7 The execution subject is a computer device. Figure 2-Figure 7 The execution subject may also be a parameter calibration device, which may be implemented as part or all of a computer device through software, hardware, or a combination of software and hardware.

[0070] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0071] In one embodiment, Figure 2 As shown, a parameter calibration method is provided. This embodiment relates to a specific process in which a computer device calibrates external parameter information according to a transformation value of first position information between a fault sensor and a first sensor and external parameter information between a second sensor and the first sensor. This embodiment includes the following steps:

[0072] S101, obtaining first position information between a first sensor and a faulty sensor, and second position information between the first sensor and a second sensor; the second sensor is a sensor that replaces the faulty sensor.

[0073] Among them, the first sensor can be a camera device or a laser radar. If the first sensor is a camera device, then the faulty sensor is the laser radar. If the first sensor is a laser radar, then the faulty sensor is the camera device. Since the laser radar or the camera device will more or less fail when the laser radar and the camera device are integrated to obtain the surrounding environment information, the failed laser radar or the failed camera device is the faulty sensor; when a failure occurs, it is natural to replace it. The failed laser radar is replaced with a new laser radar, or the failed camera device is replaced with a new camera device, and the new laser radar or the new camera device is the second sensor.

[0074] Correspondingly, the position information in the first position information and the second position information refers to the position information between the laser radar and the camera device. Among them, the position information refers to the positional relationship outside the two sensor device bodies, for example, the relative position and relative angle between the laser radar and the camera device. In practical applications, in order to enable the laser radar and the camera device to fully and effectively obtain the surrounding environment information, when installing the laser radar and the camera device, there must be a suitable relative position and relative angle between the two. Therefore, after replacing the faulty sensor with the second sensor, the relative position and relative angle between the second sensor and the first sensor need to be recalibrated to ensure that the relative position and relative angle between the second sensor and the first sensor can enable the laser radar and the camera device to fully and effectively obtain the surrounding environment information.

[0075] Specifically, taking the case where the first sensor is a camera device, the faulty sensor is a laser radar, the position information is the external reference information between the laser radar and the camera device, and the relative position and angle between the two (subsequent steps and embodiments are all described as an example), the computer device obtains the relative position and angle between the camera device and the faulty laser radar, and obtains the relative position and angle between the camera device and the new laser radar. It should be noted that since the faulty laser radar fails during normal use, the relative position and angle between the camera device and the faulty laser radar are appropriate and meet the requirements; while the new laser radar is newly installed after replacing the faulty laser radar, and the position between the new laser radar and the camera device is still in the debugging stage, so the relative position and angle between the camera device and the new laser radar are not values ​​that meet the requirements, and further adjustment is required.

[0076] S102: Adjust the external reference information between the first sensor and the second sensor according to the position information change value between the first position information and the second position information.

[0077] After obtaining the relative position and relative angle between the camera device and the faulty laser radar (i.e., the first position information), and the relative position and relative angle between the camera device and the new laser radar (i.e., the external reference information), obtain the change value between the two, i.e., the relative position change value and the relative angle change value. For example, the relative position between the camera device and the faulty laser radar is h1, and the relative angle is α1; the relative position between the camera device and the new laser radar is h2, and the relative angle is α2; then the relative position change value is h1-h2, and the relative angle change value is α1-α2; wherein, the values ​​of h1-h2 and α1-α2 can be positive or negative.

[0078] After determining the change values ​​of the relative position and relative angle between the camera device and the laser radar before and after the replacement, the external parameter information is calibrated based on the change values. The external parameter information is the external parameter information between the first sensor and the second sensor, which can also be called an external parameter matrix.

[0079] S103, calibrating the adjusted extrinsic parameter information based on the mapping relationship between the point cloud and the image.

[0080] The mapping relationship between the point cloud and the image represents the relationship between the world coordinate system (the coordinate system used by the laser radar) and the pixel coordinate system (the coordinate system used by the pixels in the image of the camera): Among them, in this mapping relationship, is the intrinsic parameter matrix of the camera device, is the external parameter matrix of the camera device, is the coordinate matrix of each point in the world coordinate system, is the coordinate matrix of the point in the pixel coordinate system.

[0081] According to the mapping relationship between the point cloud and the image, the adjusted external parameter information is further calibrated to make the calibrated external parameter information more accurate. After the laser radar and camera equipment using the calibrated external parameter information obtain the surrounding environment image, the surrounding environment information can be more accurately fused.

[0082] The parameter calibration method provided in this embodiment obtains the first position information between the first sensor and the faulty sensor, and the second position information between the first sensor and the second sensor that replaces the faulty sensor, obtains the change value between the first position information and the second position information, and adjusts the external parameter information based on the change value, and then calibrates the adjusted external parameter information based on the mapping relationship between the point cloud and the image. In this method, after obtaining the change value determined by the position relationship between the two sensors before replacement and the position relationship between the two sensors after replacement (i.e., the prior information of the sensor device), the external parameter information is first adjusted based on the change value, and then the adjusted external parameter information is calibrated. In this way, the external parameter information is first adjusted using the prior information of the sensor device directly, without the need to re-measure the relative position and angle and then adjust the process, avoiding the need to re-calibrate the entire process, saving a lot of working time, and thus improving the calibration efficiency of parameters between sensor devices.

[0083] In one embodiment, the above position information change value includes: a relative angle change value and a relative position change value; then the above S102 includes: adding the relative angle change value to the rotation matrix in the external reference information, and adding the relative position change value to the translation matrix in the external reference information.

[0084] In this embodiment, the first sensor is a camera device, the fault sensor and the second sensor are both laser radars, and the change value includes a relative position change value and a relative angle change value.

[0085] Among them, the external parameter information includes the external parameter matrix of the camera device, and the external parameter matrix can be expressed as Among them, [RT] includes the rotation matrix R (R) between the world coordinate system where the laser radar is located and the camera coordinate system where the camera device is located. 3×3 ), and the translation vector matrix T(T 3×1 ), that is, R and T reflect the posture information of the camera device, so the rotation matrix R and the translation vector matrix T can be adjusted according to the relative position change value and relative angle change value before and after the faulty lidar is updated: specifically, the relative angle change value is added to the rotation matrix, and the relative position change value is added to the translation matrix.

[0086] It can be understood that: according to the above step S102, the relative position change value is h1-h2, and the relative angle change value is α1-α2. The final result can be a positive value or a negative value. Therefore, when adjusting the external parameter information according to the relative position change value and the relative angle change value, the relative angle change value is directly added to the rotation matrix in the two external parameter information, and the relative position change value is added to the translation matrix in the external parameter information. If the change value is positive, the increase here is a positive increase. If the change value is negative, the increase here is the subtraction of the change value.

[0087] The process of calibrating the adjusted external parameter information based on the mapping relationship between the point cloud and the image is described below through an embodiment. Figure 3 As shown, the above S103 includes the following steps:

[0088] S201, mapping the point cloud data collected by the second sensor onto the image according to the mapping relationship between the point cloud and the image and the adjusted extrinsic parameter information to obtain a point cloud plane image.

[0089] After adjusting the extrinsic parameter information, the point cloud data is mapped onto the image according to the adjusted extrinsic parameter information and the mapping relationship between the point cloud and the image.

[0090] The adjusted external parameter information includes the adjusted external parameter matrix. Then, based on the mapping relationship, when mapping the point cloud data collected by the second sensor to the image, it is also necessary to obtain the internal parameter matrix of the camera device and the coordinates of the point cloud data collected by the second sensor in the world coordinate system. Based on this, in one embodiment, Figure 4 As shown, according to the mapping relationship between the point cloud and the image and the adjusted external parameter information, the point cloud data collected by the second sensor is mapped onto the image to obtain a point cloud plane image, including the following steps:

[0091] S301, obtaining the coordinates of the point cloud points in the point cloud data in the world coordinate system and the internal parameter information of the camera device.

[0092] The internal parameter information of the camera device is the internal parameter matrix. Since the internal parameter matrix of the camera device reflects the internal parameters of the camera device, after replacing the faulty laser radar, the camera device itself does not change, and its internal parameters do not change either, so the internal parameter matrix of the camera device can be directly obtained.

[0093] As for the coordinates of the point cloud points in the point cloud data in the world coordinate system, since the laser radar itself uses the world coordinate system, the three-dimensional coordinates of the laser point cloud can also be directly obtained, that is, the coordinates in the world coordinate system (Xw, Yw, Zw).

[0094] S302, substitute the adjusted external parameter information, the internal parameter information of the camera device, and the coordinates of the point cloud points in the point cloud data in the world coordinate system into the mapping relationship between the point cloud and the image, and calculate the coordinates of the point cloud points in the point cloud data in the pixel coordinate system.

[0095] In the above steps, the adjusted external parameter information, the internal parameter information of the camera device, and the coordinates of the point cloud points in the point cloud data in the world coordinate system are obtained, that is, the camera device internal parameter matrix in the mapping relationship between the point cloud and the image. External parameter matrix and the world coordinate system point coordinates (X W ,Y W ,Z W ) matrix, then, by substituting the adjusted extrinsic parameter information, the intrinsic parameter information of the camera device, and the coordinates of the point cloud points in the point cloud data in the world coordinate system into the mapping relationship between the point cloud and the image, the coordinates (μ, ν) of the point cloud points in the point cloud data in the pixel coordinate system can be calculated.

[0096] It should be noted that in practical applications, the images taken by the camera device are actually distorted (the so-called distortion means that the straight line in the world coordinate system is no longer a straight line when converted to other coordinate systems), which leads to distortion. Therefore, when determining the coordinates of the point cloud point in the pixel coordinate system, it is also necessary to perform distortion correction on the camera's intrinsic parameters. The distortion correction process includes: obtaining the distortion coefficient of the camera device, and then correcting the intrinsic parameter information of the camera device according to the distortion coefficient to obtain the coordinates (μ, ν) of the point cloud point in the pixel coordinate system in the final point cloud data.

[0097] Specifically, the distortion coefficients include radial distortion coefficients: k1, k2, k3, k4, k5, k6, and tangential distortion coefficients: p1, p2. x *x″+c x and v = f y *y″+c y Make corrections.

[0098] In this formula, cx, cy represent the coordinates of the center point of the image, fx, fy are the focal lengths expressed in pixels; in this formula,

[0099] Among them, x'=x / z, y'=y / z, where x, y, z are the coordinates of each point in the camera coordinate system, and the relationship between the camera coordinate system and the world coordinate system is: That is to say, first determine the coordinates (x, y, z) of the point cloud point in the camera coordinate system based on the relationship between the world coordinate system and the camera coordinate system, then calculate x′ and y′ based on x, y, z, and then calculate x″ and y″, and finally get the corrected (μ, ν).

[0100] S303, obtaining a point cloud plane image according to the coordinates of the point cloud points in the point cloud data in the pixel coordinate system.

[0101] After calculating the coordinates (μ, ν) of the point cloud corresponding to the point cloud data in the pixel coordinate system, the point cloud corresponding to the point cloud data can be mapped to the image to obtain the point cloud plane image.

[0102] In this embodiment, the coordinates of the point cloud corresponding to the point cloud data in the pixel coordinate system are calculated, and the pixel points at the coordinates of the point cloud in the pixel coordinate system are extracted from the image as the point cloud plane image. This realizes the spatial matching between the laser point cloud and the image taken by the camera device, which facilitates the subsequent calibration of the parameters between the laser radar and the camera.

[0103] S202, constructing a loss function related to the external parameter information based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel points in the point cloud plane image.

[0104] According to the point cloud plane image in the above step S201, the point cloud plane image is obtained based on the mapping of the point cloud data, and the point cloud data contains rich information, including three-dimensional coordinates, colors, classification values, intensity values, time, reflectivity, etc., which are not limited. Among them, the ratio of the reflected laser power projected onto the target to the total laser power projected onto the target is called the reflectivity of the target, which reflects the measure of the laser beam reflected by the obstacle at each point cloud point in the laser point cloud at its position, so there is a strong correlation between the reflectivity of the laser point cloud and the grayscale value of the point in the point cloud plane image.

[0105] Based on this, the probability distribution of the reflectivity of the point cloud in the point cloud data and the gray value of the point in the point cloud plane image can be determined, and then the loss function is constructed according to the probability distribution diagram. The constructed loss function is related to the external parameter information. Among them, the external parameter information includes the external parameter matrix (i.e., the six degrees of freedom of rotation and translation), so the loss function related to the six degrees of freedom of rotation and translation is established.

[0106] Specifically, the point cloud points in the laser point cloud are mapped onto the image to obtain the point cloud plane image, and the pixel values ​​of each point in the point cloud plane image can be directly obtained. According to the pixel values ​​of each point in the point cloud plane image, a probability distribution diagram, such as a histogram, a pie chart or a curve chart, etc., is constructed to count the grayscale values ​​of the points in the point cloud plane image, and the distribution of the grayscale values ​​of each point in the point cloud plane image can be obtained. The reflectivity of the point cloud points in the laser point cloud can also be directly obtained, and then a probability distribution diagram, such as a histogram, a pie chart or a curve chart, etc., is constructed according to the reflectivity of the point cloud points, so as to count the reflectivity of the point cloud points in the point cloud data, and the distribution of the reflectivity of the point cloud points in the point cloud data can be obtained. In order to take into account the comprehensive information of each point cloud point, the reflectivity of the point cloud points in the point cloud data and the grayscale values ​​of each point in the point cloud plane image can also be combined to comprehensively determine the distribution of each point. Finally, according to the distribution of each point in the plane image, the distribution of each point cloud point in the point cloud data, and the comprehensive distribution, a loss function related to the external parameter information is constructed.

[0107] S203, optimizing the loss function until the loss function meets a preset optimization termination condition, and obtaining external parameter information corresponding to the termination of the loss function optimization.

[0108] After constructing the loss function related to the external parameter information, the loss function is optimized. When the loss function optimization is terminated, the external parameter information corresponding to the termination of the loss function optimization is obtained. The external parameter information corresponding to the termination of the loss function optimization is the external parameter information after the camera device and the new lidar are calibrated.

[0109] Optionally, when optimizing the loss function, a preset gradient descent algorithm may be used to adjust the value of the loss function until the value of the loss function satisfies a preset optimization termination condition, thereby obtaining the external parameter information corresponding to the termination of the loss function optimization. Optionally, the optimization termination condition includes: the value of the loss function is less than a preset threshold value or the continuous change rate of the value of the loss function is within a preset range.

[0110] Among them, the gradient descent algorithm is an algorithm for finding the minimum value. It searches for the optimal trainable parameters through continuous iteration of the gradient. For example, the model parameters are first initialized, for example, the model parameters are set to 0 (can also be initialized to other values), and then the values ​​of the parameters are changed little by little, trying to make the loss function smaller until the minimum value of the loss function is reached, that is, in this embodiment, until the value of the loss function is adjusted to be less than a preset threshold or the continuous change rate of the value of the loss function is within a preset range.

[0111] In one embodiment, an adaptive learning rate algorithm (also known as the Adam algorithm) may be used to optimize the learning rate of the gradient descent algorithm, which may enable the loss function value to achieve a better convergence effect.

[0112] Generally, when selecting the learning rate, you should choose an appropriate one. If the learning rate is too small, the convergence speed will be too slow. If the learning rate is too large, some local minima will be directly ignored. Therefore, when selecting the learning rate, you should choose an appropriate learning rate to ensure the convergence effect.

[0113] The gradient descent algorithm is used to calculate the value of the loss function until the value of the loss function is less than a preset threshold or the continuous change rate of the value of the loss function is within a preset range, and the corresponding external parameter information when the loss function optimization is terminated is obtained.

[0114] In this embodiment, the point cloud points in the laser point cloud are mapped to the image, and then based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel points in the point cloud plane image, a loss function related to the external parameter information is constructed, and the loss function is optimized to obtain the corresponding external parameter information when the loss function optimization terminates. Since there is a strong correlation between the reflectivity of the point cloud points and the grayscale value of the points in the point cloud plane image, the constructed loss function can fully and comprehensively reflect the change information of each point, so that the calibrated external parameter information can be determined, and the full name is automatically calibrated by the laser radar and the camera equipment, without the need to use calibration objects, avoiding waste of manpower and material resources, and improving the reusability of the calibration method.

[0115] In one embodiment, Figure 5 As shown, in the above step S203, "constructing a loss function related to the external parameter information based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel point in the point cloud plane image" includes the following steps:

[0116] S401, constructing a reflectivity histogram according to the reflectivity of the point cloud in the point cloud data, and constructing a gray value histogram according to the gray values ​​of the pixels in the point cloud plane image.

[0117] When constructing a reflectivity histogram, the reflectivity of all the point cloud points in the point cloud data is obtained, and the reflectivity of all the point cloud points in the point cloud data is counted to obtain the number of occurrences of each reflectivity. Then, based on the number of occurrences of each reflectivity, the reflectivity value is used as the horizontal axis and the number of occurrences of the reflectivity is used as the vertical axis to obtain a reflectivity histogram. In the reflectivity histogram, x is used to represent the reflectivity value, X is used to represent the data set of the reflectivity value, and p is used to represent the reflectivity value. X (x) represents the probability value of reflectivity x in the reflectivity histogram.

[0118] When constructing a grayscale histogram, the grayscale values ​​of all pixels in the point cloud plane image are obtained, and the grayscale values ​​of all pixels in the point cloud plane image are counted to obtain the number of grayscale values. Then, based on the number of times each grayscale value appears, the grayscale value is the horizontal axis and the number of times the grayscale value appears is the vertical axis to obtain a grayscale histogram. In the grayscale histogram, y is used to represent the grayscale value, Y is used to represent the reflectivity value data set, and p is used to represent the grayscale value.Y (y) represents the probability value of gray value y in the grayscale histogram.

[0119] S402: construct a joint histogram according to the reflectivity histogram and the gray value histogram.

[0120] When constructing a joint histogram, obtain the reflectivity of all point cloud points in the point cloud data, and then obtain the grayscale values ​​of all pixels in the point cloud plane image. Statistic the reflectivity of all point cloud points in the point cloud data and the grayscale values ​​of the pixels in the point cloud plane image to obtain a set of values ​​(x, y) consisting of the reflectivity x of the point cloud point and the grayscale value y of the pixel point, and count the number of occurrences of the set of values ​​(x, y). Based on the number of occurrences of the set of values, the joint histogram is obtained with the set of values ​​(x, y) as the horizontal axis and the number of occurrences of the set of values ​​as the vertical axis. In the joint histogram, x is used to represent the reflectivity value, y is used to represent the grayscale value, (X, Y) is used to represent the reflectivity value data set, and p is used. XY (x,y) represents the probability value of the group of values ​​(x,y) in the joint histogram.

[0121] S403, constructing a loss function related to the external parameter information based on the reflectance histogram, the gray value histogram and the joint histogram.

[0122] After obtaining the reflectance histogram, gray value histogram and joint histogram, a probability distribution function is constructed according to each histogram.

[0123] Alternatively, if Figure 6 As shown, in one embodiment, a loss function related to external parameter information is constructed based on a reflectivity histogram, a gray value histogram, and a joint histogram, including the following steps:

[0124] S501, calculating the reflectivity edge probability distribution according to the reflectivity histogram, calculating the gray value edge probability distribution according to the gray value histogram, and calculating the joint probability distribution according to the joint histogram.

[0125] Specifically, the formula for calculating the reflectivity edge probability distribution based on the reflectivity histogram is:

[0126] In this formula, H(X) represents the reflectivity edge probability distribution, and pX(x) represents the probability value of reflectivity x in the reflectivity histogram.

[0127] The formula for calculating the gray value edge probability distribution based on the gray histogram is:

[0128] In this formula, H(Y) represents the gray value edge probability distribution, and pY(y) represents the probability value of reflectivity y in the reflectivity histogram.

[0129] The formula for calculating the joint probability distribution based on the joint histogram is as follows:

[0130] H(X,Y)=-∑∑p XY (x,y)logp XY (x, y), in this formula, H(XY) represents the joint marginal probability distribution, and pXY(x, y) represents the probability value of the group of values ​​(x, y) in the joint histogram.

[0131] S502, constructing a loss function related to the external parameter information according to the reflectivity edge probability distribution, the gray value edge probability distribution, and the joint probability distribution.

[0132] After determining the reflectivity edge probability distribution, gray value edge probability distribution, and joint probability distribution, the loss function is obtained based on the reflectivity edge probability distribution, gray value edge probability distribution, and joint probability distribution. The formula of the loss function is:

[0133] When constructing the loss function, first use the formula Calculate the gradient G in the loss function, and then calculate the information entropy MI(X,Y)MI(X,Y)=H(X)+H(Y)-H(X,Y), and then substitute the gradient G and information entropy MI(X,Y) into the formula Θ k+1 =Θ k +λF(MI(X,Y;Θ k )) In this case, we get the loss function

[0134] In this embodiment, a reflectivity histogram is constructed based on the reflectivity of the point cloud points in the point cloud data, a grayscale histogram is constructed based on the grayscale values ​​of the pixel points in the point cloud plane image, and a joint histogram is constructed based on the reflectivity histogram and the grayscale histogram. Then, probability distribution functions are constructed according to the reflectivity histogram, the grayscale histogram and the joint histogram. Since the probability distribution function takes into account the reflectivity of each point in the point cloud and the pixel value of each point in the plane image, the external parameter information of the loss function constructed based on the probability distribution function is the most accurate and meets the requirements when the optimization is terminated, thereby ensuring the accuracy of the external parameter calibration between the new lidar and the camera equipment.

[0135] In one embodiment, Figure 7 As shown, an embodiment of a parameter calibration method is provided, and the embodiment includes:

[0136] S601, replace the faulty laser radar, and record the relative position and relative angle between the faulty laser radar and the camera device before the faulty laser radar is replaced;

[0137] S602, recording the relative position and relative angle between the new laser radar and the camera device after the faulty laser radar is replaced;

[0138] S603, adjusting the external reference information between the new laser radar and the camera device according to the relative position change value and the relative angle change value before and after the replacement;

[0139] S604, mapping the point cloud data into the image by using the mapping relationship between the point cloud and the image and the adjusted new external parameter information between the laser radar and the camera device;

[0140] S605, establishing a loss function associated with the external parameter information between the new laser radar and the camera device;

[0141] S606, iteratively optimizing the loss function using a gradient descent algorithm;

[0142] S607, calculate whether the value of the loss function is less than a threshold; if so, execute S609;

[0143] S608, if the iteration termination condition is reached;

[0144] S609, the iteration is terminated, and the external parameter information between the new laser radar and the camera device is corrected.

[0145] The implementation principles and technical effects of each step in the parameter calibration method provided in this embodiment are similar to those in the previous parameter calibration method embodiments, and will not be repeated here. Figure 7 The implementation method of each step in the embodiment is only an example, and the implementation method is not limited. The order of each step can be adjusted in actual application as long as the purpose of each step can be achieved.

[0146] It should be understood that although Figure 2-7 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-7 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0147] In one embodiment, Figure 8 As shown, a parameter calibration device is provided, comprising: a position information acquisition module 10, a change value acquisition module 11 and a calibration module 12, wherein:

[0148] The position information acquisition module 10 is used to acquire first position information between the first sensor and the faulty sensor, and second position information between the first sensor and the second sensor; the second sensor is a sensor that replaces the faulty sensor;

[0149] A change value acquisition module 11, used to adjust the external reference information between the first sensor and the second sensor according to the position information change value between the first position information and the second position information;

[0150] The calibration module 12 is used to calibrate the adjusted extrinsic parameter information based on the mapping relationship between the point cloud and the image.

[0151] In one embodiment, the above-mentioned position information change value 11 includes: a relative angle change value and a relative position change value; then the above-mentioned change value acquisition module is specifically used to add the relative angle change value to the rotation matrix in the external reference information, and to add the relative position change value to the translation matrix in the external reference information.

[0152] In one embodiment, the first sensor is a camera device, and the fault sensor and the second sensor are both laser radars; then the calibration module 12 includes:

[0153] A mapping unit, used to map the point cloud data collected by the second sensor onto the image according to the mapping relationship between the point cloud and the image and the adjusted external parameter information, so as to obtain a point cloud plane image;

[0154] A construction unit, used to construct a loss function related to the external parameter information based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel point in the point cloud plane image;

[0155] The optimization unit is used to optimize the loss function until the loss function meets the preset optimization termination condition and obtain the external parameter information corresponding to the termination of the loss function optimization.

[0156] In one embodiment, the mapping unit includes:

[0157] An acquisition subunit, used to acquire the coordinates of the point cloud points in the point cloud data in the world coordinate system and the internal reference information of the camera device;

[0158] A calculation subunit is used to substitute the adjusted external parameter information, the internal parameter information of the camera device, and the coordinates of the point cloud points in the point cloud data in the world coordinate system into the mapping relationship between the point cloud and the image, and calculate the coordinates of the point cloud points in the point cloud data in the pixel coordinate system;

[0159] The subunit is determined to obtain a point cloud plane image according to the coordinates of the point cloud points in the point cloud data in the pixel coordinate system.

[0160] In one embodiment, the building block comprises:

[0161] A histogram subunit, used to construct a reflectivity histogram according to the reflectivity of the point cloud in the point cloud data, and to construct a gray value histogram according to the gray values ​​of the pixels in the point cloud plane image;

[0162] A joint subunit, used for constructing a joint histogram according to the reflectance histogram and the gray value histogram;

[0163] A subunit is constructed to construct a loss function related to external parameter information based on reflectance histogram, gray value histogram and joint histogram.

[0164] In one embodiment, the above-mentioned construction subunit is specifically used to calculate the reflectivity edge probability distribution according to the reflectivity histogram, calculate the grayscale value edge probability distribution according to the grayscale value histogram, and calculate the joint probability distribution according to the joint histogram; and construct a loss function related to the external parameter information according to the reflectivity edge probability distribution, the grayscale value edge probability distribution, and the joint probability distribution.

[0165] In one embodiment, the above-mentioned optimization unit is specifically used to adjust the value of the loss function using a preset gradient descent algorithm until the value of the loss function meets a preset optimization termination condition, and obtain the external parameter information corresponding to the termination of the loss function optimization.

[0166] In one embodiment, the above optimization termination condition includes: the value of the loss function is less than a preset threshold or the continuous change rate of the value of the loss function is within a preset range.

[0167] For the specific definition of the parameter calibration device, please refer to the definition of the parameter calibration method above, which will not be repeated here. Each module in the above parameter calibration device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0168] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 1aAs shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a parameter calibration method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the computer device housing, or an external keyboard, touch pad or mouse, etc.

[0169] Those skilled in the art will understand that Figure 1a The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0170] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0171] Acquire first position information between the first sensor and the faulty sensor, and second position information between the first sensor and the second sensor; the second sensor is a sensor that replaces the faulty sensor;

[0172] Adjusting the external reference information between the first sensor and the second sensor according to the position information change value between the first position information and the second position information;

[0173] Based on the mapping relationship between the point cloud and the image, the adjusted extrinsic parameter information is calibrated.

[0174] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0175] According to the change value between the first position information and the second position information, the external parameter information between the first sensor and the second sensor is adjusted, including:

[0176] The relative angle change value is added to the rotation matrix in the external parameter information, and the relative position change value is added to the translation matrix in the external parameter information.

[0177] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0178] According to the mapping relationship between the point cloud and the image and the adjusted external parameter information, the point cloud data collected by the second sensor is mapped onto the image to obtain a point cloud plane image;

[0179] Based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel points in the point cloud plane image, a loss function related to the external parameter information is constructed;

[0180] Optimize the loss function until the loss function meets the preset optimization termination condition, and obtain the external parameter information corresponding to the termination of the loss function optimization.

[0181] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0182] Obtain the coordinates of the point cloud points in the point cloud data in the world coordinate system and the internal reference information of the camera device;

[0183] Substitute the adjusted external parameter information, the internal parameter information of the camera device, and the coordinates of the point cloud points in the point cloud data in the world coordinate system into the mapping relationship between the point cloud and the image, and calculate the coordinates of the point cloud points in the point cloud data in the pixel coordinate system;

[0184] According to the coordinates of the point cloud points in the point cloud data in the pixel coordinate system, a point cloud plane image is obtained.

[0185] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0186] A reflectivity histogram is constructed according to the reflectivity of the point cloud in the point cloud data, and a gray value histogram is constructed according to the gray values ​​of the pixels in the point cloud plane image;

[0187] Constructing a joint histogram based on the reflectance histogram and the gray value histogram;

[0188] The loss function related to the external parameter information is constructed based on the reflectance histogram, gray value histogram and joint histogram.

[0189] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0190] The reflectivity edge probability distribution is calculated according to the reflectivity histogram, the gray value edge probability distribution is calculated according to the gray value histogram, and the joint probability distribution is calculated according to the joint histogram;

[0191] The loss function related to the external parameter information is constructed according to the reflectivity edge probability distribution, gray value edge probability distribution and joint probability distribution.

[0192] In one embodiment, when the processor executes the computer program, the following steps are also implemented: using a preset gradient descent algorithm to adjust the value of the loss function until the value of the loss function meets a preset optimization termination condition, and obtaining the external parameter information corresponding to the termination of the loss function optimization.

[0193] In one embodiment, the above optimization termination condition includes: the value of the loss function is less than a preset threshold or the continuous change rate of the value of the loss function is within a preset range.

[0194] The computer device provided in the above embodiment has an implementation principle and technical effects similar to those of the above method embodiment, which will not be described in detail here.

[0195] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0196] Acquire first position information between the first sensor and the faulty sensor, and second position information between the first sensor and the second sensor; the second sensor is a sensor that replaces the faulty sensor;

[0197] Adjusting the external reference information between the first sensor and the second sensor according to the position information change value between the first position information and the second position information;

[0198] Based on the mapping relationship between the point cloud and the image, the adjusted extrinsic parameter information is calibrated.

[0199] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0200] According to the change value between the first position information and the second position information, the external parameter information between the first sensor and the second sensor is adjusted, including:

[0201] The relative angle change value is added to the rotation matrix in the external parameter information, and the relative position change value is added to the translation matrix in the external parameter information.

[0202] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0203] According to the mapping relationship between the point cloud and the image and the adjusted external parameter information, the point cloud data collected by the second sensor is mapped onto the image to obtain a point cloud plane image;

[0204] Based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel points in the point cloud plane image, a loss function related to the external parameter information is constructed;

[0205] Optimize the loss function until the loss function meets the preset optimization termination condition, and obtain the external parameter information corresponding to the termination of the loss function optimization.

[0206] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0207] Obtain the coordinates of the point cloud points in the point cloud data in the world coordinate system and the internal reference information of the camera device;

[0208] Substitute the adjusted external parameter information, the internal parameter information of the camera device, and the coordinates of the point cloud points in the point cloud data in the world coordinate system into the mapping relationship between the point cloud and the image, and calculate the coordinates of the point cloud points in the point cloud data in the pixel coordinate system;

[0209] According to the coordinates of the point cloud points in the point cloud data in the pixel coordinate system, a point cloud plane image is obtained.

[0210] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0211] A reflectivity histogram is constructed according to the reflectivity of the point cloud in the point cloud data, and a gray value histogram is constructed according to the gray values ​​of the pixels in the point cloud plane image;

[0212] Constructing a joint histogram based on the reflectance histogram and the gray value histogram;

[0213] The loss function related to the external parameter information is constructed based on the reflectance histogram, gray value histogram and joint histogram.

[0214] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0215] The reflectivity edge probability distribution is calculated according to the reflectivity histogram, the gray value edge probability distribution is calculated according to the gray value histogram, and the joint probability distribution is calculated according to the joint histogram;

[0216] The loss function related to the external parameter information is constructed according to the reflectivity edge probability distribution, gray value edge probability distribution and joint probability distribution.

[0217] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: using a preset gradient descent algorithm to adjust the value of the loss function until the value of the loss function meets a preset optimization termination condition, and obtaining the external parameter information corresponding to the termination of the loss function optimization.

[0218] In one embodiment, the above optimization termination condition includes: the value of the loss function is less than a preset threshold or the continuous change rate of the value of the loss function is within a preset range.

[0219] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0220] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0221] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0222] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A parameter calibration method, characterized in that: The method comprises: Acquire first position information between a first sensor and a faulty sensor, and second position information between the first sensor and a second sensor; the second sensor is a sensor that replaces the faulty sensor; one of the first sensor and the second sensor is a camera device, and the other is a laser radar; adjusting external reference information between the first sensor and the second sensor according to a position information change value between the first position information and the second position information; Based on the mapping relationship between the point cloud and the image, the adjusted extrinsic parameter information is calibrated.

2. The method according to claim 1, characterized in that The position information change value includes: a relative angle change value and a relative position change value; Then, adjusting the external reference information between the first sensor and the second sensor according to the change value between the first position information and the second position information includes: The relative angle change value is added to the rotation matrix in the external parameter information, and the relative position change value is added to the translation matrix in the external parameter information.

3. The method according to claim 2, characterized in that The first sensor is a camera device, and the fault sensor and the second sensor are both laser radars; Then, the adjusted external parameter information is calibrated according to the mapping relationship between the point cloud and the image, including: According to the mapping relationship between the point cloud and the image and the adjusted external parameter information, the point cloud data collected by the second sensor is mapped onto the image to obtain a point cloud plane image; Constructing a loss function related to the extrinsic parameter information based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel point in the point cloud plane image; The loss function is optimized until the loss function meets a preset optimization termination condition, and the external parameter information corresponding to the termination of the loss function optimization is obtained.

4. The method according to claim 3, characterized in that The step of mapping the point cloud data collected by the second sensor onto the image according to the mapping relationship between the point cloud and the image and the adjusted external parameter information to obtain a point cloud plane image includes: Obtaining coordinates of the point cloud points in the point cloud data in the world coordinate system and the internal reference information of the camera device; Substituting the adjusted external parameter information, the internal parameter information of the camera device, and the coordinates of the point cloud points in the point cloud data in the world coordinate system into the mapping relationship between the point cloud and the image, and calculating the coordinates of the point cloud points in the point cloud data in the pixel coordinate system; The point cloud plane image is obtained according to the coordinates of the point cloud points in the point cloud data in the pixel coordinate system.

5. The method according to claim 3, characterized in that: The step of constructing a loss function related to the external parameter information based on the reflectivity of the point cloud in the point cloud data and the grayscale value of the pixel in the point cloud plane image includes: Constructing a reflectivity histogram according to the reflectivity of the point cloud in the point cloud data, and constructing a gray value histogram according to the gray values ​​of the pixel points in the point cloud plane image; Constructing a joint histogram according to the reflectivity histogram and the gray value histogram; A loss function related to the extrinsic parameter information is constructed based on the reflectivity histogram, the gray value histogram and the joint histogram.

6. The method according to claim 5, characterized in that The step of constructing a loss function related to the external parameter information based on the reflectivity histogram, the gray value histogram and the joint histogram includes: Calculating the reflectivity edge probability distribution according to the reflectivity histogram, calculating the gray value edge probability distribution according to the gray value histogram, and calculating the joint probability distribution according to the joint histogram; A loss function related to the extrinsic parameter information is constructed according to the reflectivity edge probability distribution, the gray value edge probability distribution, and the joint probability distribution.

7. The method according to claim 3, characterized in that The optimizing the loss function until the loss function satisfies a preset optimization termination condition, and obtaining the external parameter information corresponding to the termination of the optimization of the loss function, includes: The preset gradient descent algorithm is used to adjust the value of the loss function until the value of the loss function meets the preset optimization termination condition, and the external parameter information corresponding to the termination of the loss function optimization is obtained.

8. The method according to claim 7, characterized in that The optimization termination condition includes: the value of the loss function is less than a preset threshold or the continuous change rate of the value of the loss function is within a preset range.

9. A parameter calibration device, characterized in that: The device comprises: a position information acquisition module, used to acquire first position information between a first sensor and a faulty sensor, and second position information between the first sensor and a second sensor; the second sensor is a sensor that replaces the faulty sensor; one of the first sensor and the second sensor is a camera device, and the other is a laser radar; a change value acquisition module, configured to adjust the external reference information between the first sensor and the second sensor according to a position information change value between the first position information and the second position information; The calibration module is used to calibrate the adjusted external parameter information based on the mapping relationship between the point cloud and the image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images