Calibration method for external parameters of vehicle sensor

Abstract

The invention provides a calibration method and device for external parameters of a vehicle sensor, electronic equipment, a storage medium and a computer program product. According to the embodiment of the invention, a front vehicle hanging box and a detection pose of the front vehicle hanging box under a self-vehicle coordinate system are determined according to initial external parameters of the vehicle sensor; determining a first global pose of the front vehicle according to the detection pose and a hanging box angle between the head of the front vehicle and a hanging box of the front vehicle; the method comprises the following steps: acquiring a second global pose of a front vehicle, performing fusion calculation on a first global pose and the second global pose by adopting Kalman filtering, and determining a third global pose of the front vehicle; determining an external parameter correction value of the vehicle sensor according to the third global pose and the initial external parameter of the front vehicle; and finally, determining the calibrated external parameters of the vehicle sensor according to the initial external parameters of the vehicle sensor and the external parameter correction value of the vehicle sensor. Therefore, online calibration of the vehicle sensor can be realized through front and rear vehicle positioning, and the accuracy and timeliness of sensor calibration are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle sensors, and particularly relates to a vehicle sensor extrinsic parameter calibration method and device, an electronic device, a storage medium and a computer program product. BACKGROUND

[0002] With the development of automatic driving technology, applying automatic driving technology to vehicle platooning is one of the important application modes in the future automatic driving field. Automatic driving of vehicle platooning refers to platooning driving by two or more vehicles arranged longitudinally, and keeping a certain distance (generally a short distance of 10 meters or less) between each adjacent two vehicles to dynamically form a vehicle platoon for automatic driving. Taking truck platooning as an example, generally, a driver drives the front vehicle, and the following vehicles adopt automatic driving, which can be called “following vehicles”. The vehicle arranged in front between each adjacent two vehicles can be called a front vehicle, and the other vehicle can be called a rear vehicle. During the automatic driving of vehicle platooning, high positioning consistency is required between the front vehicle and the rear vehicle, so that the rear vehicle can accurately follow the front vehicle for automatic driving. If the positioning of the two vehicles deviates, the actual control accuracy of the rear vehicle will not be enough, especially the lateral accuracy. This positioning consistency requires that the rear vehicle has accurate observation of the front vehicle, and this requires that the vehicle sensors (such as front radar and multiple cameras) in the rear vehicle for observing the front vehicle have accurate calibration information, and the vehicle sensors of the following vehicles need to be kept in a fixed position. However, the truck may shake seriously and have large left-right displacement during driving, which may cause the position of the vehicle sensor to deviate. Especially after the truck drives for a long time, the installation angle of the vehicle sensor hardware may change. Therefore, in order to ensure the accuracy of the positioning of the vehicle sensor, the extrinsic parameters of the vehicle sensor need to be calibrated frequently, that is, the installation position and angle of the vehicle sensor in the ego vehicle coordinate system (the coordinate system with the center of the rear axle of the ego vehicle as the coordinate origin, the front direction of the vehicle as the x direction, and the left as the y direction) are obtained, so that the observation information of the vehicle sensor can be converted to the ego vehicle coordinate system according to the extrinsic parameters of the vehicle sensor.

[0003] In some cases, the radar and camera sensors of the vehicle are usually calibrated by a calibration room when the vehicle is refitted. For the problem of vehicle sensor extrinsic parameter calibration caused by the change of the position of the vehicle sensor, a regular recalibration method is usually used. Such a calibration method needs to recalibrate the vehicle offline regularly, which has two obvious shortcomings. One is that recalibrating the vehicle by the calibration room consumes a lot of manpower and resources. The other is that the change of the vehicle sensor cannot be found in time and recalibrated in time. The time of calibration update depends on the period of recalibration, and there is still a risk of inaccurate positioning of the vehicle sensor. SUMMARY

[0004] Embodiments of the present application provide a vehicle sensor extrinsic calibration method, device, electronic equipment, storage medium and computer program product to solve one or more technical problems.

[0005] In a first aspect, the embodiments of the present application provide a vehicle sensor extrinsic calibration method applied to a following vehicle in a vehicle platoon. The method comprises: detecting and processing vehicle data perceived by the ego vehicle according to initial extrinsics of a vehicle sensor to determine a preceding vehicle box and a detection pose of the preceding vehicle box in a coordinate system of the ego vehicle; determining a global pose of a head of the preceding vehicle according to the detection pose and a box angle between the head of the preceding vehicle and the preceding vehicle box, and determining a first global pose of the preceding vehicle based on the global pose of the head; obtaining a second global pose of the preceding vehicle, performing fusion calculation on the first global pose and the second global pose by Kalman filtering to determine a third global pose of the preceding vehicle; determining a first extrinsic correction value of the vehicle sensor according to the third global pose of the preceding vehicle and the initial extrinsics; and determining calibrated extrinsics of the vehicle sensor according to the initial extrinsics of the vehicle sensor and the first extrinsic correction value.

[0006] In a second aspect, the embodiments of the present application provide a vehicle sensor extrinsic calibration device applied to a following vehicle in a vehicle platoon. The device comprises: a box pose determination module configured to detect and process vehicle data perceived by the ego vehicle according to initial extrinsics of a vehicle sensor to determine a preceding vehicle box and a detection pose of the preceding vehicle box in a coordinate system of the ego vehicle; a preceding vehicle pose determination module configured to determine a global pose of a head of the preceding vehicle according to the detection pose and a box angle between the head of the preceding vehicle and the preceding vehicle box, and determine a first global pose of the preceding vehicle based on the global pose of the head; an extrinsic correction value determination module configured to obtain a second global pose of the preceding vehicle, perform fusion calculation on the first global pose and the second global pose by Kalman filtering to determine a third global pose of the preceding vehicle, and determine a first extrinsic correction value of the vehicle sensor according to the third global pose of the preceding vehicle and the initial extrinsics; and a sensor extrinsic calibration module configured to determine calibrated extrinsics of the vehicle sensor according to the initial extrinsics of the vehicle sensor and the first extrinsic correction value.

[0007] In a third aspect, the embodiments of the present application provide an electronic device comprising a memory, a processor and a computer program stored in the memory. The processor implements any of the above methods when executing the computer program.

[0008] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium having a computer program stored therein. The computer program is executed by a processor to implement any of the above methods.

[0009] Fifthly, embodiments of this application provide a computer program product, which includes a computer program / instruction that, when executed by a processor, implements the method described in any of the above-mentioned embodiments.

[0010] According to the embodiments of this application, firstly, the vehicle data perceived by the vehicle is detected and processed based on the initial extrinsic parameters of the vehicle sensors to determine the detection pose of the trailer of the preceding vehicle and the trailer in the vehicle coordinate system; based on the detection pose and the trailer angle between the front of the preceding vehicle and the trailer, the global pose of the front of the preceding vehicle is determined, and the first global pose of the preceding vehicle is determined based on the global pose of the front of the preceding vehicle; the second global pose of the preceding vehicle is obtained, and Kalman filtering is used to fuse the first global pose and the second global pose to determine the third global pose of the preceding vehicle; based on the third global pose of the preceding vehicle and the initial extrinsic parameters, the first extrinsic parameter correction value of the vehicle sensors is determined; based on the initial extrinsic parameters of the vehicle sensors and the first extrinsic parameter correction value of the vehicle sensors, the calibrated extrinsic parameters of the vehicle sensors are determined. This approach combines the perception results of the following vehicle on the vehicle in front with the positioning information of both vehicles to comprehensively solve for the position and attitude of the vehicle in front. This allows for the calibration of the following vehicle's sensors. This online calibration of the following vehicle's sensors, based on joint positioning of the two vehicles, significantly improves the accuracy of sensor calibration and the consistency of positioning between the two vehicles, while reducing the manpower required for offline calibration. Furthermore, after obtaining the global pose of the vehicle in front detected by the vehicle itself, the following vehicle detects and calculates the global pose of the vehicle in front using its own sensor extrinsic parameters. By fusing these two methods of global pose data from multiple sources, compensation values ​​for the following vehicle's sensor calibration correction information can be obtained. This allows for the detection of issues such as sensor swaying or data anomalies (i.e., anomalies can be identified by comparing the diagonal value of the covariance matrix of the vehicle sensor calibration correction information with a preset threshold), enabling timely anomaly alarms and further calibrating the vehicle sensors.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0012] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0013] Figure 1 This illustration shows a scenario in which the global pose of the vehicle in front is calculated from the vehicle information obtained by the vehicle itself in a vehicle sensor extrinsic parameter calibration scheme provided in an embodiment of this application.

[0014] Figure 2 A flowchart illustrating a vehicle sensor extrinsic parameter calibration scheme provided in an embodiment of this application is shown.

[0015] Figure 3 A flowchart of a method for calibrating extrinsic parameters of a vehicle sensor provided in an embodiment of this application is shown;

[0016] Figure 4 This paper shows a structural block diagram of a vehicle sensor extrinsic parameter calibration device provided in an embodiment of this application;

[0017] Figure 5 A block diagram of an electronic device used to implement embodiments of this application is shown; and

[0018] Figure 6 An internal structural diagram of a computer device used to implement embodiments of this application is shown. Detailed Implementation

[0019] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0020] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0021] Figure 1 This illustration shows a scenario in which the global pose of the preceding vehicle is calculated from vehicle information acquired by the vehicle itself, as provided in an embodiment of this application, for a vehicle sensor extrinsic parameter calibration scheme. For example... Figure 1 As shown, the vehicle sensor extrinsic parameter calibration scheme provided in this application embodiment can be applied to following vehicles in autonomous driving platooning. By using forward-facing vehicle sensors (such as LiDAR, cameras, etc.) positioned at the front of the following vehicle (the vehicle itself), the vehicle perceives its surrounding environment and obtains the perceived vehicle data. This perceived data can include the poses of multiple vehicles around the vehicle in the global coordinate system. Furthermore, based on the initial extrinsic parameters (T...) of the vehicle sensors... s1 The system detects and processes vehicle data sensed by the vehicle, using the initial extrinsic parameters (T) of the vehicle sensors.s1 The sensed vehicle data is transformed into the vehicle's own vehicle sensor coordinate system (which can be considered the vehicle's own coordinate system). Since the vehicle in front is generally located directly in front of the vehicle, and in vehicle platooning scenarios, the distance between the vehicle in front and the vehicle is generally no more than 10 meters, during detection and processing, the vehicle data belonging to the vehicle in front can be first filtered out from the vehicle data acquired by the vehicle's own vehicle sensors based on the coordinate information contained in the vehicle data, and then the initial extrinsic parameters (T) of the vehicle sensors are used. s1 The vehicle data from the preceding vehicle is converted to the vehicle's own vehicle sensor coordinate system (which can be considered the vehicle's own coordinate system). Simultaneously, due to the positional relationship between the preceding and following vehicles, when both are trucks or vans, the vehicle data acquired by the following vehicle through the forward vehicle sensor is the vehicle data of the trailer, such as the pose of the trailer (including position information on the x, y, and z axes, as well as roll, pitch, and yaw angles, where yaw is the yaw angle of rotation around the Y-axis, pitch is the pitch angle of rotation around the X-axis, and roll is the roll angle of rotation around the Z-axis). Therefore, based on the initial extrinsic parameters (T...) of the vehicle sensor... s1 The system processes vehicle data sensed by the vehicle (e.g., detected images) to determine the trailer of the preceding vehicle and its detection pose in the vehicle's coordinate system (the detection pose of the preceding vehicle's trailer). 0→1 The vehicle sensor coordinate system (which can be considered as the vehicle coordinate system) can be a coordinate system with the rear axle center of the vehicle as the origin, the front direction of the vehicle as the x-direction, and the left direction as the y-direction.

[0022] Furthermore, based on the detected pose (T) 0→1 The global pose of the front of the vehicle is determined by the front of the vehicle and the trailer angle (θ) between the front of the vehicle and the trailer of the vehicle. Based on the global pose of the front of the vehicle, the first global pose (T3) of the vehicle is determined, which means the global pose of the vehicle calculated by the vehicle is obtained. Among them, the mounting angle (θ) between the front of the vehicle and the trailer of the vehicle can be sent from the front vehicle to the vehicle. After the vehicle obtains the mounting angle (θ) between the front of the vehicle and the trailer of the vehicle, it can solve the detection pose of the front of the vehicle in the vehicle coordinate system through the mounting angle (θ). Then, based on the transformation relationship between the global coordinate system and the vehicle coordinate system of the global pose of the vehicle detected by the vehicle (T0=(z0,y0,z0,roll0,pith0,yaw0)), the detection pose of the front of the vehicle in the vehicle coordinate system is converted into the global pose of the front of the vehicle. Then, the global pose of the front of the vehicle is determined as the first global pose (T3) of the front vehicle, which is the global pose of the front vehicle calculated by the vehicle.

[0023] Figure 2 This illustration shows a flowchart of a vehicle sensor extrinsic parameter calibration scheme provided in an embodiment of this application.Figure 2 As shown, with Figure 1 The diagram in the image corresponds to the first global pose (T3) of the vehicle in front, where the detection pose T3 of the vehicle in front is the same as the first global pose (T3) of the vehicle in front. Figure 2 The steps in the process before obtaining the detection pose T3 of the preceding vehicle are the same as those mentioned above. Figure 1 The description is the same, that is, based on the global pose of the vehicle detected by the vehicle (the global pose of the rear vehicle T0) and the detection pose of the trailer of the preceding vehicle in the vehicle's coordinate system obtained by the vehicle (the detection pose of the preceding vehicle T0). 0→1 = (x, y, z, roll, pitch, yaw): The relative pose of the trailer of the preceding vehicle and the following vehicle. Since the distance between the preceding and following vehicles is relatively short, it can be assumed that they are basically on the same plane during calculation. Therefore, it can be obtained by performing a two-dimensional transformation on only the values ​​of x, y, and yaw. Based on the initial extrinsic parameters (T) of the vehicle sensors... s1 The pose of the trailer of the preceding vehicle in the self-vehicle coordinate system is obtained (T2 = (x2, y2, z2, roll2, pith2, yaw2): the global pose of the trailer of the preceding vehicle calculated by the following vehicle based on the observation information), and then the pose is detected (T 0→1 The global pose of the front of the vehicle is determined by the coupling angle (θ) between the front of the vehicle and the trailer of the vehicle (coupling angle detection), and the first global pose of the front vehicle (front vehicle detection pose T3) is determined based on the global pose of the front of the vehicle.

[0024] Furthermore, the vehicle can communicate with the vehicle in front to obtain the second global pose of the vehicle in front, which it has detected and sent back to the vehicle (the global pose of the vehicle in front is T1 = (x1, y1, z1, roll1, pith1, yaw1): the pose of the vehicle in front's global positioning system). This results in two methods for obtaining the global pose of the vehicle in front: one is the global pose T3 obtained by the vehicle itself, and the other is the global pose T1 detected by the vehicle in front's own global positioning system. By fusing and optimizing these two methods—that is, the detected pose T3 and the global pose T1—a more accurate global pose T4 can be obtained. This fusion and optimization can be achieved using Kalman filtering, specifically by fusing the first global pose (T3) and the second global pose (T1) using Kalman filtering to determine the third global pose (T4) of the vehicle in front.

[0025] This enables autonomous vehicle platooning, taking trucks and other vans as an example, where the following vehicle uses its own LiDAR and cameras to detect the status of the trailer of the preceding vehicle in real time, and combines the global pose of the following vehicle to calculate the global pose of the trailer of the preceding vehicle. Then, by using the trailer angle detected by the preceding vehicle itself, the global pose of the preceding vehicle based on the observations of the following vehicle can be calculated. Figure 2(The preceding vehicle's detected pose T3). On the other hand, the preceding vehicle itself also has global positioning capabilities (such as a satellite positioning system). Therefore, the preceding vehicle can also send a global pose T1 detected by the global positioning system to the following vehicle. By fusing and optimizing the preceding vehicle poses obtained in these two ways, that is, the preceding vehicle detected pose T3 and the preceding vehicle global pose T1, a more accurate preceding vehicle global pose T4 can be obtained.

[0026] When the following vehicle uses its own LiDAR and camera sensors to detect the status of the trailer of the preceding vehicle, it needs to use the extrinsic parameters of the LiDAR and camera sensors to convert the observation results of the vehicle sensors into the vehicle's coordinate system. As mentioned earlier, due to factors such as shaking during vehicle movement, the extrinsic parameters of these sensors may be inaccurate. Therefore, a correction value can be added to the initial extrinsic parameters of the preceding vehicle's forward-facing vehicle sensors. This correction value is optimized during the fusion of the detection pose and the global pose. That is, it can be based on the third global pose (T4) of the preceding vehicle and the initial extrinsic parameters (T... s1 ), determine the first extrinsic parameter correction value (T) of the vehicle sensor. Δ Finally, based on the initial extrinsic parameters (T) of the vehicle sensors... s1 ) and the first extrinsic parameter correction value (T) of the vehicle sensor Δ ), determine the calibrated vehicle sensor extrinsic parameters (T) s2 ).

[0027] Furthermore, the process of obtaining the extrinsic parameter correction value described above can be repeated multiple times, based on the calibrated vehicle sensor extrinsic parameters (T). s2 The vehicle data perceived by the vehicle is processed and filtered to identify vehicles belonging to the preceding vehicle. The perceived vehicle data includes the poses of at least one vehicle surrounding the vehicle. The process of determining the trailer of the preceding vehicle and its detection pose (T) in the vehicle's coordinate system is repeated. 0→1 The process of determining the third global pose (T4) of the preceding vehicle must be performed at least once; based on the third global pose (T4) of the preceding vehicle and the calibrated vehicle sensor extrinsic parameters (T... s2 ), to obtain at least one second extrinsic parameter correction value (T) Δ ); based on at least one second extrinsic parameter correction value (T) Δ The comparison between the compensation value and the preset threshold generates a detection result indicating whether the vehicle sensor is abnormal or normal. At least one second extrinsic parameter correction value (T) is used. Δ If the compensation value of the vehicle sensor is less than or equal to a preset threshold, a normal detection result for the vehicle sensor is generated; and the initial extrinsic parameters (T) of the vehicle sensor are set. s1 ) and the second extrinsic parameter correction value (T) of the vehicle sensor Δ The product of ) is used as the calibrated vehicle sensor extrinsic parameter (T). s2Among them, the vehicle sensor extrinsic parameters (T) after the previous calibration. s2 It can also be converted into the initial extrinsic parameters (T) of the vehicle sensors. s1 From this, we can obtain the corrected extrinsic parameter value (T) after multiple optimizations. Δ After it converges to a stable value, a relatively accurate vehicle sensor extrinsic parameter (T) is obtained. s2 In this process, the extrinsic parameter correction values ​​and vehicle sensor extrinsic parameters obtained each time can be stored in the cloud. When the autonomous vehicle (self-driving vehicle) starts, the latest vehicle sensor extrinsic parameters can be retrieved from the cloud, and the new extrinsic parameter correction values ​​and vehicle sensor extrinsic parameters obtained during the operation of the self-driving vehicle can be uploaded to the cloud. The extrinsic parameter correction values ​​and vehicle sensor extrinsic parameters are automatically updated in the cloud.

[0028] The execution entity in this application embodiment can be an application, service, instance, functional module in software form, virtual machine (VM), container, or cloud server, or hardware device with data processing function (such as server or terminal device) or hardware chip (such as CPU, GPU, FPGA, NPU, AI accelerator card, or DPU). The device for calibrating vehicle sensor extrinsic parameters can be deployed on the computing device of the application provider offering the corresponding service or on a cloud computing platform providing computing power, storage, and network resources. The cloud computing platform can provide services in the following modes: IaaS (Infrastructure as a Service), PaaS (Platform as a Service), SaaS (Software as a Service), or DaaS (Data as a Service). Taking a platform providing SaaS (Software as a Service) as an example, the cloud computing platform can utilize its own computing resources to provide training for the vehicle sensor extrinsic parameter calibration model or the execution of the vehicle sensor extrinsic parameter calibration module. The specific application architecture can be built according to service requirements. For example, the platform can provide construction services based on the above model to application parties or individuals using platform resources, and further call the above model and implement online or offline vehicle sensor extrinsic parameter calibration functions based on vehicle sensor extrinsic parameter calibration requests submitted by relevant client or server devices.

[0029] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0030] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0031] This application provides a method for calibrating the extrinsic parameters of vehicle sensors, applicable to following vehicles in a vehicle platoon, such as... Figure 3 The diagram shows a flowchart of a vehicle sensor extrinsic parameter calibration method 300 according to an embodiment of this application. The method 300 may include:

[0032] In step S301, based on the initial extrinsic parameters (T) of the vehicle sensors... s1 The system detects and processes vehicle data perceived by the vehicle itself to determine the detection pose (T) of the trailer of the preceding vehicle and the trailer in the vehicle's coordinate system. 0→1 ).

[0033] The vehicle sensor extrinsic parameter calibration scheme provided in this application embodiment can be applied to following vehicles in autonomous driving platooning. By using forward-facing vehicle sensors (such as LiDAR, cameras, etc.) positioned at the front of the following vehicle (the vehicle itself), the vehicle perceives its surrounding environment and obtains perceived vehicle data. This perceived data can include the poses of multiple vehicles around the vehicle in the global coordinate system. Furthermore, based on the initial extrinsic parameters (T...) of the vehicle sensors... s1 The system detects and processes vehicle data sensed by the vehicle, using the initial extrinsic parameters (T) of the vehicle sensors. s1 The sensed vehicle data is transformed into the vehicle's own vehicle sensor coordinate system (which can be considered the vehicle's own coordinate system). Since the vehicle in front is generally located directly in front of the vehicle, and in vehicle platooning scenarios, the distance between the vehicle in front and the vehicle is generally no more than 10 meters, during detection and processing, the vehicle data belonging to the vehicle in front can be first filtered out from the vehicle data acquired by the vehicle's own vehicle sensors based on the coordinate information contained in the vehicle data, and then the initial extrinsic parameters (T) of the vehicle sensors are used. s1The vehicle data from the preceding vehicle is converted to the vehicle's own vehicle sensor coordinate system (which can be considered the vehicle's own coordinate system). Simultaneously, due to the positional relationship between the preceding and following vehicles, when both are trucks or vans, the vehicle data acquired by the following vehicle through the forward vehicle sensor is the vehicle data of the trailer, such as the pose of the trailer (including position information on the x, y, and z axes, as well as roll, pitch, and yaw angles, where yaw is the yaw angle of rotation around the Y-axis, pitch is the pitch angle of rotation around the X-axis, and roll is the roll angle of rotation around the Z-axis). Therefore, based on the initial extrinsic parameters (T...) of the vehicle sensor... s1 The system detects and processes vehicle data perceived by the vehicle itself to determine the trailer of the preceding vehicle and its detection pose in the vehicle's coordinate system (preceding vehicle detection T). 0→1 ).

[0034] For example, the initial extrinsic parameters (T) of the vehicle sensors s1 The data can be stored in the local storage space of the vehicle sensors in the following vehicles of the autonomous driving platoon, or it can be stored in the cloud. When used, it can be temporarily called from the cloud in real time, or it can be pre-set in the local storage space. This application does not impose any restrictions on this.

[0035] Both the lead vehicle and the trailing vehicle can be equipped with a global positioning system (GPS). This GPS can be used to collect the real-time global pose of the respective vehicles. The GPS can be, for example, a satellite navigation system or a positioning system based on high-precision map matching. After the lead vehicle and the trailing vehicle are powered on, they can communicate through communication modules installed on both vehicles. For example, the lead vehicle can collect its own global pose in real time through its GPS, and the current global pose acquired by the lead vehicle can be recorded as the second global pose (T1). This second global pose (T1) can then be sent to the communication module of the trailing vehicle through its communication module, so that the trailing vehicle can receive and process the second global pose (T1) of the lead vehicle. Similarly, the trailing vehicle can also collect its own global pose in real time through its GPS, such as the current global pose (T0) acquired by the trailing vehicle.

[0036] It should be noted that the global poses acquired by the preceding vehicle and the vehicle itself can be in a global coordinate system. Typically, conversion between the global coordinate system and the vehicle's own coordinate system is possible; for example, the global poses of each vehicle can be converted to the respective coordinate system of the preceding vehicle or the vehicle itself. The vehicle's coordinate system can be set according to actual conditions. For example, it can be a coordinate system established with the vehicle's center as the origin and the length, width, and height directions as the three coordinate directions; or it can be a coordinate system with the rear axle center of the vehicle as the origin, the front direction as the x-axis, and the left direction as the y-axis. This application does not impose any restrictions on this.

[0037] Since the vehicle coordinate system (the vehicle coordinate system of the preceding vehicle) and the vehicle coordinate system (the vehicle coordinate system of the following vehicle) are not the same, in order to facilitate the consistency of positioning between the preceding and following vehicles, the vehicle can transform the second global pose (T1) of the preceding vehicle to the vehicle coordinate system of the following vehicle. In specific transformations, for example, the preceding vehicle can send relevant parameters of its own coordinate system to its own vehicle, which then transforms the second global pose (T1) of the preceding vehicle into its own vehicle coordinate system. At the same time, it also transforms the global pose (T0) of the following vehicle into its own vehicle coordinate system. Then, the transformation relationship between the vehicle coordinate systems of the preceding and following vehicles is used to transform the pose of the preceding vehicle into its own vehicle coordinate system and then into the vehicle coordinate system of the following vehicle. Alternatively, assuming that the second global pose (T1) of the preceding vehicle and the global pose (T0) of the following vehicle can both carry relevant information about their respective global and vehicle coordinate systems, mathematical operations can be directly performed on the second global pose (T1) of the preceding vehicle and the global pose (T0) of the following vehicle. For example, multiplying the inverse transformation matrix of the global pose (T0) of the following vehicle by the second global pose (T1) will transform the second global pose (T1) of the preceding vehicle into the vehicle coordinate system of the following vehicle.

[0038] Furthermore, both the aforementioned second global pose (T1) and the vehicle's global pose (T0) can be represented using multi-dimensional vectors. For example, a global pose composed of position and multi-dimensional angles can be used. For instance, the vehicle's global pose can be represented by T0, specifically as: T0 = (x0, y0, z0, roll0, pith0, yaw0); the second global pose can be represented by T1, specifically as: T1 = (x1, y1, z1, roll1, pith1, yaw1). Other poses involved in this application also involve similar representations. These poses include position information along the x, y, and z axes, as well as angle information for roll, pitch, and yaw. Here, yaw represents the yaw angle of rotating the object around the Y-axis, pitch represents the pitch angle of rotating the object around the X-axis, and roll represents the roll angle of rotating the object around the Z-axis.

[0039] Taking a truck or similar van as an example, the truck cab and trailer are generally connected by a flexible connection. Both the truck and the vehicle can have their own cab and trailer. The global poses of the truck and the vehicle can be the global poses of their respective cabs and / or trailers, and this application does not impose any restrictions on the specific determination method. Both the truck and the vehicle can be equipped with various vehicle sensors such as LiDAR and cameras. After powering on, the vehicle can use these sensors to perceive and collect surrounding environmental data. This environmental data can include vehicle data around the vehicle, such as vehicle data from the truck in front and other vehicles. This vehicle data can be in the form of point cloud data, RGB image data, etc., and this application does not impose any restrictions on this. Since the vehicle data mentioned above is perceived by the vehicle itself, meaning it's collected in its own coordinate system, the vehicle can use the transformed pose of the preceding vehicle in its own coordinate system to detect the preceding vehicle's data. For example, image recognition or detection methods can be used to process point cloud data and RGB images, detecting the trailer and its detection pose from the point cloud data and RGB images. Because the vehicle is behind the preceding vehicle, it typically cannot detect the front of the preceding vehicle, only the trailer. Therefore, the vehicle data perceived by the vehicle usually includes data about the preceding vehicle's trailer, thus obtaining its detection pose. Similarly, the detection pose of the preceding vehicle's trailer can be represented using multi-dimensional vectors, such as a global pose composed of position and multi-dimensional angles. The detection pose of the preceding vehicle's trailer can be represented using T... 0→1 Specifically, it can be represented as: T 0→1 = (x,y,z,roll,pitch,yaw), which is the relative pose of the trailer of the preceding vehicle and the following vehicle. Since the distance between the preceding vehicle and the following vehicle is relatively close, it can be assumed that the two are basically on the same plane during the calculation. Therefore, it can be obtained by performing a two-dimensional transformation on only the values ​​of x,y,yaw.

[0040] In step S302, according to the detected pose (T) 0→1 The global pose of the front vehicle is determined by the front vehicle's front end and the trailer angle (θ) between the front vehicle's front end and the trailer of the front vehicle, and the first global pose (T3) of the front vehicle is determined based on the global pose of the front vehicle's front end.

[0041] After obtaining the detection pose (T) of the trailer of the preceding vehicle. 0→1After that, the global pose of the foreve vehicle's front end can be determined based on the geometric positional relationship between the foreve vehicle's front end and its trailer, such as the trailer angle (θ). Then, the first global pose (T3) of the foreve vehicle is determined based on this global pose. Specifically, the foreve vehicle can detect the geometric positional relationship between its own front end and the trailer in real time, and then send this relationship to the communication module of the assisted vehicle through its own transmission module. This allows the assisted vehicle to obtain the real-time geometric positional relationship of the foreve vehicle. This relationship can include the distance between the foreve vehicle's front end and the trailer, the angle between them, etc.

[0042] The hook-up angle between the front of the preceding vehicle and its trailer can be denoted as θ. After the autonomous vehicle obtains the hook-up angle sent by the preceding vehicle, it can also obtain the length of the trailer sent by the preceding vehicle. Then, the autonomous vehicle's automatic driving system can determine the length of the trailer, the hook-up angle, and the detection pose (T) of the trailer. 0→1 Mathematical calculations are performed to determine the detection pose of the front of the vehicle. For example, trigonometric functions and the sine theorem can be used to determine the length of the trailer, the trailer angle, and the detection pose (T) of the trailer. 0→1 Mathematical calculations can be performed, and other auxiliary parameters can also be combined for calculation, as long as the detection pose of the front vehicle's front end can be calculated. Furthermore, the length of the trailer of the front vehicle can be transmitted from the front vehicle to the voluntary vehicle, or it can be calculated by the voluntary vehicle based on the perceived vehicle data of the front vehicle. Further, based on the transformation relationship between the global coordinate system and the voluntary vehicle's coordinate system of the voluntary vehicle's detected global pose (T0), the detection pose of the front vehicle's front end in the voluntary vehicle's coordinate system can be converted into the global pose of the front vehicle's front end, and then the global pose of the front vehicle's front end can be determined as the first global pose (T3) of the front vehicle, which is the global pose of the front vehicle calculated by the voluntary vehicle.

[0043] In this embodiment of the application, the detection pose (T) of the trailer of the preceding vehicle can also be achieved by first detecting the trailer's position (T) of the trailer. 0→1 After transformation to the global coordinate system, the global pose (T2) of the trailer of the preceding vehicle is obtained. Combined with the geometric positional relationship between the front of the preceding vehicle and the trailer, the global pose of the front of the preceding vehicle can be calculated. This method of calculation through coordinate transformation and geometric positional relationship is relatively simple and intuitive, thus improving the efficiency and accuracy of calculating the global pose of the front of the preceding vehicle. Alternatively, the global pose of the front of the preceding vehicle can be calculated using the length and angle of the trailer, as well as the global pose of the trailer. This process is relatively straightforward and simple, and considers more factors, further improving the efficiency and accuracy of calculating the global pose of the front of the preceding vehicle. Furthermore, the first global pose (T3) of the preceding vehicle can also be obtained by combining the global pose of the front of the preceding vehicle calculated by the vehicle itself with other positional information.

[0044] Similarly, the global pose of the vehicle's front end can also be represented using multi-dimensional vectors. For example, the global pose can be composed of position and multi-dimensional angles. For instance, the global pose of the trailer of the preceding vehicle can be represented by T2, specifically as: T2 = (x2, y2, z2, roll2, pith2, yaw2); the global pose of the front end of the preceding vehicle or the first global pose of the preceding vehicle can be represented by T3, specifically as: T3 = (x3, y3, z3, roll3, pith3, yaw3).

[0045] Furthermore, through the correlation calculations in the above steps, it can be seen that the preceding vehicle only needs to send its relatively simplified global pose to the following vehicle. The following vehicle can then accurately calculate the global pose of the preceding vehicle's front end based on this, combined with its own perceived vehicle data. Therefore, while achieving joint positioning of the preceding and following vehicles, it reduces the dependence of both vehicles on the accuracy of global positioning, improving the consistency of positioning between the two vehicles. It can also be seen that the method in this embodiment can maintain the consistency of positioning between the preceding and following vehicles even in situations where satellite signals (i.e., global positioning system signals) are poor, such as under bridges, toll booths, and tunnels.

[0046] In step S303, the second global pose (T1) of the preceding vehicle is obtained, and the first global pose (T3) and the second global pose (T1) are fused using Kalman filtering to determine the third global pose (T4) of the preceding vehicle; based on the third global pose (T4) of the preceding vehicle and the initial extrinsic parameters (T... s1 ), determine the first extrinsic parameter correction value (T) of the vehicle sensor. Δ ).

[0047] As mentioned earlier, the vehicle can communicate with the vehicle in front to obtain the second global pose (T1) of the vehicle in front, which it has detected and sent back to the vehicle. This results in two methods for obtaining the global pose of the vehicle in front: one is the global pose (T3) calculated by the vehicle itself, and the other is the global pose (T1) detected by the vehicle in front's own global positioning system. By fusing and optimizing these two methods, a more accurate global pose (T4) can be obtained. This fusion optimization can be achieved using Kalman filtering. Specifically, the first global pose (T3) and the second global pose (T1) are fused using Kalman filtering. Both the first and second global poses are then input into the Kalman filter for optimal solution, yielding the optimal global pose of the vehicle in front, thus determining the third global pose (T4).

[0048] In the aforementioned process, when the following vehicle uses its own LiDAR and camera sensors to detect the status of the trailer of the preceding vehicle, it needs to use the extrinsic parameters of the LiDAR and camera sensors to convert the observation results of the vehicle sensors into the vehicle's coordinate system. As mentioned earlier, due to factors such as shaking during vehicle movement, the extrinsic parameters of these sensors may be inaccurate. Therefore, a correction value can be added to the initial extrinsic parameters of the preceding vehicle's forward-facing vehicle sensors. This correction value is optimized during the fusion of the detection pose and the global pose. That is, it can be based on the third global pose (T4) of the preceding vehicle and the initial extrinsic parameters (T... s1 ), determine the first extrinsic parameter correction value (T) of the vehicle sensor. Δ Finally, based on the initial extrinsic parameters (T) of the vehicle sensors... s1 ) and the first extrinsic parameter correction value (T) of the vehicle sensor Δ ), determine the calibrated vehicle sensor extrinsic parameters (T) s2 ).

[0049] Kalman filtering is used to fuse the first global pose (T3) and the second global pose (T1) to determine the third global pose (T4) of the preceding vehicle. Based on the third global pose (T4) of the preceding vehicle and the initial extrinsic parameters (T... s1 Determine the first extrinsic parameter correction value (T) of the vehicle sensor. Δ The formula for calculating ) can be written as: T4 = T1T Δ Where P0 represents the pose covariance matrix of the leading vehicle's global positioning system; P1 represents the global pose covariance matrix of the following vehicle calculated based on observation information; and H represents the state transition matrix. T3, T1, T4, and T can be determined using this formula. s1 and T Δ The calculation relationship between them.

[0050] Similarly, the third global pose (T4) of the preceding vehicle and the initial extrinsic parameters (T4) of the vehicle's sensors obtained after the above fusion calculation are also considered. s1 ), First external parameter correction value (T) Δ ) and calibrated vehicle sensor extrinsic parameters (T) s2 It can also be represented using multi-dimensional vectors. For example, the global pose can be composed of position and multi-dimensional angles. For instance, the third global pose of the vehicle in front, obtained after fusion calculation, can be represented by T4, specifically: T4 = (x4, y4, z4, roll4, pith4, yaw4); the initial extrinsic parameters of the vehicle's sensors can be represented by T... s1 Specifically, it is represented as: T s1 =(xs1,ys1,z) s1 ,roll s1 ,piths1,yaw s1The first extrinsic parameter correction value of the vehicle's sensor is T. Δ Specifically, it is represented as: T Δ =(x Δ ,y Δ ,z Δ ,roll Δ ,pith Δ ,yaw Δ ); The external parameters of the vehicle sensors after self-calibration are adopted using T s2 Specifically, it is represented as: T s2 =(xs2,ys2,z) s2 ,roll s2 ,piths2,yaw s2 When the process of obtaining the extrinsic parameter correction value is repeated multiple times, the calibrated vehicle sensor extrinsic parameter (T) becomes... s2 This can be converted into the initial extrinsic parameters (T) of the vehicle sensors. s1 ).

[0051] In step S304, based on the initial extrinsic parameters (T) of the vehicle sensor... s1 ) and the first extrinsic parameter correction value (T) of the vehicle sensor Δ ), determine the calibrated vehicle sensor extrinsic parameters (T) s2 ).

[0052] In one possible implementation, the above is based on the initial extrinsic parameters (T) of the vehicle sensor. s1 ) and the first extrinsic parameter correction value (T) of the vehicle sensor Δ ), determine the calibrated vehicle sensor extrinsic parameters (T) s2 The method can be to use the initial extrinsic parameters (T) of the vehicle sensors. s1 ) and the first extrinsic parameter correction value (T) of the vehicle sensor Δ The product of ) is used as the calibrated vehicle sensor extrinsic parameter (T). s2 ).

[0053] Specifically, it can be represented as: T s2 =T s1 *T Δ Of course, it can also be determined through other calculation methods, and this application does not impose any restrictions on this.

[0054] In one possible implementation, the above-mentioned vehicle sensor extrinsic parameter calibration scheme may further include: based on the calibrated vehicle sensor extrinsic parameters (T... s2The system detects and processes the vehicle data sensed by the vehicle itself, filtering out vehicle data belonging to the preceding vehicle. The perceived vehicle data includes the poses of at least one vehicle surrounding the vehicle. The process then repeats the determination of the preceding vehicle's trailer and its detection pose (T) in the vehicle's coordinate system. 0→1 The process of determining the third global pose (T4) of the preceding vehicle is performed at least once; based on the third global pose (T4) of the preceding vehicle and the calibrated vehicle sensor extrinsic parameters (T... s2 ), to obtain at least one second extrinsic parameter correction value (T) Δ According to the at least one second extrinsic parameter correction value (T) Δ The comparison result between the compensation value and the preset threshold generates the detection result of whether the vehicle sensor is abnormal or normal.

[0055] In this embodiment of the application, the vehicle uses calibrated vehicle sensor extrinsic parameters (T) s2 The process of detecting and processing vehicle data perceived by the autonomous vehicle, and filtering out vehicle data belonging to the preceding vehicle from the perceived vehicle data, can be achieved using target recognition algorithms or target detection algorithms to identify the vehicles included in the detected vehicle data and obtain the identified candidate vehicles. Generally, after identifying or detecting each candidate vehicle, information such as the color, size, model, and pose of each candidate vehicle can also be obtained. The pose of each candidate vehicle can be recorded as a candidate pose. After obtaining the candidate poses of each candidate vehicle, the autonomous vehicle typically uses the calibrated vehicle sensor extrinsic parameters (T...) s2 Theoretically, the global pose of the preceding vehicle calculated by the detection system and the global pose transmitted from the preceding vehicle should be consistent or very close. Therefore, we can match each candidate pose with the second global pose (T1) transmitted from the preceding vehicle. From these candidate poses, we can find the one that matches the second global pose (T1) of the preceding vehicle and use it as the target pose. Simultaneously, we can find the candidate vehicle corresponding to this target pose; this candidate vehicle is the preceding vehicle. In the specific pose matching process, we can transform the second global pose (T1) of the preceding vehicle and each candidate pose to the same coordinate system. Then, we calculate the distance between each candidate pose and the second global pose (T1), obtaining multiple distance values. We then select the candidate pose with the smallest distance value. This candidate pose with the smallest distance value is the pose most closely related to the second global pose (T1), meaning they may be the pose of the same vehicle, and therefore can be used as the target pose.

[0056] Correspondingly, after the vehicle selects the preceding vehicle from the candidate vehicles, it can also obtain the target pose of the preceding vehicle. This target pose can include the pose of the preceding vehicle's trailer, which means the detection pose (T) of the preceding vehicle's trailer can be obtained. 0→1This method, by detecting perceived data and matching the transformed pose with the preceding vehicle, refines the process of determining the detection pose of the trailer of the preceding vehicle, making the final detection pose of the trailer of the preceding vehicle more accurate, and also improving the efficiency of determining the detection pose of the trailer of the preceding vehicle.

[0057] The detection pose (T) of the trailer of the vehicle in front is obtained from multiple vehicle data points. 0→1 After that, the process of determining the trailer of the preceding vehicle and its pose in the vehicle's coordinate system (T) can be repeated. 0→1 The process of determining the third global pose (T4) of the preceding vehicle must be performed at least once; based on the third global pose (T4) of the preceding vehicle and the calibrated vehicle sensor extrinsic parameters (T... s2 ), to obtain at least one second extrinsic parameter correction value (T) Δ ); based on at least one second extrinsic parameter correction value (T) Δ The system compares the compensation value (which can be the diagonal value of the covariance matrix, i.e., the variance) with a preset threshold to generate a detection result indicating whether the vehicle sensor is abnormal or normal. Here, covariance represents the stability of the sensor's extrinsic parameter calibration; a larger covariance indicates greater instability, higher noise, and lower weight.

[0058] In some embodiments, based on the at least one second extrinsic parameter correction value (T) Δ The method of generating the detection result of the vehicle sensor being abnormal or normal by comparing the compensation value of ) with the preset threshold can be that at least one second extrinsic parameter correction value (T) Δ If the compensation value is greater than a preset threshold and the duration is greater than a preset time length or the number of occurrences is greater than a preset number, a detection result of the vehicle sensor abnormality is generated.

[0059] For example, when an autonomous driving system following a vehicle is activated, because the number of repetitions is relatively small, the amount of data obtained is also small, and the second extrinsic parameter correction value (T) is relatively low. Δ The initial compensation value (e.g., covariance) may be large. As the system operates, the fusion process causes the second extrinsic parameter correction value (T) to be larger. Δ The convergence gradually decreases, and the second extrinsic parameter correction value (T) is obtained. Δ The compensation value (e.g., covariance) will also decrease. When the compensation value (e.g., covariance) decreases to below a preset threshold, it can be considered that the calibration compensation has stabilized, indicating that the vehicle sensor is normal. The calibration correction value at this time can be uploaded to the cloud for storage for subsequent determination of the calibrated vehicle sensor extrinsic parameters (T). s2 )use.

[0060] The external parameter correction value (T) involved in the embodiments of this application is... Δ(Including the first extrinsic parameter correction value, the second extrinsic parameter correction value, and the weighted average extrinsic parameter correction value) can be represented as an identity matrix, whose initial value can be 1, that is, in the initial extrinsic parameter (T) of the vehicle sensor. s1 Under accurate conditions, the external parameter correction value (T) Δ ) can be 1; the external parameter correction value (T) Δ To a certain extent, this represents the magnitude of the vehicle sensor's position error or rotation angle. Generally, the rotation angle of a vehicle sensor is typically within 10 degrees (e.g., 0-5 degrees), while the extrinsic parameter correction value (T) of the vehicle sensor... Δ The external parameter correction value (T) of the above-mentioned vehicle sensor should be kept within 0.1 degrees to achieve good detection results. Δ The preset threshold for the compensation value (e.g., covariance) can be set to 0.1 degrees, and the corresponding preset threshold for the compensation value (e.g., covariance) can be set to 0.01.

[0061] For example, after generating the detection result of the vehicle sensor malfunction, a calibration alarm for the malfunctioning vehicle sensor can be activated and / or the malfunctioning vehicle sensor can be disabled in the positioning system.

[0062] For example, if the second extrinsic parameter correction value (T) of a certain vehicle sensor Δ The compensation value (e.g., covariance) is always greater than the preset threshold, indicating that the second extrinsic parameter correction value (T) is not correct. Δ If the calculation does not converge and the observation state of the vehicle sensor is inconsistent with the system state, it indicates that the vehicle sensor may be abnormal, possibly due to hardware vibration or data anomaly. In this case, a calibration alarm can be issued in time, and the vehicle sensor can be disabled in the positioning system.

[0063] In some embodiments, the correction based on the at least one second extrinsic parameter (T) Δ The comparison result between the compensation value and the preset threshold of the vehicle sensor generates a detection result indicating whether the vehicle sensor is abnormal or normal, including: at least one second extrinsic parameter correction value (T) Δ If the compensation value of the vehicle sensor is less than or equal to a preset threshold, a normal detection result for the vehicle sensor is generated; correspondingly, based on the initial extrinsic parameters (T) of the vehicle sensor... s1 ) and the first extrinsic parameter correction value (T) of the vehicle sensor Δ ), determine the calibrated vehicle sensor extrinsic parameters (T) s2 The method can be to use the initial extrinsic parameters (T) of the vehicle sensors. s1 ) and the second extrinsic parameter correction value (T) of the vehicle sensor Δ The product of ) is used as the calibrated vehicle sensor extrinsic parameter (T). s2 ).

[0064] Among them, the vehicle sensor extrinsic parameters (T) after the previous calibration s2 It can also be converted into the initial extrinsic parameters (T) of the vehicle sensors. s1 From this, we can obtain the corrected extrinsic parameter value (T) after multiple optimizations. Δ After it converges to a stable value, a relatively accurate vehicle sensor extrinsic parameter (T) is obtained. s2 ).

[0065] In some embodiments, the calibration scheme for the vehicle sensor extrinsic parameters may further include: adjusting the first extrinsic parameter correction value (T) to... Δ ) and the at least one second extrinsic parameter correction value (T) Δ Perform a weighted average to obtain the weighted average corrected external parameter value (T). Δupdate ); the initial extrinsic parameters (T) of the vehicle sensor s1 ) and the weighted average corrected external parameter value (T) Δupdate The product of ) is used as the calibrated vehicle sensor extrinsic parameter (T). s2 ).

[0066] In other words, the multiple second extrinsic parameter values ​​(T) obtained from the aforementioned repeated execution can be modified. Δ A weighted average (a weighted average of the results from multiple runs) is performed to obtain a more accurate, stable, and convergent external parameter correction value. For example, the second external parameter correction value (T) obtained within a time period of 5 days (120 hours) can be used. Δ There are n in total, which can be specifically represented as: T Δi =(x Δi ,y Δi ,z Δi ,roll Δi pitch Δi ,yaw Δi )i=[1,n];In the process of fusion calculation using Kalman filtering, the second extrinsic parameter correction value (T) is obtained each time. Δ At the same time, a compensation value (e.g., covariance) corresponding to the correction value of the second extrinsic parameter will be obtained, that is, for each correction value of the second extrinsic parameter (T) Δ Each of these corresponds to a compensation value (covariance), and there are a total of n values, which can be represented as: (var_x) Δi ,var_y Δi ,var_z Δi ,var_roll Δi ,var_pitch Δi ,var_yaw Δi i = [1, n]. Then, the second extrinsic parameter correction value (T) can be used. Δ ) and its corresponding compensation value (e.g., covariance), calculate the second extrinsic parameter correction value (T)Δ The weighted average of the position information along the x, y, and z axes and the angle information along the roll, pitch, and yaw axes, i.e., the average of the position information along the x, y, and z axes and the angle information along the roll, pitch, and yaw axes, xmean, ymean, zmean, rollmean, pitchmean, and yawmean, can be specifically expressed as:

[0067]

[0068] Furthermore, the average values ​​of x, y, z, roll, pitch, and yaw calculated above are used as the weighted average correction values ​​for the extrinsic parameters (T). Δupdate Specifically, it is represented as: T Δupdate =(x mean ,y mean ,z mean ,roll mean ,picth mean ,yaw mean Finally, the initial extrinsic parameters (T) of the vehicle sensors are... s1 ) and the weighted average of the corrected external parameters (T) Δupdate The product of ) is used as the calibrated vehicle sensor extrinsic parameter (T). s2 Specifically, it can be represented as: T s2 =T s1 *T Δupdate .

[0069] In one possible implementation, the above-mentioned vehicle sensor extrinsic parameter calibration scheme may further include: setting the initial extrinsic parameters (T) of the vehicle sensor... s1 The first extrinsic parameter correction value (T) of the vehicle sensor Δ ) and the second extrinsic parameter correction value (T) of the vehicle sensor Δ Uploaded to the cloud; received vehicle sensor extrinsic parameters (T) calibrated by the cloud. s2 ).

[0070] In this embodiment, the extrinsic parameter correction values ​​and extrinsic parameters of the vehicle sensors obtained each time can be stored in the cloud. When the autonomous vehicle (self-driving vehicle) starts, the latest extrinsic parameters of the vehicle sensors can be retrieved from the cloud, and the new extrinsic parameter correction values ​​and extrinsic parameters of the vehicle sensors obtained during the operation of the self-driving vehicle can be uploaded to the cloud. The extrinsic parameter correction values ​​and extrinsic parameters of the vehicle sensors are automatically updated in the cloud.

[0071] Correspondingly, cloud computing power can also be used in this process to calculate at least one second extrinsic parameter correction value (T). ΔThe compensation value, and the comparison result between the compensation value and the preset threshold, etc., are then processed in the cloud based on the initial extrinsic parameters (T) of the vehicle sensors. s1 ) and the first extrinsic parameter correction value (T) of the vehicle sensor Δ ), determine the calibrated vehicle sensor extrinsic parameters (T) s2 For example, the initial extrinsic parameters (T) of the vehicle sensors. s1 ) and the second extrinsic parameter correction value (T) of the vehicle sensor Δ The product of ) is used as the calibrated vehicle sensor extrinsic parameter (T). s2 For example, the extrinsic parameter correction values ​​of the vehicle sensor extrinsic parameters from multiple runs (repeatedly executing the aforementioned steps) can be weighted and averaged in the cloud, and then the vehicle sensor extrinsic parameters and their correction values ​​in the cloud can be updated. Upon the next vehicle startup, the updated and corrected vehicle sensor extrinsic parameters (T) can be retrieved from the cloud. s2 ).

[0072] It should be understood that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowchart above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0073] Corresponding to the examples and method embodiments provided in this application, this application also provides a calibration device for vehicle sensor extrinsic parameters. For example... Figure 4The diagram shown is a structural block diagram of a vehicle sensor extrinsic parameter calibration device 400 according to an embodiment of this application. The device 400 may include: a trailer mount pose determination module 401, used to process vehicle data sensed by the vehicle based on the initial extrinsic parameters of the vehicle sensors, and determine the detection pose of the trailer mount of the preceding vehicle and the trailer mount in the vehicle's coordinate system; and a front vehicle pose determination module 402, used to determine the global pose of the front vehicle's front end based on the detection pose and the trailer mount angle between the front vehicle's front end and the trailer mount, and based on the global pose of the front vehicle's front end... The pose determination module 403 is used to obtain the second global pose of the vehicle ahead, and to perform fusion calculation on the first global pose and the second global pose using Kalman filtering to determine the third global pose of the vehicle ahead; based on the third global pose of the vehicle ahead and the initial extrinsic parameters, the first extrinsic parameter correction value of the vehicle sensor is determined; the sensor extrinsic parameter calibration module 404 is used to determine the calibrated extrinsic parameters of the vehicle sensor based on the initial extrinsic parameters of the vehicle sensor and the first extrinsic parameter correction value of the vehicle sensor.

[0074] In one possible implementation, the above-mentioned device may further include: a detection processing module, configured to detect and process vehicle data perceived by the vehicle itself based on calibrated vehicle sensor extrinsic parameters, and filter vehicle data belonging to a preceding vehicle from the vehicle data perceived by the vehicle itself, wherein the vehicle data perceived by the vehicle itself includes the poses of at least one vehicle surrounding the vehicle perceived by the vehicle itself; a repetition execution module, configured to repeatedly execute the process of determining the trailer of the preceding vehicle and the detection pose of the trailer of the preceding vehicle in the vehicle coordinate system to determining the third global pose of the preceding vehicle at least once; and obtain at least one second extrinsic parameter correction value based on the third global pose of the preceding vehicle and the calibrated vehicle sensor extrinsic parameters; and a detection result generation module, configured to generate a detection result indicating whether the vehicle sensor is abnormal or normal based on the comparison result of the compensation value of the at least one second extrinsic parameter correction value and a preset threshold.

[0075] In some embodiments, the above-mentioned detection result generation module may include: an abnormal detection result generation submodule, used to generate a detection result of the vehicle sensor abnormality when the compensation value of the at least one second external parameter correction value is greater than a preset threshold and the duration is greater than a preset time length or the number of durations is greater than a preset number.

[0076] For example, the above-mentioned device may further include: an alarm activation submodule, used to activate the calibration alarm of the abnormal vehicle sensor and / or deactivate the abnormal vehicle sensor in the positioning system after generating the detection result of the abnormality of the vehicle sensor.

[0077] In some embodiments, the above-mentioned detection result generation module may include: a normal detection result generation submodule, used to generate a normal detection result of the vehicle sensor when the compensation value of the at least one second extrinsic parameter correction value is less than or equal to a preset threshold; the above-mentioned sensor extrinsic parameter calibration module 404 may include: a vehicle sensor extrinsic parameter determination submodule, used to take the product of the initial extrinsic parameter of the vehicle sensor and the second extrinsic parameter correction value of the vehicle sensor as the calibrated vehicle sensor extrinsic parameter.

[0078] In some embodiments, the above apparatus may further include: a weighted averaging module, configured to perform a weighted average of the first extrinsic parameter correction value and the at least one second extrinsic parameter correction value to obtain a weighted averaged extrinsic parameter correction value; and a vehicle sensor extrinsic parameter weighted averaging module, configured to use the product of the initial extrinsic parameter of the vehicle sensor and the weighted averaged extrinsic parameter correction value as the calibrated vehicle sensor extrinsic parameter.

[0079] In one possible implementation, the sensor extrinsic calibration module 404 may include: a vehicle sensor extrinsic calibration module, used to take the product of the initial extrinsic parameter of the vehicle sensor and the first extrinsic parameter correction value of the vehicle sensor as the calibrated vehicle sensor extrinsic parameter.

[0080] In some embodiments, the above-described apparatus may further include: an uploading module for uploading the initial extrinsic parameters of the vehicle sensor, the first extrinsic parameter correction value of the vehicle sensor, and the second extrinsic parameter correction value of the vehicle sensor to the cloud; and a receiving module for receiving the vehicle sensor extrinsic parameters calibrated in the cloud.

[0081] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0082] Figure 5 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 5 As shown, the electronic device includes a memory 501 and a processor 502. The memory 501 stores a computer program that can run on the processor 502. When the processor 502 executes the computer program, it implements the method described in the above embodiments. The number of memories 501 and processors 502 can be one or more.

[0083] The electronic device also includes:

[0084] Communication interface 503 is used to communicate with external devices and perform data exchange and transmission.

[0085] If the memory 501, processor 502, and communication interface 503 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0086] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0087] The vehicle sensor extrinsic parameter calibration method provided in this disclosure can be used on a computer device, which can be a terminal, and its internal structure diagram can be as follows. Figure 6 As shown. The computer device includes a processor, memory, communication interface, display screen (display unit), and input device connected via a system bus and I / O interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for calibrating extrinsic parameters of vehicle sensors. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse. Those skilled in the art will understand that… Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0088] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0089] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the methods provided in any embodiment of this application.

[0090] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0091] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in this application.

[0092] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0093] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0094] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0095] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0096] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0097] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0098] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0099] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0100] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0101] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for calibrating extrinsic parameters of vehicle sensors, applied to following vehicles in a vehicle platoon, the method comprising: Based on the initial extrinsic parameters of the vehicle sensors, the vehicle data perceived by the vehicle is detected and processed to determine the trailer of the preceding vehicle and its detection pose in the vehicle's coordinate system. Based on the detected pose and the trailer angle between the front of the vehicle and the trailer of the vehicle, the global pose of the front of the vehicle is determined, and the first global pose of the vehicle is determined based on the global pose of the front of the vehicle. The second global pose of the preceding vehicle is obtained, and the first global pose and the second global pose are fused and calculated using Kalman filtering to determine the third global pose of the preceding vehicle. Based on the third global pose of the preceding vehicle and the initial extrinsic parameters, the first extrinsic parameter correction value of the vehicle sensor is determined. The calibrated vehicle sensor extrinsic parameters are determined based on the initial extrinsic parameters of the vehicle sensor and the first extrinsic parameter correction value of the vehicle sensor.

2. The method according to claim 1, wherein, The method further includes: The vehicle data perceived by the vehicle is detected and processed according to the calibrated vehicle sensor extrinsic parameters, and the vehicle data belonging to the preceding vehicle is filtered out from the vehicle data perceived by the vehicle. The vehicle data perceived by the vehicle includes the pose of at least one vehicle around the vehicle perceived by the vehicle. Repeat the process of determining the trailer of the preceding vehicle and the detection pose of the trailer in the vehicle coordinate system to determining the third global pose of the preceding vehicle at least once; obtain at least one second extrinsic parameter correction value based on the third global pose of the preceding vehicle and the calibrated vehicle sensor extrinsic parameters. Based on the comparison result between the compensation value of the at least one second extrinsic parameter correction value and the preset threshold, a detection result of whether the vehicle sensor is abnormal or normal is generated.

3. The method according to claim 2, wherein, The step of generating a detection result for whether the vehicle sensor is abnormal or normal based on the comparison result of the compensation value of the at least one second extrinsic parameter correction value and a preset threshold includes: If the compensation value of at least one second external parameter correction value is greater than a preset threshold, and the duration is greater than a preset time length or the number of durations is greater than a preset number of durations, a detection result of the vehicle sensor abnormality is generated.

4. The method according to claim 3, wherein, The method further includes: After generating the detection result of the vehicle sensor malfunction, activate the calibration alarm of the malfunctioning vehicle sensor and / or disable the malfunctioning vehicle sensor in the positioning system.

5. The method according to claim 2, wherein, The step of generating a detection result for whether the vehicle sensor is abnormal or normal based on the comparison result of the compensation value of the at least one second extrinsic parameter correction value and a preset threshold includes: If the compensation value of at least one second external parameter correction value is less than or equal to a preset threshold, a normal detection result of the vehicle sensor is generated. The step of determining the calibrated vehicle sensor extrinsic parameters based on the initial extrinsic parameters of the vehicle sensor and the first extrinsic parameter correction value of the vehicle sensor includes: The product of the initial extrinsic parameter of the vehicle sensor and the second extrinsic parameter correction value of the vehicle sensor is used as the calibrated extrinsic parameter of the vehicle sensor.

6. The method according to claim 2, wherein, The method further includes: The first external parameter correction value and the at least one second external parameter correction value are weighted and averaged to obtain the weighted average external parameter correction value. The product of the initial extrinsic parameters of the vehicle sensor and the corrected extrinsic parameters after weighted averaging is used as the calibrated extrinsic parameters of the vehicle sensor.

7. The method according to claim 1, wherein, The step of determining the calibrated vehicle sensor extrinsic parameters based on the initial extrinsic parameters of the vehicle sensor and the first extrinsic parameter correction value of the vehicle sensor includes: The product of the initial extrinsic parameter of the vehicle sensor and the first extrinsic parameter correction value of the vehicle sensor is used as the calibrated extrinsic parameter of the vehicle sensor.

8. The method according to claim 2, wherein, The method further includes: The initial extrinsic parameters of the vehicle sensor, the first extrinsic parameter correction value of the vehicle sensor, and the second extrinsic parameter correction value of the vehicle sensor are uploaded to the cloud. Receive vehicle sensor extrinsic parameters after cloud calibration.

9. A calibration device for vehicle sensor extrinsic parameters, applied to following vehicles in a vehicle platoon, the device comprising: The trailer pose determination module is used to detect and process the vehicle data perceived by the vehicle based on the initial extrinsic parameters of the vehicle sensors, and determine the trailer of the preceding vehicle and the detection pose of the trailer of the preceding vehicle in the vehicle coordinate system. The front vehicle pose determination module is used to determine the global pose of the front vehicle head based on the detected pose and the trailer angle between the front vehicle head and the trailer of the front vehicle, and to determine the first global pose of the front vehicle based on the global pose of the front vehicle head. The extrinsic parameter correction value determination module is used to obtain the second global pose of the preceding vehicle, perform fusion calculation on the first global pose and the second global pose using Kalman filtering to determine the third global pose of the preceding vehicle; and determine the first extrinsic parameter correction value of the vehicle sensor based on the third global pose of the preceding vehicle and the initial extrinsic parameters. The sensor extrinsic parameter calibration module is used to determine the calibrated vehicle sensor extrinsic parameters based on the initial extrinsic parameters of the vehicle sensor and the first extrinsic parameter correction value of the vehicle sensor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1-8.

11. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1-8.

12. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the method of any one of claims 1-8.

Images