Calibration method and device and vehicle

Abstract

The invention provides a calibration method and device and a vehicle, the method can be applied to the field of intelligent vehicles, and the method comprises the steps: obtaining a first image collected by a vehicle-mounted sensor; determining first type information of a first part in the first image; according to the first type information and a first corresponding relation, a first external parameter is determined, the first corresponding relation is a corresponding relation between the component type and a calibratable external parameter, and the calibratable external parameter comprises the first external parameter; and carrying out online calibration on the first component according to the first external parameter. Through the method, the online calibration precision of the sensor in the vehicle can be improved, the online calibration cost of the sensor can be reduced, and the method can be applied to different scenes and has good generalization.

Description

Technical Field

[0001] The present application relates to the field of smart vehicles, and more specifically, to a calibration method, device, and vehicle. Background Art

[0002] Autonomous driving technology has become a new development trend in the automotive industry and is gaining increasing popularity. Accurate sensor extrinsic parameters are essential for achieving high-precision fusion perception. Over vehicle use, collisions, and aging, the actual sensor extrinsic parameters inevitably deviate from the initial parameters. Failure to promptly update these parameters can lead to a decrease in perception accuracy or even failure, compromising driving safety and potentially causing serious accidents.

[0003] Currently, online calibration solutions can timely update sensor external parameters to ensure vehicle driving safety. However, current online calibration solutions suffer from high calibration costs, poor calibration accuracy, and limited application scenarios, which urgently need to be addressed. Summary of the Invention

[0004] The present application provides a calibration method, device, and vehicle, which can improve the accuracy of online calibration of sensors in a vehicle and reduce the cost of online calibration of sensors.

[0005] In a first aspect, a calibration method is provided, characterized in that the method includes: acquiring a first image captured by a vehicle-mounted sensor; determining first type information of a first component in the first image; determining a first external parameter based on the first type information and a first correspondence, wherein the first correspondence is a correspondence between a component type and a calibratable external parameter, and the calibratable external parameter includes the first external parameter; and performing online calibration based on the first external parameter.

[0006] In a possible implementation, the vehicle-mounted sensor may be a camera device or a radar, wherein the camera device may be: a front-view camera, a surround-view camera, a rear-view camera, a side-view camera, or a built-in camera.

[0007] In a possible implementation, the first component may include one or more of: a door seam, a vehicle handle, a wheel fender, a wheel fender outline, a marker, a lower edge of the vehicle, and a license plate.

[0008] In one possible implementation, the first type information may include feature point type information or contour line type information. For example, when the first component is a license plate, the first type information may indicate that the first component is a license plate and that the license plate includes one or more feature points. For another example, when the first component is a lower edge of a vehicle, the first type information may indicate that the first component is a lower edge of a vehicle and that the lower edge includes a contour line.

[0009] In a possible implementation, before executing the calibration method, the vehicle may pre-configure the first correspondence in an offline manner.

[0010] In an embodiment of the present application, the first type information of the first component can be determined based on the image collected by the vehicle-mounted sensor, and the first external parameter can be determined based on the first type information and the first corresponding relationship, and then online calibration can be performed based on the first external parameter. In this way, the accuracy of online calibration of sensors in the vehicle can be improved and the cost of online calibration of sensors can be reduced. Moreover, the method can be applicable to different scenarios and has good generalization.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: determining first position information of the first component in the first image; performing online calibration based on the first external parameter includes: determining a first compensation value corresponding to the first external parameter based on the first position information and second position information, the second position information indicating a preset reference position of the first component in the first image; and performing online calibration based on the first compensation value.

[0012] In a possible implementation, before executing the calibration method, the vehicle may pre-configure the second position information in an offline manner.

[0013] In the embodiment of the present application, a first compensation value for a first external parameter can be determined based on the first position information of the first component in the first image and the second position information preset for the first component in the first image, and online calibration can be performed based on the first compensation value. In this way, even if the on-board sensor experiences a large angle deviation, the on-board sensor can be corrected based on the first compensation value, thereby ensuring that the on-board sensor always maintains a good working condition.

[0014] In combination with the first aspect, in certain implementations of the first aspect, the first type information and the first position information are obtained by inputting the first image into a first prediction model, and the first prediction model is obtained based on a training sample, and the training sample includes: a first sample image, first sample type information and first sample position information.

[0015] In one possible implementation, the first prediction model may be a neural network model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN). Alternatively, the first prediction model may be a machine learning algorithm, such as a support vector machine (SVM).

[0016] In a possible implementation, before executing the calibration method, a first prediction model may be pre-trained based on training samples, and the first prediction model may be configured in the vehicle.

[0017] In the embodiment of the present application, the first type information and the first position information are obtained through the first prediction model, so that the detection of the type and position of the first component can be automated, thereby improving processing efficiency and reliability of detection results.

[0018] In combination with the first aspect, in certain implementations of the first aspect, the first position information includes: position information of N feature points, where N is an integer greater than 1, and determining the first compensation value corresponding to the first external parameter based on the first position information and the second position information includes: removing noise from the position information of the N feature points to obtain position information of Q feature points, where Q is less than or equal to N; and determining the first compensation value based on the position information of the Q feature points and the second position information.

[0019] In a possible implementation, performing noise removal processing on the position information of the N feature points may include: performing noise removal by clustering the feature points and / or calculating a mean.

[0020] In an embodiment of the present application, the position information of N feature points can be first subjected to noise removal processing to obtain relatively accurate first position information, and then a first compensation value can be determined based on the first position information and the second position information. In this way, a more accurate first compensation value can be obtained, thereby further improving the accuracy of online calibration of sensors in the vehicle.

[0021] In combination with the first aspect, in certain implementations of the first aspect, the noise removal processing is performed on the position information of the N feature points to obtain the position information of the Q feature points, including: fitting a first curve based on the position information of the N feature points; when the distance between the M feature points and the first curve is greater than or equal to a preset threshold, removing the position information of the M feature points from the first position information to obtain the position information of the Q feature points, the N feature points including the M feature points, M is greater than or equal to 1, and M is less than N.

[0022] In an embodiment of the present application, when the first type of information is feature information of the contour line type, the N feature points on the contour line can be fitted into a first curve, and the M feature points that are far away from the first curve can be removed. In this way, for the position information of the N feature points, the purpose of removing noise can be achieved well, thereby obtaining a more accurate first compensation value.

[0023] In combination with the first aspect, in certain implementations of the first aspect, the online calibration based on the first compensation value includes: performing multi-frame filtering on the first compensation value to obtain a second compensation value; and performing online calibration based on the second compensation value.

[0024] In one possible implementation, performing multi-frame filtering on the first compensation value may include: accumulating n first compensation values to obtain a set of statistical data, sorting the n first compensation values in the statistical data to remove outliers, and obtaining a second compensation value when the standard deviation of the statistical data is less than a preset threshold.

[0025] In an embodiment of the present application, the first compensation value can be subjected to multi-frame filtering to obtain a second compensation value, and then online calibration can be performed based on the second compensation value. In this way, a stable online calibration result (i.e., the second compensation value) can be output, and the accuracy of the sensor online calibration can be further improved.

[0026] In combination with the first aspect, in certain implementations of the first aspect, the first type of information includes: characteristic information of the contour line type, and determining the first external parameter based on the first type of information and the first correspondence includes: selecting at least one Euler angle from multiple Euler angles as the first external parameter based on the characteristic information of the contour line type and the first correspondence.

[0027] For example, when the first type of information is the lower edge of the vehicle body, the yaw and roll angles may be selected as the first external parameters.

[0028] In an embodiment of the present application, when the first type of information is feature information of a contour line type, at least one Euler angle can be selected from multiple Euler angles as the first external parameter. In this way, the application of contour features is supported, and the calibration method can be applied to more application scenarios.

[0029] In a second aspect, a calibration device is provided, which includes an acquisition unit and a processing unit; the acquisition unit is used to acquire a first image captured by a vehicle-mounted sensor; the processing unit is used to: determine first type information of a first component in the first image; determine a first external parameter based on the first type information and a first correspondence, the first correspondence being a correspondence between a component type and a calibratable external parameter, the calibratable external parameter including the first external parameter; and perform online calibration based on the first external parameter.

[0030] In combination with the second aspect, in certain implementations of the second aspect, the processing unit is further used to: determine the first position information of the first component in the first image; the processing unit is specifically used to: determine the first compensation value corresponding to the first external parameter based on the first position information and the second position information, the second position information being used to indicate a preset reference position of the first component in the first image; and perform online calibration based on the first compensation value.

[0031] In combination with the second aspect, in certain implementations of the second aspect, the first type information and the first position information are obtained by inputting the first image into a first prediction model, and the first prediction model is obtained based on a training sample, and the training sample includes: a first sample image, first sample type information and first sample position information.

[0032] In combination with the second aspect, in certain implementations of the second aspect, the first position information includes: position information of N feature points, where N is an integer greater than 1; the processing unit is specifically used to: remove noise from the position information of the N feature points to obtain position information of Q feature points, where Q is less than or equal to N; and determine the first compensation value based on the position information of the Q feature points and the second position information.

[0033] In combination with the second aspect, in certain implementations of the second aspect, the processing unit is specifically used to: fit a first curve based on the position information of the N feature points; when the distance between the M feature points and the first curve is greater than or equal to a preset threshold, remove the position information of the M feature points from the first position information to obtain the position information of the Q feature points, the N feature points include the M feature points, M is greater than or equal to 1, and M is less than N.

[0034] In combination with the second aspect, in certain implementations of the second aspect, the processing unit is specifically used to: perform multi-frame filtering on the first compensation value to obtain a second compensation value; and perform online calibration based on the second compensation value.

[0035] In combination with the second aspect, in certain implementations of the second aspect, the first type of information includes: characteristic information of the contour type; the processing unit is specifically used to select at least one Euler angle from multiple Euler angles as the first external parameter based on the characteristic information of the contour type and the first correspondence.

[0036] In a third aspect, a calibration device is provided, comprising: at least one processor and a memory, wherein the at least one processor is coupled to the memory and is configured to read and execute instructions in the memory, so that the device implements the method in any one of the implementations of the first aspect above.

[0037] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a program code, and when the computer program code is run on a computer, the computer executes the method in any one of the implementation modes of the above-mentioned first aspect.

[0038] In a fifth aspect, a chip is provided, which includes a circuit for executing the method in any one of the implementations of the first aspect.

[0039] In a sixth aspect, a computer program product is provided, which includes a computer program. When the computer program is run, the computer executes the method in any one of the implementation modes of the first aspect.

[0040] In a seventh aspect, a vehicle is provided, comprising: a calibration device according to any one of the implementations of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a functional schematic diagram of a vehicle provided in an embodiment of the present application;

[0042] Figure 2 This is the system architecture to which the calibration method provided in the embodiments of the present application is applicable;

[0043] Figure 3 is a schematic flow chart of a calibration method provided in an embodiment of the present application;

[0044] Figure 4 is a schematic flow chart of another calibration method provided in an embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of vehicle features obtained by a side-view fisheye camera provided in an embodiment of the present application;

[0046] Figure 6 This is a schematic diagram of reasoning using a vehicle feature network provided in an embodiment of the present application;

[0047] Figure 7 This is a schematic diagram of the features of the vehicle's license plate point acquired by a forward-looking fisheye camera provided in an embodiment of the present application;

[0048] Figure 8 is a schematic flow chart of a multi-frame filtering method provided in an embodiment of the present application;

[0049] Figure 9 is a schematic flow chart of another calibration method provided in an embodiment of the present application;

[0050] Figure 10 This is a schematic diagram of a vehicle edge changing with the sensor angle provided by an embodiment of the present application;

[0051] Figure 11 This is a schematic diagram of a vehicle contour line fitting and denoising provided by an embodiment of the present application;

[0052] Figure 12 is a schematic diagram of a calibration device provided in an embodiment of the present application;

[0053] Figure 13 It is a schematic diagram of another calibration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solution in this application will be described below with reference to the accompanying drawings.

[0055] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In this application, "at least one" refers to one or more, and "more than one" refers to two or more. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0056] In the embodiments of this application, prefixes such as "first" and "second" are used only to distinguish different description objects and have no limiting effect on the position, order, priority, quantity, or content of the described objects. The use of prefixes such as ordinal numbers in the embodiments of this application to distinguish description objects does not constitute a limitation on the described objects. For a statement of the described objects, please refer to the description in the context of the claims or embodiments, and the use of such prefixes should not constitute an unnecessary limitation.

[0057] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0058] Figure 1 It is a functional schematic diagram of the vehicle 100 provided in an embodiment of the present application.

[0059] The vehicle 100 may include various subsystems, such as a perception system 120 and a computing platform 130. Alternatively, the vehicle 100 may include more or fewer subsystems, and each subsystem may include one or more components. Furthermore, each subsystem and component of the vehicle 100 may be interconnected via wired or wireless means.

[0060] Perception system 120 may include several sensors for sensing information about the environment surrounding vehicle 100. For example, perception system 120 may include a positioning system, which may be a global positioning system (GPS), a Beidou system, or other positioning systems. Perception system 120 may include one or more of an inertial measurement unit (IMU), a laser radar, a millimeter-wave radar, an ultrasonic radar, and a camera.

[0061] Some or all functions of the vehicle 100 may be controlled by a computing platform 130. The computing platform 130 may include processors 131 to 13n (n is a positive integer). A processor is a circuit capable of processing signals. In one implementation, the processor may be a circuit capable of reading and executing instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor may implement certain functions through the logical relationships of a hardware circuit. The logical relationships of the hardware circuit may be fixed or reconfigurable. For example, the processor may be a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration file to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, the processor may also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In addition, the computing platform 130 may also include a memory for storing instructions, and some or all of the processors 131 to 13n may call the instructions in the memory to implement corresponding functions.

[0062] The computing platform 130 may control functions of the vehicle 100 based on input received from various subsystems, such as the perception system 120. In some embodiments, the computing platform 130 may be used to provide control over many aspects of the vehicle 100 and its subsystems.

[0063] Optionally, the above components are just an example. In actual applications, the components in the above modules may be added or deleted according to actual needs.

[0064] The vehicle 100 in this application may include: road vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment, etc. For example, the vehicle 100 may be a vehicle (such as a commercial vehicle, a passenger car, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), agricultural equipment (such as a lawn mower, a harvester, etc.), amusement equipment, a toy vehicle, etc. The embodiments of this application do not specifically limit the type of vehicle.

[0065] The following takes vehicle 100 as an example of an autonomous driving vehicle to illustrate the technical problems that need to be solved in this application and the technical solutions adopted.

[0066] Autonomous driving technology has become a new development direction for vehicles and is gaining increasing popularity. Accurate sensor extrinsics are essential for achieving high-precision fusion perception. Over vehicle use, collisions, and aging, the actual sensor extrinsics inevitably deviate from the initial extrinsics. Failure to promptly update these extrinsics can lead to a decrease in vehicle perception accuracy or even failure, compromising driving safety and potentially causing serious accidents. While traditional calibration methods using targets offer high accuracy, they are expensive for manufacturers to build calibration workshops. For users, sending their vehicles to 4S dealerships for recalibration also incurs significant time and financial costs. Therefore, online vehicle calibration solutions are essential.

[0067] At present, online calibration solutions have problems such as high calibration cost, poor calibration accuracy and limited application scenarios, which need to be solved urgently.

[0068] For example, one online calibration solution can capture lane line images with an on-board camera, derive the world coordinate linear equations of the lane lines based on these images, and then optimize the camera's extrinsic parameters based on these world coordinate linear equations to achieve extrinsic calibration of the camera. This online calibration solution has very limited application scenarios. Specifically, it cannot be used when the lane lines are not parallel or there are insufficient lane lines in the field of view.

[0069] For example, an online calibration solution can detect the posture of an on-board camera. When the camera's posture changes, it extracts matching calibration parameters from pre-set calibration data based on the change in the camera's posture, and uses these calibration parameters to update the camera's extrinsic parameters. This calibration method requires pre-establishing a mapping relationship between images and calibration parameters, resulting in a large amount of collected data and high application costs. Furthermore, the multiple extrinsic parameters used for calibration are often coupled to each other, and the aforementioned mapping relationship is discrete. It is difficult to achieve a one-to-one correspondence between the image and calibration parameters in the mapping relationship, which limits the accuracy of the final calibration parameters.

[0070] The embodiments of the present application provide a calibration method, device, and vehicle, which can improve the accuracy of online calibration of sensors in a vehicle and reduce the cost of online calibration of sensors. In addition, the method can be applied to different scenarios and has good generalization.

[0071] Figure 2 This is the system architecture applicable to the calibration method provided in the embodiment of the present application. This system architecture can be applied to Figure 1 In the vehicle 100.

[0072] like Figure 2 As shown, the system architecture may include: onboard sensors and an onboard computing platform. The onboard sensors may include radar, cameras, and the like. After the vehicle is started, the onboard sensors may acquire sensor data in real time and input the sensor data into the onboard computing platform. The onboard computing platform may be computing platform 130 in vehicle 100. After acquiring the sensor data, the onboard computing platform may utilize a pre-trained ego vehicle feature detection network to detect the sensor data and extract the ego vehicle feature category and location. The onboard computing platform may then perform feature pre-processing and filtering on the ego vehicle features and locations to select calibrable extrinsic parameters. The onboard computing platform may then optimize the calibrable extrinsic parameters to ensure that the pre-stored ego vehicle feature locations coincide with the real-time feature locations, thereby outputting the ego vehicle features. Finally, the onboard computing platform may accumulate multiple frames of ego vehicle features to obtain stable sensor extrinsic parameter values or extrinsic parameter compensation values.

[0073] Figure 3 is a schematic flow chart of a calibration method provided in an embodiment of the present application. The execution subject of method 300 may be a vehicle. When the execution subject of method 300 is vehicle 100, it may be executed by computing platform 130 in vehicle 100, or may also be executed by a system-on-chip (SoC) in computing platform 130, or may also be executed by a processor in computing platform 130. The following describes method 300 using a vehicle as the execution subject. Method 300 may include steps S301 to S304.

[0074] S301: Acquire a first image captured by a vehicle-mounted sensor.

[0075] Optionally, the vehicle-mounted sensor may be a camera or a radar, wherein the camera may be a front-view camera, a surround-view camera, a rear-view camera, a side-view camera, or a built-in camera. Further optionally, the camera may be a side-view fisheye camera.

[0076] S302: Determine first type information of a first component in a first image.

[0077] Optionally, the first component may include one or more of: a door seam, a vehicle handle, a wheel fender, a wheel fender outline, a marker, a lower edge of the vehicle, and a license plate.

[0078] Optionally, the first type information may include feature point type information or contour line type information. For example, when the first component is a license plate, the first type information may indicate that the first component is a license plate and includes one or more feature points. For another example, when the first component is a lower edge of a vehicle, the first type information may indicate that the first component is a lower edge of a vehicle and includes a contour line.

[0079] S303: Determine a first external parameter according to the first type information and the first corresponding relationship.

[0080] The first corresponding relationship is a corresponding relationship between a component type and a calibratable external parameter, and the calibratable external parameter includes a first external parameter.

[0081] Optionally, before step S303 , the vehicle may configure the first correspondence in an offline manner. For example, the vehicle obtains a configuration file before leaving the factory or during maintenance at a 4S shop, and the configuration file includes the first correspondence.

[0082] S304: Perform online calibration based on the first external parameter.

[0083] In an embodiment of the present application, the first type information of the first component can be determined based on the image collected by the vehicle-mounted sensor, and the first external parameter can be determined based on the first type information and the first corresponding relationship, and then online calibration can be performed based on the first external parameter. In this way, the accuracy of online calibration of sensors in the vehicle can be improved and the cost of online calibration of sensors can be reduced. Moreover, the method can be applicable to different scenarios and has good generalization.

[0084] In one possible implementation, before step S304, method 300 further includes determining first position information of the first component in the first image. In step S304, the vehicle can determine a first compensation value corresponding to the first extrinsic parameter based on the first position information and the second position information, and perform online calibration based on the first compensation value. The second position information indicates a preset reference position of the first component in the first image. This allows the vehicle sensor to be corrected based on the first compensation value even if a large-angle deviation occurs, ensuring that the sensor remains in good working condition.

[0085] Optionally, the vehicle may pre-configure the second location information in an offline manner.

[0086] In one possible implementation, the first type information and the first position information are obtained by inputting the first image into a first prediction model. The first prediction model is trained based on training samples, including the first sample image, the first sample type information, and the first sample position information. Thus, using the first prediction model, the detection of the type and position of the first component can be automated, improving processing efficiency and the reliability of detection results, further enhancing the efficiency of sensor calibration.

[0087] Optionally, the specific process of training the first prediction model based on the training sample can be: inputting the first sample image into the first prediction model to obtain the predicted first type information and first position information, training the first prediction model according to the first type information, first position information, first sample type information and first sample position information, when the first type information finally output by the first prediction model is close to the first sample type information, and the first position information is close to the first sample position information, the training of the first prediction model is completed.

[0088] Optionally, before applying the first prediction model, a pre-trained first prediction model may be configured in the vehicle.

[0089] In a possible implementation, the first prediction model may be a neural network model, such as CNN or RNN, or the first prediction model may be a machine learning algorithm, such as SVM.

[0090] In one possible implementation, the first position information includes position information of N feature points, where N is an integer greater than 1. Determining a first compensation value corresponding to the first extrinsic parameter based on the first position information and the second position information includes performing noise removal on the position information of the N feature points to obtain position information of Q feature points, where Q is less than or equal to N. Determining the first compensation value based on the position information of the Q feature points and the second position information. This allows for a more accurate first compensation value to be obtained, thereby further improving the accuracy of online sensor calibration in the vehicle.

[0091] For example, when the first component is a license plate, the first position information may include: position information of the left license plate corner point, position information of the left license plate nail corner point, position information of the right license plate corner point, and position information of the right license plate nail corner point.

[0092] Optionally, performing noise removal processing on the position information of the N feature points may include: performing noise removal by clustering the feature points and / or calculating a mean.

[0093] In one possible implementation, when the first type of information includes: performing noise removal on position information of N feature points to obtain position information of Q feature points, the method includes: fitting a first curve based on the position information of the N feature points; and when the distance between the M feature points and the first curve is greater than or equal to a preset threshold, removing the position information of the M feature points from the first position information to obtain the position information of the Q feature points, where the N feature points include M feature points, M is greater than or equal to 1, and M is less than N. In this way, for contour line-type feature information, effective noise removal can be achieved, thereby obtaining a more accurate first compensation value.

[0094] In one possible implementation, the first type of information includes characteristic information of a contour type, and determining the first extrinsic parameter based on the first type of information and the first correspondence includes selecting at least one Euler angle from a plurality of Euler angles as the first extrinsic parameter based on the characteristic information of the contour type and the first correspondence. This supports the use of contour features, making the calibration method applicable to a wider range of application scenarios.

[0095] For example, when the first type of information is the lower edge of the vehicle body, the yaw and roll angles may be selected as the first external parameters.

[0096] In one possible implementation, online calibration of a vehicle based on a first compensation value includes: performing multi-frame filtering on the first compensation value to obtain a second compensation value; and performing online calibration based on the second compensation value. This allows for a stable online calibration result (i.e., the second compensation value), further improving the accuracy of online sensor calibration.

[0097] Optionally, performing multi-frame filtering on the first compensation value may include: accumulating n first compensation values to obtain a set of statistical data, sorting the n first compensation values in the statistical data to remove abnormal values, and obtaining a second compensation value when the standard deviation of the statistical data is less than a preset threshold.

[0098] It should be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be referenced by each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0099] Figure 4 It is a schematic flow chart of a calibration method provided in an embodiment of the present application. Method 400 may be a specific description of steps S301 to S303 in method 300.

[0100] like Figure 4 As shown, method 400 can be divided into steps performed by an offline module and steps performed by an online module. The offline module can be used to configure basic parameters and neural networks. The purpose of setting the offline module is to configure different vehicle characteristics and calibrable external parameters for different sensors; the online module can obtain the sensor external parameter value by comparing the real-time vehicle feature position with the reference feature position provided by the offline module, thereby completing the calibration.

[0101] The offline module may execute steps S401 to S404.

[0102] S401: Determine available vehicle features in an image captured by a sensor.

[0103] Optionally, the available ego vehicle features may include: available ego vehicle features acquired by the sensor at a fixed position and available ego vehicle features acquired after the sensor has a large-angle deviation. When the sensor has a large-angle deviation, the ego vehicle features sensed by the sensor at the fixed position will be lost, and other available ego vehicle features will be added. Therefore, when performing sensor extrinsic parameter calibration, it is necessary to consider acquiring available ego vehicle features after the sensor has a large-angle deviation. The fixed position of the sensor may be the initial position of the sensor, and a large-angle deviation of the sensor may be understood as the angle at which the sensor deflects relative to the fixed position exceeds a preset threshold (for example, the yaw angle deflection is greater than or equal to 5°).

[0104] Optionally, the above-mentioned sensor may be a vehicle-mounted sensor, such as a camera device and a radar. Further optionally, the camera device may be a side-view fisheye camera.

[0105] For example, Figure 5As shown, the available vehicle features in the image captured by the side-view fisheye camera include: door gaps, logos, front wheel arches, lower edge of the vehicle, handlebars, and rear wheel arch outlines. The image captured by the side-view fisheye camera can be the first image in method 300.

[0106] S402, configuring the vehicle feature network.

[0107] For example, Figure 6 As shown, data collected by sensors (e.g., images) can be input into the ego-vehicle feature network to obtain the ego-vehicle feature category and location. The training data for the ego-vehicle feature network can be data collected by sensors, simulated images generated through mathematical model simulation or 3D reconstruction, or images enhanced from basic data (e.g., by selection, cropping, or perspective transformation).

[0108] The vehicle feature category may be indicated by the first type information in method 300 , the vehicle feature position may be indicated by the first position information in method 300 , and the vehicle feature network may be the first prediction model in method 300 .

[0109] It should be understood that the method of acquiring data and model training from the vehicle feature network in the embodiment of the present application is not unique, and those skilled in the art can make corresponding replacements based on actual needs.

[0110] S403: Configure a reference position corresponding to the vehicle feature category.

[0111] For example, configuring the reference position corresponding to the ego vehicle feature category may include the reference position of the ego vehicle feature category when the sensor is at a fixed position and after a large angle deviation. This reference position may refer to the position in the vehicle body coordinate system. The vehicle body coordinate system is used because the basic position of the ego vehicle feature in the vehicle body coordinate system is essentially unchanged, facilitating stable online calibration of the sensor even after a large angle deviation. This reference position may be indicated by the second position information in method 300.

[0112] Optionally, the reference position can be obtained through digital model information or three-dimensional reconstruction information of the vehicle.

[0113] Alternatively, a characteristic position of the vehicle in a reference coordinate system corresponding to the sensor, or a characteristic position of the vehicle in an image coordinate system may be used as a reference position.

[0114] S404: Configure scalable extrinsic parameters corresponding to the vehicle feature category.

[0115] Because the ego-vehicle characteristics within the sensor may not fully support the calibration of all six degrees of freedom (DOF) extrinsic parameters, forcibly calibrating all six degrees of freedom will result in degraded extrinsic parameter accuracy. Therefore, to ensure the accuracy of sensor extrinsic parameters during online calibration, a table of calibrable extrinsic parameters can be pre-configured based on the ego-vehicle characteristics in the sensor, and the calibrable extrinsic parameters can be determined based on this table.

[0116] The scalable external parameter table may be an example of the first corresponding relationship in method 300 , and the scalable external parameter may correspond to the first external parameter in method 300 .

[0117] Optionally, the calibrable external parameter table may be as shown in Table 1.

[0118] Table 1

[0119]

[0120] For example, Figure 7 As shown in the figure, the forward-looking fisheye camera's field of view includes four types of feature points: the left license plate corner, the left license plate nail corner, the right license plate corner, and the right license plate nail corner. According to the corresponding relationship in Table 1, the extrinsic parameters that can be marked are: roll angle, pitch angle, and yaw angle. At this time, if the fisheye camera has a large angle deviation, only the left license plate corner can be observed in the fisheye camera's field of view. According to the corresponding relationship in Table 1, the extrinsic parameters that can be marked are pitch angle and yaw angle.

[0121] The online module may execute steps S411 to S415 .

[0122] S411, ego-vehicle feature extraction.

[0123] For example, the online module can extract the vehicle feature category and sensor location through the vehicle feature network for subsequent online calibration. The point set corresponding to the extracted feature category is denoted as: S i-t(k) ={p i1 ,…,p ini}, where i represents the feature category, t(k) represents the time when the feature is acquired, and ni represents the number of feature points. If the current feature is a single-point feature, such as the left license plate point, then ni = 1.

[0124] S412, feature preprocessing.

[0125] For example, when extracting vehicle features in step S411, the extracted features may contain noise. In order to remove the noise or avoid false detection, the extracted features may be preprocessed.

[0126] Optionally, since the position of the ego-vehicle feature is relatively fixed, the detection results of multiple frames can be accumulated, and denoising can be performed by clustering the feature points based on the feature category, calculating the mean, etc.

[0127] For example, the mean value of single-point feature accumulation m frames can be calculated, that is, S i-t(k) ,S i-t(k) ,…,S i-t(k+m) Combine into a large point set S im , calculate the mean of the points in the set or consider the center point of the point set as the position of the true point.

[0128] S413, sensor pose optimization.

[0129] For example, after obtaining a point set that has undergone feature preprocessing, the positions of the reference feature points of the corresponding category and the optimizable extrinsic parameters corresponding to the effective feature combination can be identified. By projecting the reference feature points into the current sensor feature space, the sensor extrinsic parameters are optimized (the specific optimization type can be determined by the feature combination) so that the distance from the projected point to the actual feature point is minimized, that is:

[0130]

[0131] Among them, p ik represents the kth point in the currently detected i-th group of feature categories, p ik-ref Indicates p ik The corresponding reference point (the 3D position coordinates of the reference point can be written by the offline module), Represents the projection point of the reference point onto the image captured by the sensor through the external reference; ‖.‖ 2 Represents the calculation of the two-norm, which is the square of the distance between two points. Projection is the act of transforming the reference point into the coordinates of the current sensor coordinate system using the intrinsic and extrinsic coordinates.

[0132] The above process may correspond to the process of obtaining the first compensation value in method 300 .

[0133] When the sensor is a camera, the specific steps are as follows:

[0134] (1) Calculate the position of the reference feature point in the camera coordinate system using the following formula:

[0135] p ik-cam =R v2c (p ik-car -t xyz ) (1-2)

[0136] Among them, p ik-car (x v ,y v ,z v ) is the position of the reference feature point in the vehicle coordinate system, which is a three-dimensional vector. xyz (t x ,t y ,t z) is the position of the camera center in the vehicle coordinate system. v2c is a 3×3 matrix representing the rotation matrix from the vehicle body to the camera coordinate system. Since the degree of freedom of the rotation matrix is 3, the rotation matrix can also be expressed in Euler angles, yaw pitch roll, and a possible conversion relationship is:

[0137] When the rotation angle of the vehicle coordinate system to the camera coordinate system is R, the Euler angle calculation shows that the z-axis of the vehicle coordinate system is upward, the x-axis is forward, and the y-axis is left. Therefore, the angle of rotation around the z-axis is the yaw angle, the angle of rotation around the y-axis is the pitch angle, and the angle of rotation around the x-axis is the roll angle. The rotation order is yaw→pitch→roll. It can be seen that:

[0138]

[0139] (2) Project the position of the reference point in the camera coordinate system into the image captured by the camera.

[0140] Since there is no depth information in the image, the points in the camera coordinate system need to be normalized using the following formula:

[0141]

[0142] Among them, p ik-cam (x ik-cam ,y ik-cam ,z ik-cam ) represents the position of the point in the camera coordinate system, z ik-cam Indicates depth.

[0143] (3) The normalized points are transformed into the image coordinate system through the internal parameters, which can be achieved through the formula.

[0144]

[0145] Among them, K represents the 3×3 internal parameter matrix, which can be expressed as fx and fy are the focal lengths, and cx and cy are the image center positions. is the position coordinate of the reference feature point in the final image, The homogeneous representation of , dist() indicates distortion processing.

[0146] S414, multi-frame filtering processing.

[0147] In step S413 , calibration data for a period of time is accumulated, and the accumulated calibration data can be filtered to obtain an online calibration result. The specific filtering process will be introduced in method 800 .

[0148] S415, output the calibrated external parameters.

[0149] Specifically, after obtaining the multi-frame filtering results in step S414, the compensation value of the extrinsic parameter (ie, the second compensation value in method 300) can be output for updating the sensor extrinsic parameter or issuing a sensor abnormality alarm.

[0150] In the embodiments of this application, since the reference position of the vehicle's characteristics is already written into the offline module, the vehicle's sensor deviation detection and compensation capabilities are supported as soon as the vehicle leaves the production line, eliminating the need for calibration in a calibration room, thereby reducing sensor calibration costs. Furthermore, the coordinated use of the offline and online modules can improve the accuracy of online sensor calibration in the vehicle, allowing calibration to be performed whether the vehicle is stationary or in motion.

[0151] Figure 8 It is a schematic flow chart of a multi-frame filtering method provided in an embodiment of the present application. Method 800 can be applied to step S414 of method 400. Method 800 can include steps S801 to S805.

[0152] S801, obtaining a single external parameter calibration result.

[0153] S802: Accumulate n calibration results to obtain a set of statistical data.

[0154] Wherein, n can be a positive integer.

[0155] S803, sorting the statistical data to remove outliers.

[0156] S804: Determine whether the standard deviation of the statistical data is less than or equal to a preset threshold.

[0157] S805: Output online calibration results.

[0158] Specifically, in step S804 , when the standard deviation of the statistical data is less than or equal to a preset threshold, in step S805 , the online calibration result in the statistical data (eg, the second compensation value in method 300 ) may be output.

[0159] In the embodiment of the present application, by performing multi-frame filtering processing on the result of a single external parameter calibration, a relatively accurate online calibration result can be obtained, thereby further improving the accuracy of sensor calibration.

[0160] Figure 9 It is a schematic flowchart of another calibration method provided in an embodiment of the present application. Method 900 may be a specific description of steps S301 to S303 in method 300.

[0161] like Figure 9As shown, method 900 can be divided into steps performed by an offline module and steps performed by an online module. The offline module can be used to configure basic parameters and networks. The purpose of setting the offline module is to configure different vehicle characteristics and calibrable external parameters for different sensors; the online module can obtain the actual sensor external parameter value by comparing the real-time vehicle characteristic position with the reference characteristic position provided by the offline module, thereby completing the calibration.

[0162] The offline module may execute steps S901 to S904 .

[0163] S901: Determine available vehicle features in an image captured by a sensor.

[0164] Optionally, the vehicle feature may be a contour-type feature, which may include one or more of the following: a vehicle edge, a door gap, a vehicle handle, a front bumper, and a license plate. The vehicle edge may be understood as the area below the door and the lower edge of the vehicle body.

[0165] S902: Configure the vehicle feature network.

[0166] Alternatively, you can use Figure 6 The ego-vehicle feature network shown processes the ego-vehicle features to obtain the ego-vehicle feature position and category.

[0167] The vehicle feature category may be indicated by the first type information in method 300 , the vehicle feature position may be indicated by the first position information in method 300 , and the vehicle feature network may be the first prediction model in method 300 .

[0168] S903: Process the feature position and category of the vehicle.

[0169] Since the contour type features have many feature points, it is not appropriate to set all the contours as feature points. In addition, since the contour type features have obvious curve forms, the contour features can be fitted with 3D lines in the vehicle body coordinate system (curve fitting or segmented fitting), and the curves and some key 3D feature points (for example, the contour starting point, turning points, etc.) can be stored.

[0170] S904: Determine the calibrable angle corresponding to the contour feature.

[0171] like Figure 10 As shown in , taking the lower edge of the vehicle body (hereinafter referred to as the lower edge) as an example, we can first obtain the position change of the lower edge contour after the three Euler angles yawpitch roll deviate and are projected into the image. Figure 10It can be seen that the contour feature is more sensitive to changes in yaw and roll, that is, there will be obvious up and down changes when deviation occurs, but it is relatively insensitive to changes in the pitch angle. The pitch angle only changes in the extension direction of the vehicle (in actual application, due to changes in viewing angle, it is difficult to determine the starting position of the contour), while the up and down position changes are not obvious. Therefore, the calibrable angles corresponding to this contour feature can be the yaw and roll angles.

[0172] The online module may execute steps S911 to S915 .

[0173] S911, ego-vehicle feature extraction.

[0174] For example, the online module can extract the ego vehicle feature category and position through the ego vehicle feature network for subsequent online calibration.

[0175] S912, feature preprocessing.

[0176] For example, when extracting vehicle features in step S911, the extracted features may contain noise. In order to remove the noise or avoid false detection, the extracted features may be preprocessed.

[0177] Optionally, since the position of the ego-vehicle feature is relatively fixed, the detection results of multiple frames can be accumulated to remove noise. That is, after accumulating multiple frames, feature points that are far away from the line can be removed by line fitting.

[0178] For example, for a line-type contour feature S m-1 , fitting curve f to the points in the set l (p), the distance between each point and the fitted curve can be calculated. If it is greater than a certain threshold, it is considered as noise and removed. Figure 11 As shown, taking the left fisheye camera as an example, the method of removing noise is:

[0179] (1) Using the image x-axis and y-axis, the vehicle characteristic point curve L (corresponding to Figure 11 ), the curve L may be the first curve in the method 300 .

[0180] (2) For a point p(x j ,y j ), its horizontal axis coordinate x j Substitute into the function to get the nearest point y j-est =f(x j ), then p j-est (x j ,y j-est ) can be regarded as p(x j ,y j ) to the corresponding point on the fitting line.

[0181] (3) The distance from point p to curve f is defined as Dist(p,L)=(y j-est -y j ) 2 If the distance exceeds a certain value, the viewpoint p is removed as a noise point, otherwise, point p can be retained.

[0182] S913, sensor pose optimization.

[0183] For example, point sampling is performed on the reference position curve corresponding to the vehicle edge to obtain reference points of the vehicle edge contour. These reference points are projected onto the image of the fisheye camera using the extrinsic parameters of the sensor. The calibrated extrinsic parameters are optimized to minimize the distance between these reference points and the vehicle edge curve extracted from the real-time image. This can be specifically achieved using the following formula:

[0184]

[0185] in, Indicates that the external parameters of other feature points except the vehicle edge feature point are optimized. The specific calculation method is the same as formula (1-1) and will not be repeated here. If there are no feature points except the vehicle edge feature point, this item in the formula is 0; Represents the point where the reference feature point is projected into the image Fit the curve L to the image j distance.

[0186] S914, multi-frame filtering processing.

[0187] Among them, in step S913, calibration data for a period of time is accumulated, and the accumulated calibration data can be filtered to obtain an online calibration result. The specific filtering process has been introduced in method 800 and will not be repeated here.

[0188] S915: Output the calibrated external parameters.

[0189] Specifically, after obtaining the multi-frame filtering results in step S915, the external parameters can be output for updating the sensor external parameters or issuing a sensor abnormality alarm.

[0190] In an embodiment of the present application, when the features of the contour line type are obtained based on the image collected by the sensor, the offline module and the online module are used in conjunction to more efficiently determine the calibrable external parameters, and the calibration method can be used in more application scenarios.

[0191] Figure 121 is a schematic diagram of a calibration device 1200 provided in an embodiment of the present application. The device 1200 may include an acquisition unit 1210, a storage unit 1220, and a processing unit 1230. The acquisition unit 1210 is used to acquire instructions and / or data, the storage unit 1220 is used to implement corresponding storage functions and store corresponding instructions and / or data, and the processing unit 1230 is used to perform data processing to enable the device 1200 to implement the aforementioned calibration method.

[0192] The device 1200 includes: an acquisition unit 1210 and a processing unit 1230; the acquisition unit 1210 is used to acquire a first image captured by a vehicle-mounted sensor; the processing unit 1230 is used to: determine first type information of a first component in the first image; determine a first external parameter based on the first type information and a first correspondence, the first correspondence being a correspondence between a component type and a calibrable external parameter, the calibrable external parameter including the first external parameter; and perform online calibration based on the first external parameter.

[0193] In one possible implementation, the processing unit 1230 is further used to: determine the first position information of the first component in the first image; the processing unit 1230 is specifically used to: determine the first compensation value corresponding to the first external parameter based on the first position information and the second position information, the second position information being used to indicate a preset reference position of the first component in the first image; and perform online calibration based on the first compensation value.

[0194] In one possible implementation, the first type information and the first position information are obtained by inputting the first image into a first prediction model. The first prediction model is obtained based on a training sample, and the training sample includes: a first sample image, first sample type information and first sample position information.

[0195] In one possible implementation, the first position information includes: position information of N feature points, where N is an integer greater than 1; the processing unit 1230 is specifically used to: remove noise from the position information of the N feature points to obtain position information of Q feature points; and determine a first compensation value based on the position information of the Q feature points and the second position information.

[0196] In one possible implementation, the processing unit 1230 is specifically used to: fit a first curve based on the position information of N feature points; when the distance between the M feature points and the first curve is greater than or equal to a preset threshold, remove the position information of the M feature points from the first position information to obtain the position information of Q feature points, where the N feature points include M feature points, M is greater than or equal to 1, and M is less than N.

[0197] In one possible implementation, the first type of information includes: characteristic information of the contour type; and the processing unit 1230 is specifically configured to select at least one Euler angle from a plurality of Euler angles as the first extrinsic parameter based on the characteristic information of the contour type and the first corresponding relationship.

[0198] In a possible implementation, the processing unit 1230 is specifically configured to: perform multi-frame filtering on the first compensation value to obtain a second compensation value; and perform online calibration according to the second compensation value.

[0199] Alternatively, if the device 1200 is located in the vehicle 100, the processing unit 1230 may be Figure 1 The processor 131 is shown.

[0200] Figure 13 It is a schematic diagram of another calibration device 1300 provided in an embodiment of the present application.

[0201] The device 1300 includes a memory 1310, a processor 1320, and a communication interface 1330. The memory 1310, processor 1320, and communication interface 1330 are connected via an internal connection path. The memory 1310 is used to store instructions, and the processor 1320 is used to execute the instructions stored in the memory 1310 to control the communication interface 1330 to obtain information, so that the device 1300 implements the aforementioned calibration method. Optionally, the memory 1310 can be coupled to the processor 1320 via an interface or integrated with the processor 1320.

[0202] It should be noted that the communication interface 1330 may be a transceiver such as, but not limited to, a transceiver. The communication interface 1330 may also include an input / output interface.

[0203] The processor 1320 stores one or more computer programs, which include instructions. When the instructions are executed by the processor 1320, the calibration device 1300 executes the calibration method in the above embodiments.

[0204] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 1320 or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1310, and the processor 1320 reads the information in the memory 1310 and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0205] Optionally, Figure 13 The communication interface 1330 can be implemented Figure 12 The acquisition unit 1210 in Figure 13 The memory 1310 in the embodiment can be implemented Figure 12 The storage unit 1230, Figure 13 The processor 1320 in the embodiment can implement Figure 12 The processing unit 1230 in .

[0206] Optionally, the device 1200 or the device 1300 may be located at Figure 1 Among the vehicles 100.

[0207] Optionally, the device 1200 or the device 1300 may be Figure 1 A computing platform 130 in the vehicle.

[0208] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program code, and when the computer program code is run on a computer, the computer executes the above Figures 3 and 4 ,or Figures 8 and 9 Any of the methods in .

[0209] The present application also provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above Figures 3 and 4 ,or Figures 8 and 9 Any of the methods in .

[0210] The embodiment of the present application also provides a chip, including: a circuit, which is used to execute the above Figures 3 and 4 ,or Figures 8 and 9 Any of the methods in .

[0211] The embodiment of the present application also provides a vehicle, including Figure 12 or Figure 13 Any of the calibration devices shown.

[0212] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0213] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0215] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0216] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0217] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0218] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A calibration method, characterized in that: The method comprises: Acquire a first image captured by a vehicle-mounted sensor; determining first type information of a first component in the first image; Determining a first external parameter according to the first type information and a first corresponding relationship, where the first corresponding relationship is a corresponding relationship between a component type and a calibratable external parameter, and the calibratable external parameter includes the first external parameter; Perform online calibration based on the first external parameter.

2. The method according to claim 1, wherein The method further comprises: determining first position information of the first component in the first image; The performing online calibration according to the first external parameter includes: determining a first compensation value corresponding to the first extrinsic parameter based on the first position information and the second position information, wherein the second position information indicates a preset reference position of the first component in the first image; Online calibration is performed according to the first compensation value.

3. The method according to claim 2, wherein The first type information and the first position information are obtained by inputting the first image into a first prediction model, and the first prediction model is obtained based on a training sample, and the training sample includes: a first sample image, first sample type information and first sample position information.

4. The method according to claim 2 or 3, wherein: The first position information includes: position information of N feature points, where N is an integer greater than 1. Determining a first compensation value corresponding to the first external parameter based on the first position information and the second position information includes: Denoising the position information of the N feature points to obtain position information of Q feature points, where Q is less than or equal to N; The first compensation value is determined according to the position information of the Q feature points and the second position information.

5. The method according to claim 4, wherein The performing noise removal on the position information of the N feature points to obtain the position information of the Q feature points includes: Fitting a first curve according to the position information of the N feature points; When the distance between the M feature points and the first curve is greater than or equal to a preset threshold, the position information of the M feature points is removed from the first position information to obtain the position information of the Q feature points, the N feature points include the M feature points, M is greater than or equal to 1, and M is less than N.

6. The method according to any one of claims 2 to 5, characterized in that The performing online calibration according to the first compensation value includes: performing multi-frame filtering on the first compensation value to obtain a second compensation value; Online calibration is performed according to the second compensation value.

7. The method according to any one of claims 1 to 6, characterized in that The first type of information includes: characteristic information of a contour type, and determining the first external parameter according to the first type of information and the first corresponding relationship includes: At least one Euler angle is selected from a plurality of Euler angles as the first extrinsic parameter according to the characteristic information of the contour line type and the first corresponding relationship.

8. A calibration device, characterized in that: The device includes an acquisition unit and a processing unit; The acquisition unit is configured to acquire a first image captured by the vehicle-mounted sensor; The processing unit is configured to: determining first type information of a first component in the first image; Determining a first external parameter according to the first type information and a first corresponding relationship, where the first corresponding relationship is a corresponding relationship between a component type and a calibratable external parameter, and the calibratable external parameter includes the first external parameter; Perform online calibration based on the first external parameter.

9. The device according to claim 8, wherein The processing unit is further configured to: determine first position information of the first component in the first image; The processing unit is specifically configured to: determining a first compensation value corresponding to the first extrinsic parameter based on the first position information and the second position information, wherein the second position information indicates a preset reference position of the first component in the first image; Online calibration is performed according to the first compensation value.

10. The device according to claim 9, characterized in that The first type information and the first position information are obtained by inputting the first image into a first prediction model, and the first prediction model is obtained based on a training sample, and the training sample includes: a first sample image, first sample type information and first sample position information.

11. The device according to claim 9 or 10, characterized in that The first position information includes: position information of N feature points, where N is an integer greater than 1; The processing unit is specifically configured to: Denoising the position information of the N feature points to obtain position information of Q feature points, where Q is less than or equal to N; The first compensation value is determined according to the position information of the Q feature points and the second position information.

12. The device according to claim 11, wherein The processing unit is specifically configured to: Fitting a first curve according to the position information of the N feature points; When the distance between the M feature points and the first curve is greater than or equal to a preset threshold, the position information of the M feature points is removed from the first position information to obtain the position information of the Q feature points, the N feature points include the M feature points, M is greater than or equal to 1, and M is less than N.

13. The device according to any one of claims 9 to 12, characterized in that The processing unit is specifically configured to: performing multi-frame filtering on the first compensation value to obtain a second compensation value; Online calibration is performed according to the second compensation value.

14. The device according to any one of claims 8 to 13, characterized in that The first type of information includes: characteristic information of the contour type; The processing unit is specifically configured to select at least one Euler angle from a plurality of Euler angles as the first extrinsic parameter according to the feature information of the contour line type and the first corresponding relationship.

15. A control device, characterized in that: The method comprises a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store computer programs or instructions, and the processor is used to execute the computer program or instructions in the memory, so that the method according to any one of claims 1 to 7 is executed. 16 . A chip comprising a circuit for executing the method according to claim 1 . 17 . A computer-readable storage medium storing a program code, wherein when the computer program code is executed on a computer, the computer is caused to execute the method according to claim 1 .

18. A vehicle, characterized in that: Comprising the device according to any one of claims 8 to 14.

Images