Camera external parameter optimization method and vehicle-mounted terminal

Abstract

The invention provides a camera external parameter optimization method and a vehicle-mounted terminal. The camera external parameter optimization method comprises the following steps: acquiring a characteristic angular point parameter and a vehicle pose parameter based on a camera installation pose so as to acquire a re-projection error and a pose error; further solving a partial derivative based on the re-projection error and the pose error so as to obtain a target extrinsic parameter; and finally, judging whether the target external parameters meet preset optimization requirements or not, and taking the target external parameters as camera external parameters when the target external parameters meet the preset optimization requirements. The method does not need to depend on a specific calibration object or environment, and does not need to spend a large amount of expenditure to construct a calibration field, thereby reducing the cost. And moreover, a tedious operation process is not needed, and a calculation mode is relatively simple.

Description

Technical Field

[0001] The present application relates to the technical field of external parameter optimization, and in particular to a camera external parameter optimization method and a vehicle-mounted terminal. Background Art

[0002] In the automatic parking system, accurately calibrated camera extrinsics are crucial to achieve accurate environmental perception and vehicle positioning. In actual daily use, the vehicle may be affected by factors such as tire pressure changes and load changes, causing the calibrated camera extrinsics to no longer adapt to the current vehicle status.

[0003] Traditional camera extrinsic calibration methods often rely on specific calibration objects or calibration environments, which are difficult to implement in complex parking scenarios.

[0004] Another camera extrinsic calibration method is to optimize the camera extrinsic parameters by constructing a three-dimensional model of the environment. This usually involves a large number of image acquisition, feature extraction and matching, and three-dimensional point cloud generation steps. Each step requires complex calculations and processing, resulting in a huge overall amount of calculations, high requirements on computing resources and is relatively time-consuming, which makes it subject to certain limitations in practical applications. Summary of the invention

[0005] In order to solve the above-mentioned technical problems, the present application provides a camera extrinsic parameter optimization method and a vehicle-mounted terminal that do not rely on specific scenarios and have a simple calculation process.

[0006] Specifically, the present application provides a camera extrinsic parameter optimization method, which uses the camera installation posture as the initial extrinsic parameter. The camera extrinsic parameter optimization method includes the following steps:

[0007] S100: Acquire parking space co-view information based on the initial external parameters, generate feature corner point parameters according to the parking space co-view information, and simultaneously acquire vehicle posture parameters.

[0008] S200: Obtain a reprojection error according to the initial external parameters, feature corner point parameters and vehicle posture parameters, and obtain a posture error according to the vehicle posture parameters.

[0009] S300: calculating partial derivatives of the feature corner point parameters and initial extrinsic parameters simultaneously according to the reprojection error to obtain target corner point parameters and target extrinsic parameters, and calculating partial derivatives of the vehicle posture parameters simultaneously according to the reprojection error and posture error to obtain target posture.

[0010] And S400: judging whether the target external parameter meets the preset optimization requirement based on the target posture and the target corner point parameter, so as to update the initial external parameter to the target external parameter if it meets the requirement.

[0011] In the above technical solution, there is no need to rely on specific calibration objects or environments, and the camera external parameters can be optimized in complex parking scenarios. There is no need to spend a lot of money to build a calibration site for camera calibration, which improves the applicability and flexibility of the system while reducing costs. The calculation and update will be automatically completed in the vehicle background, without the need for cumbersome operation procedures, and the calculation method is relatively simple.

[0012] Furthermore, the step S100 of acquiring the parking space co-viewing information includes:

[0013] S101: Detecting the marked parking space information and the parking space corner point information within a preset range of the target vehicle based on the initial external parameters to obtain real-time parking space information, and using the real-time parking space information obtained last time as historical parking space information.

[0014] S102: Convert the historical parking space information to the vehicle coordinate system corresponding to the real-time parking space information, and match the real-time parking space information with the historical parking space information. If the match is successful, store the real-time parking space information in a first cache corresponding to the historical parking space information; otherwise, create a second cache to store the real-time parking space information in the second cache.

[0015] In the above technical solution, the real-time parking space information obtained last time is used as the historical parking space information, which helps the system track the changes in the parking space information and improve the accuracy of detection and optimization; the historical parking space information is converted into the vehicle coordinate system corresponding to the real-time parking space information, which effectively unifies the data format and enables the parking space information to be matched under the same coordinate system, avoiding the problem of data inconsistency caused by the difference in coordinate systems; by matching the real-time parking space information with the historical parking space information, the system can identify and process the changes in the parking space information, effectively manage and store the data of successful or failed matching, and avoid data redundancy in the cache.

[0016] Furthermore, after step S102, the method further includes:

[0017] Determine whether the real-time parking space information detection within the preset range has been completed. If it has been completed, obtain the parking space co-viewing information based on the first cache and the second cache; otherwise, return to step S101.

[0018] In the above technical solution, by judging whether the real-time parking space information detection within the preset range is completed, the system can ensure that it obtains sufficiently comprehensive parking space information, which helps to provide a complete parking space co-viewing information and ensure that the optimization process is based on comprehensive data.

[0019] Furthermore, the generating of characteristic corner point parameters in step S100 includes:

[0020] The parking space corner point information is matched based on the parking space common view information to obtain the parking space corner point information with a matching number greater than a preset matching threshold as a feature corner point parameter.

[0021] In the above technical solution, corner points with a high number of matches usually have higher credibility and can more accurately reflect the characteristics of parking corner points in the actual environment; by filtering out parking corner points with a large number of matches, the system can select more stable and recurring corner points as features. These stable feature points perform consistently under different environmental conditions, reducing errors caused by noise or changes.

[0022] Furthermore, each real-time parking space information corresponds to a frame of detection image, and the step S100 of obtaining the vehicle posture parameters includes:

[0023] The wheel speed and angular velocity corresponding to each frame of the detection image are synchronized, and the wheel information of the target vehicle is obtained.

[0024] And the vehicle posture parameters are obtained according to the synchronized wheel speed and angular velocity and the wheel information.

[0025] In the above technical solution, the wheel speed and angular velocity of each frame of detection image are synchronized and combined with the wheel information to obtain the vehicle's motion state more comprehensively. The processing of this comprehensive information helps to more accurately calculate the vehicle's posture parameters, thereby improving the overall positioning accuracy; the accurately acquired vehicle posture parameters can provide more refined data support for subsequent vehicle control and navigation, which is crucial to achieving high-precision autonomous driving or automatic parking functions.

[0026] Furthermore, the step S200 includes:

[0027] The characteristic corner point parameters, vehicle posture parameters and initial extrinsic parameters are all converted to the starting point coordinate system to obtain detection corner points, detection posture and detection extrinsic parameters.

[0028] And respectively calculate the reprojection error of each detection corner point according to the detection corner point, detection external parameter and detection posture corresponding to each frame of detection image; at the same time, calculate the posture error according to the detection posture of two adjacent frames of detection images.

[0029] In the above technical solution, the feature corner point parameters, vehicle posture parameters and initial extrinsic parameters are converted to the starting point coordinate system, ensuring that all data are processed in the same coordinate system. This unified coordinate system method avoids data errors caused by different coordinate systems, and helps to more accurately calculate and analyze reprojection errors and posture errors; by calculating the reprojection errors of each detected corner point, the projection accuracy of the feature corner point in the image can be effectively evaluated. This measurement helps to identify and correct errors in the camera model, thereby improving the accuracy of the overall camera calibration; calculating the posture error based on the detection posture of two adjacent frames of detection images can accurately monitor the posture changes of the vehicle during movement, which helps to identify and correct posture deviations caused by sensor drift or model errors, and ensure the stability and reliability of the system in a dynamic environment.

[0030] In addition, by analyzing the reprojection error and pose error, the system can perform targeted optimization to improve the camera extrinsic parameters and vehicle pose estimation. This error-driven optimization method can effectively improve system performance and reduce the error in the final result.

[0031] Furthermore, after completing step S100, the method further includes:

[0032] Determine whether the parking space co-viewing information and the vehicle posture parameters meet the optimization activation requirements. If so, execute step S200; otherwise, after entering another parking perception scene, return to step S100; wherein, the optimization activation requirements include that the vehicle motion state in the current parking perception scene at least includes left turn, right turn, straight driving and stationary, and includes at least preset marked parking spaces.

[0033] In the above technical solution, step S200 will only be executed when the parking space co-view information and the vehicle posture parameters meet the optimization start-up requirements. This conditional triggering mechanism can ensure that the optimization process is only carried out when the data quality is high enough, thereby improving the efficiency and effect of the optimization; if the parking space co-view information or the vehicle posture parameters do not meet the optimization requirements, step S200 is skipped, avoiding unnecessary calculations and processing, saving computing resources, reducing system burden, and improving overall efficiency.

[0034] Furthermore, the step S400 further includes:

[0035] When the target external parameter does not meet the preset optimization requirements, the preset range is increased, the parking perception scene is changed, or an abnormality is reported.

[0036] In the above technical solution, the system can dynamically adjust the optimization strategy according to the actual situation, improve the data processing capabilities, and can flexibly process under different data conditions to ensure the validity and accuracy of the data; and the flexible processing method not only improves the system stability and processing efficiency, but also reduces the failure rate and improves the user experience.

[0037] Furthermore, after step S400, the method further includes: returning to step S100 based on the target external parameter to continue iterative optimization.

[0038] In the above technical solution, through an iterative optimization process, the camera extrinsic parameters can be adjusted in real time when the vehicle state changes, so that the vehicle system always maintains high-precision environmental perception and vehicle positioning.

[0039] Based on the same concept, the present application also provides a vehicle-mounted terminal, which is at least equipped with a camera and a camera extrinsic parameter optimization system, and the camera extrinsic parameter optimization system uses the camera extrinsic parameter optimization method to optimize the extrinsic parameters of the camera.

[0040] In the above technical solution, the vehicle-mounted terminal based on the camera extrinsic parameter optimization method can significantly improve the camera calibration accuracy, enhance the performance of the autonomous driving system, optimize data processing efficiency, and reduce system maintenance costs by configuring the camera and the camera extrinsic parameter optimization system; the intelligent optimization process not only improves the user experience, but also improves the reliability and adaptability of the system. The intelligent tuning and scalability of the system further enhance the comprehensive performance of the vehicle-mounted terminal, enabling it to perform well in various application scenarios.

[0041] Compared with the prior art, the beneficial effects of this application are:

[0042] This application obtains feature corner point parameters and vehicle pose parameters based on the camera installation pose; then obtains the reprojection error based on the camera installation pose, feature corner point parameters and vehicle pose parameters, and obtains the pose error based on the vehicle pose parameters; further obtains the partial derivative based on the reprojection error and the pose error, and then obtains the target extrinsic parameter; finally, determines whether the target extrinsic parameter meets the preset optimization requirements, and if so, uses the target extrinsic parameter as the camera extrinsic parameter.

[0043] This application does not need to rely on specific calibration objects or environments, can optimize camera extrinsic parameters in complex parking scenarios, and does not require a large amount of funds to build a calibration site for camera calibration, thus reducing costs; and does not require cumbersome operation procedures, and the calculation method is relatively simple. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of the camera extrinsic parameter optimization method described in this application.

[0045] Figure 2 This is a flow chart of the method for obtaining parking space co-view information described in this application.

[0046] Figure 3 This is a framework diagram of the vehicle-mounted terminal described in this application. DETAILED DESCRIPTION

[0047] A camera extrinsic parameter optimization method and a vehicle-mounted terminal of the present application are further described in detail below in conjunction with specific embodiments and drawings.

[0048] See also Figure 1 The embodiment of the present application provides a camera extrinsic parameter optimization method, which uses the camera installation posture as the initial extrinsic parameter. The camera extrinsic parameter optimization method includes the following steps:

[0049] S100: Acquire parking space co-view information based on the initial external parameters, generate feature corner point parameters according to the parking space co-view information, and simultaneously acquire vehicle posture parameters.

[0050] For further information, see Figure 2 The step S100 of acquiring the parking space co-viewing information includes:

[0051] S101: Detecting the marked parking space information and the parking space corner point information within a preset range of the target vehicle based on the initial external parameters to obtain real-time parking space information, and using the real-time parking space information obtained last time as historical parking space information.

[0052] In some embodiments, a camera installation posture designed with a mechanical structure is used as an initial value of a camera extrinsic parameter, namely, the initial extrinsic parameter; and then based on the initial extrinsic parameter, a preset frame image within a preset range of the target vehicle is acquired through the camera.

[0053] It should be noted that the camera extrinsic parameter optimization method is usually applied in vehicle parking scenarios, and the preset range is, for example, the parking lot where the target vehicle is located.

[0054] Optionally, the image acquired in real time is analyzed through image processing or sensor data processing, and the midpoint coordinates of the marked parking space (i.e., the marked parking space information) and the coordinates of the four corner points of the marked parking space are extracted, and the midpoint coordinates and the coordinates of the corner points are used as the real-time parking space information; wherein, the vehicle coordinate system of the real-time parking space information is determined based on the current position of the target vehicle.

[0055] Optionally, the vehicle coordinate system takes the center of the target vehicle as the origin, the vehicle's forward direction is the positive direction of the Y axis, and the right side of the forward direction is the positive direction of the X axis.

[0056] It should be noted that the division of each coordinate axis of the vehicle coordinate system is not limited to the above-mentioned one, and other forms of expression are also possible.

[0057] S102: Convert the historical parking space information to the vehicle coordinate system corresponding to the real-time parking space information, and match the real-time parking space information with the historical parking space information. If the match is successful, store the real-time parking space information in a first cache corresponding to the historical parking space information; otherwise, create a second cache to store the real-time parking space information in the second cache.

[0058] In some embodiments, the midpoint coordinates and corner point coordinates of the previously detected parking space are converted to the vehicle coordinate system of the current target vehicle and matched. If the match is successful, the current real-time parking space information is updated to the cache area (i.e., the first cache) corresponding to the previously detected historical parking space information; otherwise, a new cache area (i.e., the second cache) is created to store the real-time parking space information.

[0059] Optionally, whether the current real-time parking space information and the historical parking space information match is determined by calculating the intersection over union (IOU) ratio; IOU = intersection area / union area; then a threshold (such as 0.5) is set, and when the IOU is greater than this threshold, the match is considered successful.

[0060] In other embodiments, whether the real-time parking space information matches the historical parking space information may also be determined by using a geometric transformation model, Euclidean distance, a corner point detection algorithm, and the like.

[0061] The Euclidean distance between the coordinates of the midpoints or the coordinates of the corner points may be calculated, and if the distance is within a preset threshold range, the match is considered successful.

[0062] Alternatively, geometric transformation models such as affine transformation or perspective transformation are used to match the shape of the parking space.

[0063] Alternatively, Harris corner detection is used to extract corner points of parking spaces, and then a feature matching algorithm such as FLANN (Fast Library for Approximate Nearest Neighbors) is used for matching.

[0064] In addition, each piece of real-time parking space information uniquely corresponds to a parking space ID, and the parking space ID and the corresponding real-time parking space information are stored in the corresponding cache at the same time.

[0065] It should be noted that the first cache and the second cache are only used to distinguish the description for convenience, and the second cache is a newly created cache area, but the two are essentially the same and exist in the same memory.

[0066] In the above technical solution, the real-time parking space information obtained last time is used as the historical parking space information, which helps the system track the changes in the parking space information and improve the accuracy of detection and optimization; the historical parking space information is converted into the vehicle coordinate system corresponding to the real-time parking space information, which effectively unifies the data format and enables the parking space information to be matched under the same coordinate system, avoiding the problem of data inconsistency caused by the difference in coordinate systems; by matching the real-time parking space information with the historical parking space information, the system can identify and process the changes in the parking space information, effectively manage and store the data of successful or failed matching, and avoid data redundancy in the cache.

[0067] Furthermore, after step S102, the method further includes:

[0068] Determine whether the real-time parking space information detection within the preset range has been completed. If it has been completed, obtain the parking space co-viewing information based on the first cache and the second cache; otherwise, return to step S101.

[0069] In some embodiments, the co-viewing relationship generally refers to the same parking space visible in multiple frames of images. The parking space co-viewing relationship can be recorded by a mapping table or a graph database to record the co-viewing relationship between real-time parking space information.

[0070] In other embodiments, the co-viewing relationship between real-time parking space information may also be recorded through a spatial index database, a knowledge graph, etc.

[0071] The specific method for recording the co-viewing relationship can be selected by those skilled in the art according to actual application requirements.

[0072] In the above technical solution, by judging whether the real-time parking space information detection within the preset range is completed, the system can ensure that it obtains sufficiently comprehensive parking space information, which helps to provide a complete parking space co-viewing information and ensure that the optimization process is based on comprehensive data.

[0073] Furthermore, the generating of characteristic corner point parameters in step S100 includes:

[0074] The parking space corner point information is matched based on the parking space common view information to obtain the parking space corner point information with a matching number greater than a preset matching threshold as a feature corner point parameter.

[0075] In some embodiments, four corner points of each frame image are extracted according to the parking space co-viewing relationship, a descriptor is generated for each corner point, a feature point matching algorithm is used to perform corner point matching between different parking space images, and reliable matching point pairs are screened out according to the matching results; the number of occurrences of each corner point in all matches is counted, and the parking space corner point information with a matching number greater than a preset matching threshold (such as 10 times) is selected as the feature corner point parameter. These stable feature points can provide better visual constraints for subsequent external parameter optimization.

[0076] It should be noted that those skilled in the art can set the preset matching threshold according to actual application requirements, not limited to 10 times only.

[0077] In the above technical solution, corner points with a high number of matches usually have higher credibility and can more accurately reflect the characteristics of parking space corner points in the actual environment; by filtering out parking space corner points with a large number of matches, the system can select more stable and repeatedly appearing corner points as features, and these stable feature points perform consistently under different environmental conditions, reducing errors caused by noise or changes.

[0078] Further, each real-time parking space information corresponds to a frame of detection image, and the obtaining of the vehicle pose parameters in step S100 includes:

[0079] Synchronize the wheel speed and angular velocity corresponding to each frame of detection image, and obtain the wheel information of the target vehicle.

[0080] In some embodiments, synchronize the wheel speed and IMU information (i.e., angular velocity) corresponding to each frame of image. Assume that the timestamps of the nth and n + 1th frames of images are t0 and t1. If the timestamps of the wheel speed and IMU satisfy t0 < t ≤ t1, then synchronize them to the (n + 1)th frame of image as the observation value at the t1 moment. Assume that m frames of wheel speed and k frames of IMU information are synchronized at the t1 moment (both m and k are integers greater than zero), which can be expressed as:

[0081] IMU: (w0, w1,... w k ) Angular velocity

[0082] WHEEL: (Wh0, Wh1,... Wh m ) Number of wheel speed pulses

[0083] The wheel information includes at least the wheel circumference and the number of teeth of the wheel.

[0084] Further, obtain the vehicle pose parameters according to the synchronized wheel speed and angular velocity, and the wheel information.

[0085] Specifically, calculate the vehicle pose through the synchronized data to obtain the vehicle pose parameters:

[0086]

[0087] R n+1 = ΔR * R n

[0088]

[0089] Among them, R n+1 and P n+1represents the rotational component and translational component of the vehicle posture at the n+1th moment, w represents the angular velocity of the IMU gyroscope, and ΔS represents the vehicle displacement distance calculated using the wheel speed pulse WHEEL.

[0090] In the above technical solution, the wheel speed and angular velocity of each frame of detection image are synchronized and combined with the wheel information to obtain the vehicle's motion state more comprehensively. The processing of this comprehensive information helps to more accurately calculate the vehicle's posture parameters, thereby improving the overall positioning accuracy; the accurately acquired vehicle posture parameters can provide more refined data support for subsequent vehicle control and navigation, which is crucial to achieving high-precision autonomous driving or automatic parking functions.

[0091] S200: Obtain a reprojection error according to the initial external parameters, feature corner point parameters and vehicle posture parameters, and obtain a posture error according to the vehicle posture parameters.

[0092] Furthermore, the step S200 includes:

[0093] The characteristic corner point parameters, vehicle posture parameters and initial extrinsic parameters are all converted to the starting point coordinate system to obtain detection corner points, detection posture and detection extrinsic parameters.

[0094] In some embodiments, an optimization problem is constructed by first defining optimization parameters:

[0095] Detect corner points: x i0 ,y i0 ,x i1 ,y i1 ,x i2 ,y i2 ,x i3 ,y i3 ...i∈(0,m); where x i0 ,y i0 ,x i1 ,y i1 ,x i2 ,y i2 ,x i3 ,y i3 are the coordinates of the four corner points of the target parking space, and there are m parking spaces in total.

[0096] Detection external reference: Rt wc ; The camera's external parameters have a total of six degrees of freedom, three of which are rotational components and three of which are translational components.

[0097] Detection pose: When the degree of freedom is limited to 3DOF, it can be expressed as:

[0098]

[0099] xi ,y i ,yaw i ,……i∈(0,n)

[0100] Where n is the number of observation frames, x i ,y i ,yaw i They are respectively the x component, y component and heading angle of the detected pose of the i-th observation frame.

[0101] It should be noted that the above parameters are all coordinates relative to the vehicle coordinate system (ie, the starting point coordinate system) corresponding to the parking start time.

[0102] Furthermore, the reprojection error of each detection corner point is calculated according to the detection corner point, detection extrinsic parameter and detection posture corresponding to each frame of detection image; at the same time, the posture error is calculated according to the detection posture of two adjacent frames of detection images.

[0103] In some embodiments, the reprojection error of the corner points is detected:

[0104]

[0105] in, Represents the image observation value of the detected corner point in the i-th observation frame, pt j(0,1,2,3) represents the optimized j-th detected corner point, T i The transformation relationship from the i-th observation frame coordinate system to the starting point coordinate system can be expressed as:

[0106]

[0107] The posture error includes heading angle and position, specifically:

[0108] Heading Angle:

[0109]

[0110] Location:

[0111]

[0112] Among them, obs i ,obs i-1 is the vehicle position information of the i-th and i-1-th observation frames, including the heading angle of the i-th observation frame The x component of the detected pose of the i-th observation frame The y component of the detected pose of the i-th observation frame and the heading angle of the i-1th observation frame The x component of the detected pose of the i-1th observation frame The y component of the detected pose of the i-1th observation frame

[0113] In the above technical solution, the feature corner point parameters, vehicle posture parameters and initial extrinsic parameters are converted to the starting point coordinate system, ensuring that all data are processed in the same coordinate system. This unified coordinate system method avoids data errors caused by different coordinate systems, and helps to more accurately calculate and analyze reprojection errors and posture errors; by calculating the reprojection errors of each detected corner point, the projection accuracy of the feature corner point in the image can be effectively evaluated. This measurement helps to identify and correct errors in the camera model, thereby improving the accuracy of the overall camera calibration; calculating the posture error based on the detection posture of two adjacent frames of detection images can accurately monitor the posture changes of the vehicle during movement, which helps to identify and correct posture deviations caused by sensor drift or model errors, and ensure the stability and reliability of the system in a dynamic environment.

[0114] In addition, by analyzing the reprojection error and pose error, the system can perform targeted optimization to improve the camera extrinsic parameters and vehicle pose estimation. This error-driven optimization method can effectively improve system performance and reduce the error in the final result.

[0115] S300: calculating partial derivatives of the feature corner point parameters and initial extrinsic parameters simultaneously according to the reprojection error to obtain target corner point parameters and target extrinsic parameters, and calculating partial derivatives of the vehicle posture parameters simultaneously according to the reprojection error and posture error to obtain target posture.

[0116] In some embodiments, for the feature corner point parameter pt j(0,1,2,3) The partial derivatives are calculated as follows:

[0117]

[0118] The partial derivatives are calculated for the initial external parameters as follows:

[0119]

[0120] The partial derivatives of the vehicle posture parameters are calculated, including the reprojection error and the posture error, as follows:

[0121] The partial derivative of the reprojection error is calculated as follows:

[0122]

[0123] Among them, x j(0,1,2,3) and j(0,1,2,3) Represents the optimized parking space corner point parameters.

[0124] The partial derivative of the pose error is calculated as follows:

[0125]

[0126] And S400: judging whether the target external parameter meets the preset optimization requirement based on the target posture and the target corner point parameter, so as to update the initial external parameter to the target external parameter if it meets the requirement.

[0127] In some embodiments, after obtaining the result of the joint optimization, it is necessary to perform a preliminary check on the calculated target extrinsic parameters to determine whether they meet the preset optimization requirements, such as position error ±20mm and angle error ±5°; if so, the camera extrinsic parameters are updated to the target extrinsic parameters.

[0128] Furthermore, the step S400 further includes:

[0129] When the target external parameter does not meet the preset optimization requirements, the preset range is increased, the parking perception scene is changed, or an abnormality is reported.

[0130] In some embodiments, the optimization accuracy of the target external parameters can be improved by increasing the range of collecting feature corner point parameters and vehicle posture parameters (i.e., the preset range), which helps to obtain optimization results that better meet the preset optimization requirements.

[0131] In other embodiments, the external parameter optimization may be performed during the next parking process; or the optimization error information may be reported to the user end, which helps to identify potential problems and handle or adjust them in a timely manner.

[0132] In the above technical solution, the system can dynamically adjust the optimization strategy according to the actual situation, improve the data processing capabilities, and can flexibly process under different data conditions to ensure the validity and accuracy of the data; and the flexible processing method not only improves the system stability and processing efficiency, but also reduces the failure rate and improves the user experience.

[0133] Furthermore, after step S400, the method further includes: returning to step S100 based on the target external parameter to continue iterative optimization.

[0134] In the above technical solution, through an iterative optimization process, the camera extrinsic parameters can be adjusted in real time when the vehicle state changes, so that the vehicle system always maintains high-precision environmental perception and vehicle positioning.

[0135] Optionally, the present application may also set optimization start-up requirements. Setting optimization start-up requirements can avoid the problem that the single perception information and parameter coupling lead to large calculation parameter errors and cannot meet the actual application requirements.

[0136] Specifically, after completing step S100:

[0137] It is determined whether the parking space co-viewing information and the vehicle posture parameters meet the optimization start-up requirements. If so, step S200 in the first embodiment is executed; otherwise, after entering another parking perception scene, the process returns to step S100.

[0138] In some embodiments, the optimization activation requirement includes that the vehicle motion state in the current parking perception scene at least includes left turn, right turn, straight driving and stationary, and at least includes preset marked parking spaces.

[0139] Specifically: in the parking perception scene, there are vehicles in a state of sufficient turning, including left turn and right turn; there are vehicles in a state of moving straight in the parking perception scene; there are vehicles in a state of stationary movement in the parking perception scene; there are enough stable marked parking spaces in the parking perception scene.

[0140] Among them, the first three requirements can be obtained based on the vehicle posture parameters, and the last requirement can be obtained based on the parking space common view information.

[0141] In the above technical solution, step S200 will only be executed when the parking space co-view information and the vehicle posture parameters meet the optimization start-up requirements. This conditional triggering mechanism can ensure that the optimization process is only carried out when the data quality is high enough, thereby improving the efficiency and effect of the optimization; if the parking space co-view information or the vehicle posture parameters do not meet the optimization requirements, step S200 is skipped, avoiding unnecessary calculations and processing, saving computing resources, reducing system burden, and improving overall efficiency.

[0142] For further information, see Figure 3 The present application also provides a vehicle-mounted terminal, which is configured with at least a camera and a camera extrinsic parameter optimization system. The camera extrinsic parameter optimization system uses the camera extrinsic parameter optimization method described in the above embodiment to optimize the extrinsic parameters of the camera.

[0143] In the above technical solution, the vehicle-mounted terminal based on the camera extrinsic parameter optimization method can significantly improve the camera calibration accuracy, enhance the performance of the autonomous driving system, optimize data processing efficiency, and reduce system maintenance costs by configuring the camera and the camera extrinsic parameter optimization system; the intelligent optimization process not only improves the user experience, but also improves the reliability and adaptability of the system. The intelligent tuning and scalability of the system further enhance the comprehensive performance of the vehicle-mounted terminal, enabling it to perform well in various application scenarios.

[0144] In summary, the present application provides a camera extrinsic parameter optimization method and a vehicle-mounted terminal; first, feature corner point parameters and vehicle pose parameters are obtained based on the camera installation posture; then, the reprojection error is obtained according to the camera installation posture, feature corner point parameters and vehicle pose parameters, and the pose error is obtained according to the vehicle pose parameters; further, partial derivatives are obtained based on the reprojection error and pose error to obtain the target extrinsic parameter; finally, it is determined whether the target extrinsic parameter meets the preset optimization requirements, and if so, the target extrinsic parameter is used as the camera extrinsic parameter. The present application does not need to rely on specific calibration objects or environments, can optimize camera extrinsic parameters in complex parking scenes, and does not require a large amount of funds to build a calibration field for camera calibration, which reduces costs; and does not require cumbersome operation procedures, and the calculation method is relatively simple.

[0145] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application to this. Those of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present application. All these changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0146] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0148] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The application can also be implemented as a device program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0149] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0150] Although the description of the present application is carried out in conjunction with the above specific embodiments, it is obvious that those skilled in the art can make many substitutions, modifications and changes based on the above content. Therefore, all such substitutions, improvements and changes are included in the spirit and scope of the appended claims.

Claims

1. A camera extrinsic parameter optimization method, characterized in that: The camera installation posture is used as the initial external parameter, and the camera external parameter optimization method includes the following steps: S100: Acquire parking space common view information based on the initial external parameters, generate feature corner point parameters according to the parking space common view information, and simultaneously acquire vehicle posture parameters; S200: Obtaining a reprojection error according to the initial external parameters, characteristic corner point parameters and vehicle posture parameters, and obtaining a posture error according to the vehicle posture parameters; S300: calculating partial derivatives of the feature corner point parameters and the initial extrinsic parameters according to the reprojection error to obtain target corner point parameters and target extrinsic parameters, and calculating partial derivatives of the vehicle posture parameters according to the reprojection error and the posture error to obtain a target posture; and S400: Determine whether the target extrinsic parameter meets preset optimization requirements based on the target pose and target corner point parameters, and update the initial extrinsic parameter to the target extrinsic parameter if the requirements are met.

2. The camera extrinsic parameter optimization method according to claim 1, characterized in that: The step S100 of acquiring the parking space co-viewing information includes: S101: Detecting the marked parking space information and the parking space corner point information within the preset range of the target vehicle based on the initial external parameters to obtain real-time parking space information, and using the real-time parking space information obtained last time as historical parking space information; and S102: Convert the historical parking space information to the vehicle coordinate system corresponding to the real-time parking space information, and match the real-time parking space information with the historical parking space information. If the match is successful, store the real-time parking space information in a first cache corresponding to the historical parking space information; otherwise, create a second cache to store the real-time parking space information in the second cache.

3. The camera extrinsic parameter optimization method according to claim 2, characterized in that: After step S102, the method further includes: determining whether the real-time parking space information detection within a preset range has been completed; if so, acquiring the parking space co-viewing information based on the first cache and the second cache; otherwise, returning to step S101.

4. The camera extrinsic parameter optimization method according to claim 3, characterized in that: The step S100 of generating characteristic corner point parameters includes: The parking space corner point information is matched based on the parking space common view information to obtain the parking space corner point information with a matching number greater than a preset matching threshold as a feature corner point parameter.

5. The camera extrinsic parameter optimization method according to claim 4, characterized in that: Each real-time parking space information corresponds to a frame of detection image, and the step S100 of obtaining vehicle posture parameters includes: Synchronize the wheel speed and angular velocity corresponding to each frame of the detection image, and obtain the wheel information of the target vehicle; and The vehicle posture parameters are obtained according to the synchronized wheel speed and angular velocity and the wheel information.

6. The camera extrinsic parameter optimization method according to claim 5, characterized in that: The step S200 includes: Convert the characteristic corner point parameters, vehicle posture parameters and initial external parameters to the starting point coordinate system to obtain detection corner points, detection posture and detection external parameters; and The reprojection error of each detection corner point is calculated according to the detection corner point, detection external parameter and detection pose corresponding to each frame detection image; At the same time, the posture error is calculated based on the detection postures of two adjacent frames of detection images.

7. The camera extrinsic parameter optimization method according to claim 5, characterized in that: After completing step S100, the method further includes: determining whether the parking space co-viewing information and the vehicle posture parameters meet the optimization start requirement, and if so, executing step S200; otherwise, after entering another parking perception scene, returning to step S100; The optimization activation requirements include that the vehicle motion state in the current parking perception scene at least includes left turn, right turn, straight driving and stationary, and at least includes preset marked parking spaces.

8. The camera extrinsic parameter optimization method according to claim 1, characterized in that: The step S400 further includes: When the target external parameter does not meet the preset optimization requirements, the preset range is increased, the parking perception scene is changed, or an abnormality is reported.

9. The camera extrinsic parameter optimization method according to claim 1, characterized in that: After step S400, the method further includes: returning to step S100 based on the target external parameter to continue iterative optimization.

10. A vehicle-mounted terminal, characterized in that: At least a camera and a camera extrinsic parameter optimization system are configured, and the camera extrinsic parameter optimization system optimizes the extrinsic parameters of the camera using the camera extrinsic parameter optimization method as described in any one of claims 1 to 9.

Images