Vehicle camera self-positioning calibration method and robot

Abstract

The invention provides a vehicle camera self-positioning calibration method and a robot, and belongs to the technical field of intelligent vehicle calibration, and the method comprises the steps that the robot receives a calibration instruction including vehicle three-dimensional structure data, calibration plate optimal position and attitude data corresponding to each to-be-calibrated camera, and calibration pattern data; running an SLAM algorithm based on a registration result of the laser radar point cloud and the vehicle three-dimensional structure data, planning to an optimal path of a current to-be-calibrated camera and moving in place; adjusting the six-degree-of-freedom attitude of the calibration plate to a preset optimal attitude; sending the calibration pattern and the real-time attitude of the calibration plate to the vehicle, so that the vehicle extracts the pattern features and solves the external parameters of the current camera; the real-time posture of the vehicle is sent to the vehicle, and the vehicle compares the camera recognition result to verify the calibration precision. According to the method, dependence on a fixed calibration room or a road calibration site is avoided, automation and closed-loop precision verification of the whole camera calibration process are achieved, and the method is suitable for EOL calibration and after-sales calibration of a vehicle production line.

Description

Technical Field

[0001] This invention relates to the field of vehicle intelligent sensing and automatic calibration technology, and in particular to a vehicle camera self-localization calibration method and robot. Background Technology

[0002] With the increasing popularity of intelligent vehicles and advanced driver assistance systems (ADAS), vehicles are typically equipped with multiple onboard cameras for environmental perception, and rely on the extrinsic accuracy of these cameras to ensure the reliability of core functions such as lane keeping, automatic parking, and obstacle detection.

[0003] Existing camera calibration methods heavily rely on fixed calibration rooms or long straight road scenarios. On the one hand, dedicated calibration rooms, target walls, and host computer systems need to be built in the vehicle production line (EOL), which cannot flexibly arrange workstations and passively occupy production cycles. On the other hand, after-sales calibration relies more on manual operation and road conditions, which is not only costly but also cannot guarantee the consistency of calibration results.

[0004] Furthermore, regardless of whether production line calibration or road calibration is used, the accuracy of the camera's target recognition capability cannot be automatically quantified after calibration. This means that even if the external parameters are solved normally, hidden faults may occur due to cumulative installation errors or deviations in the visual inspection model. Manual retesting is required to confirm whether the camera is truly usable, which has become a bottleneck for the large-scale delivery of ADAS.

[0005] Therefore, we propose a vehicle camera self-localization calibration method and a robot.

[0006] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by providing a vehicle camera self-localization calibration method and robot, thereby solving the technical problems mentioned in the background section.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for self-localization calibration of a vehicle camera includes the following steps:

[0010] S1. The calibration robot receives calibration instructions from the operator. The calibration instructions include the three-dimensional structural data of the vehicle to be calibrated, the number of cameras to be calibrated, the optimal position and attitude data of the calibration board corresponding to each camera, as well as the calibration pattern and test target position data, and starts the camera calibration process accordingly.

[0011] S2. Match the point cloud data and 3D structure data collected by the calibration robot's LiDAR to determine the real-time position of the vehicle to be calibrated. Run the SLAM algorithm to plan the optimal movement path for the robot to reach the current camera to be calibrated, and drive to the endpoint of the optimal movement path so that the calibration board faces the camera to be calibrated.

[0012] S3. Drive the rotating shaft mechanism of the calibration robot to adjust the position and attitude of the calibration plate, so that the real-time position and attitude of the calibration plate gradually approach the optimal position and attitude. The adjustment is completed when the relative error between the real-time position and attitude and the optimal position and attitude is lower than the preset threshold.

[0013] S4. With the calibration plate in a fixed position and orientation, the calibration pattern is displayed on the LCD screen of the calibration plate, and the calibration pattern and the real-time position and orientation of the calibration plate are sent to the vehicle to be calibrated, so that the vehicle to be calibrated can start the corresponding camera to take pictures and calculate the external parameters based on the position and orientation to complete the calibration.

[0014] S5. Drive the calibration robot to the test target position, making the calibration robot the test target of the camera target detection function, and send the real-time position of the calibration robot relative to the vehicle to the vehicle for comparison and verification of calibration accuracy; when the calibration accuracy is not up to standard, keep the current camera number and return to step S2 to re-execute the calibration; when the calibration accuracy is up to standard, increment the camera number and continue to execute step S2 to calibrate the next camera.

[0015] Once all cameras to be calibrated have been calibrated, the S6 will transmit the calibration completion status back to the handheld calibration device or vehicle cockpit system via the communication link to end the calibration process.

[0016] S1 specifically includes:

[0017] Establish a wireless communication link with a handheld calibration device or vehicle cockpit system through a calibration robot; receive a calibration task package containing a calibration start command;

[0018] The calibration task package is used to analyze the three-dimensional structural data of the vehicle to be calibrated, the number of cameras to be calibrated, the optimal position and attitude data of the calibration board for each camera, the calibration pattern data, and the position of the test target.

[0019] The data is written to the controller buffer to provide parameter input for subsequent calibration control algorithm scheduling; the integrity and legality of the received data are checked. If the check fails, a request is made to resend the task package. If the check passes, the calibration process begins.

[0020] S2 specifically includes:

[0021] Continuously collect point cloud data from the robot's LiDAR and perform surface segmentation to identify the vehicle's outline; perform feature matching between the vehicle outline and the 3D structural data to obtain the spatial pose of the vehicle to be calibrated relative to the robot;

[0022] Based on the SLAM modeling results and the optimal position and attitude data of the calibration board corresponding to the current camera, the robot's movement trajectory is planned; the robot chassis walking mechanism is then scheduled to move along the planned trajectory to the end point of the trajectory.

[0023] The robot's shell rotation mechanism rotates the calibration plate to face the camera to be calibrated, completing the spatial alignment before the camera acquires data.

[0024] S3 specifically includes:

[0025] Start the real-time attitude monitoring program and obtain the real-time position and attitude of the calibration board, and calculate the difference vector between the real-time attitude of the calibration board and the stored optimal attitude.

[0026] Generate adjustment control quantities for the shaft drive and drive the shaft mechanism to adjust the calibration plate attitude according to the difference vector direction;

[0027] The process is repeated until all components of the difference vector are less than the set threshold; the rotating shaft mechanism is locked and enters a stable holding state, providing static attitude constraints for calibration pattern acquisition.

[0028] S4 specifically includes:

[0029] After the calibration board's attitude stabilizes, the LCD screen displays the calibration pattern corresponding to the camera; the calibration pattern generation completion mark and the real-time position and attitude of the calibration board are sent to the vehicle controller;

[0030] The vehicle starts the camera to be calibrated to capture the calibration pattern and extract the calibration features; the vehicle calculates the extrinsic parameters of the camera based on the real-time position and attitude of the calibration board.

[0031] The vehicle will feed back its calibrated status to the robot controller through the same communication link.

[0032] S5 specifically includes:

[0033] Plan the robot's movement path according to the test target position of the corresponding camera in the task package; the robot moves to the test target position and adjusts the shell posture to face the current camera to be calibrated;

[0034] The robot sends its real-time position to the vehicle as a visual detection reference standard; the vehicle compares the relative position parameters detected and identified by the camera with the reference standard and feeds back the comparison result to the robot.

[0035] If the comparison deviation is less than the accuracy threshold, switch to the next camera and continue execution; if the deviation is greater than the accuracy threshold, keep the current camera number and return to step S2 for recalibration.

[0036] S6 specifically includes:

[0037] The calibration cycle terminates when the number of completed camera calibrations reaches the total number of cameras to be calibrated; the calibration status, accuracy verification results, and abnormal retest history of each camera are compiled.

[0038] The calibration results are packaged into structured data to generate a calibration summary message; the calibration summary message is sent to a handheld calibration device or vehicle cockpit system via a wireless communication module.

[0039] Enter standby mode or return to the charging station to complete the calibration task.

[0040] A vehicle camera self-localization and calibration robot includes:

[0041] The lidar module is used to collect environmental point clouds and perform 3D contour recognition of the vehicle. Combined with pre-stored 3D structural data of the vehicle, it determines the real-time position and attitude of the vehicle relative to the robot.

[0042] The calibration board assembly has a fixed LCD screen for displaying the calibration pattern corresponding to the camera to be calibrated, and the calibration board assembly has a six-degree-of-freedom adjustable attitude.

[0043] A multi-axis drive mechanism is connected to the calibration plate assembly and, under the control of the controller, adjusts the position and angle of the calibration plate assembly in a coordinated manner to achieve the preset optimal calibration posture.

[0044] A motorized chassis or mobile actuator is used to drive the robot to the calibration position and target detection test position corresponding to the target camera.

[0045] The controller is installed inside the robot body and receives calibration task instructions through a communication link and executes the entire calibration process control.

[0046] The wireless communication module is used to transmit calibration instructions, calibration pattern data, external parameter calculation status, and calibration accuracy verification results between robots, handheld calibration devices, and vehicles.

[0047] The memory is used to store the vehicle's three-dimensional structure data, the optimal attitude parameters of the calibration board corresponding to each camera to be calibrated, the calibration pattern index, and the calibration results.

[0048] The beneficial effects of this invention are as follows:

[0049] This invention replaces fixed targets and visual positioning equipment in calibration rooms with LiDAR environmental perception and SLAM self-localization technology, enabling calibration robots to complete camera calibration in ordinary ground environments (such as production line corridors, parking lots, and repair workshops) without the need for dedicated calibration rooms or road test areas, significantly reducing production line construction and after-sales maintenance costs. A multi-axis drive mechanism performs six-degree-of-freedom closed-loop attitude adjustment on the calibration board, allowing the real-time attitude of the calibration board to converge to the optimal posture and eliminating deviations caused by manual placement. This ensures consistent accuracy and controllable deviation in calibration results under different vehicle, site, and operator conditions.

[0050] This invention makes the robot the visual testing target after the extrinsic parameters are solved. By recognizing the vehicle and comparing it with the robot's real-time position to calculate the root mean square error, it achieves automatic closed-loop confirmation of the camera's target detection accuracy. This avoids the quality risks in traditional solutions where the extrinsic parameters appear correct but the visual recognition performance deteriorates without being detected. By sending the calibration pattern and the real-time position and attitude of the calibration board to the vehicle, the extrinsic parameter calculation is completed at the vehicle end, avoiding secondary calculation errors introduced by external estimation to the robot. This ensures that the calibration results accurately reflect the camera's installation status and imaging model, reducing the risk of false detections and false negatives in the ADAS algorithm due to extrinsic parameter deviations.

[0051] The calibration process of this invention is fully automated, closed-loop, and batch-executed. Multiple robots can be deployed in parallel during the end-of-life (EOL) stage of the production line to improve cycle time, or single-vehicle calibration can be completed in the after-sales workshop or user parking lot, ensuring consistent calibration quality throughout the vehicle's lifecycle, reducing after-sales costs and improving safety. When the accuracy verification fails to meet the standard, the robot automatically returns to the attitude adjustment stage and recalibrates the current camera until the accuracy threshold is reached before switching to the next camera. This eliminates the need for manual retesting and re-judgment of pass / fail, thereby significantly improving calibration efficiency and result reliability. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of a vehicle camera self-localization calibration method according to the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example 1: As Figure 1 As shown, this embodiment provides a vehicle camera self-localization calibration method, including the following steps:

[0055] S1. Calibration instruction receiving step: The calibration robot receives the calibration instructions issued by the operator. The calibration instructions include the three-dimensional structural data of the vehicle to be calibrated, the number of cameras to be calibrated, the optimal position and attitude data of the calibration board corresponding to each camera to be calibrated, as well as the calibration pattern and test target position data, and initiates the camera calibration process accordingly.

[0056] S2. Path planning and movement steps: Based on the point cloud data collected by the calibration robot's LiDAR and the three-dimensional structure data, the real-time position of the vehicle to be calibrated is determined. The SLAM algorithm is run to plan the optimal movement path for the robot to reach the current camera to be calibrated, and the robot travels to the end of the path so that the calibration board faces the camera to be calibrated.

[0057] S3. Attitude adjustment step: Drive the rotating shaft mechanism of the calibration robot to adjust the position and attitude of the calibration plate, so that the real-time position and attitude of the calibration plate gradually approach the optimal position and attitude. When the relative error between the real-time position and attitude and the optimal position and attitude is lower than the preset threshold, the adjustment is completed.

[0058] S4. Calibration data acquisition steps: With the position and attitude of the calibration board fixed, the calibration pattern is displayed on the LCD screen of the calibration board, and the calibration pattern and the real-time position and attitude of the calibration board are sent to the vehicle to be calibrated, so that the vehicle to be calibrated can start the corresponding camera to take pictures and calculate the external parameters based on the position and attitude to complete the calibration.

[0059] S5. Calibration accuracy verification step: Based on the aforementioned SLAM results, drive the calibration robot to move to the test target position, making the calibration robot the test target for the camera target detection function, and send the real-time position of the calibration robot relative to the vehicle to the vehicle for comparison and verification of calibration accuracy; when the calibration accuracy is not up to standard, keep the current camera number and return to step S2 to re-execute the calibration; when the calibration accuracy is up to standard, increment the camera number and continue to execute step S2 to calibrate the next camera;

[0060] S6. Calibration result output steps: After all cameras to be calibrated have been calibrated, the calibration completion status will be transmitted back to the handheld calibration device or vehicle cockpit system via the communication link to end the calibration process.

[0061] S1 specifically includes the following sub-steps:

[0062] S110. Communication Link Establishment and Identity Authentication: The calibration robot establishes a communication link with the handheld calibration device or vehicle cockpit system through a wireless communication peripheral. The communication method can be Wi-Fi, BLE, 5G or any combination thereof. After the link is established, the identity authentication process is triggered, including robot device number matching and dynamic session key negotiation.

[0063] Upon successful authentication, the system status is set to S=1 (communication available); if authentication fails, the status remains S=0 (unavailable) and the calibration process is refused.

[0064] S120, Calibration Command Packet Reception: Under the condition of S=1, the robot receives the calibration command packet and records each field in the command packet into the buffer, including:

[0065]

[0066] Where: i is the camera number, satisfying 1≤i≤N c All coordinates and angles are established in the vehicle coordinate system; after receiving the data, the system status is set to state S=2 (to be verified).

[0067] S130. Calibration Parameter Parsing and Scheduling Table Construction: Parse all fields in S120 and construct the scheduling table in camera number order.

[0068]

[0069] in: It is a calibration schedule (used for subsequent path planning → attitude adjustment → data acquisition → accuracy verification); the schedule is written to the cache and triggers the next stage of data validity verification.

[0070] S140. Integrity and Consistency Verification: Verify the completeness of fields and logical consistency of the instruction packet using the following verification function:

[0071] Φ = α·Φ1 + β·Φ2 + γ·Φ3

[0072] Where: Φ1 = 1 indicates that all fields are not empty; otherwise, Φ1 = 0;

[0073] Φ2=1 represents N c =len({P i})=len({G i})=len({T i}); otherwise Φ2=0;

[0074] Φ3 = 1 indicates that the timestamp and task ID have not expired and are not duplicated; otherwise, Φ3 = 0.

[0075] α, β, and γ are weighting coefficients, which can be taken as α = β = γ = 1;

[0076] The decision logic is as follows: Verification passed; Verification failed;

[0077] If the verification fails, the robot sends the error type (missing field / quantity conflict / parameter expired) through the original link and waits for the instruction packet to be resent before proceeding to the next stage.

[0078] S150. Enter calibration preparation state: When Φ = 3, the robot performs the following actions:

[0079] ①Latch D 3D N c ,

[0080] ② Initialize the SLAM module, path planning module, and attitude control module;

[0081] ③ Perform a health check on the equipment, including the status of the battery level, lidar, drive motor, LCD screen, and communication module;

[0082] ④ When all inspection results are normal, set: S = 3 (calibration preparation complete).

[0083] Then it automatically switches to the S210 path planning and movement step to begin calibration for the i=1th camera.

[0084] S2 specifically includes the following sub-steps:

[0085] S210. Real-time vehicle localization and point cloud filtering: The calibration robot uses LiDAR to collect point cloud data, performs ground filtering, obstacle masking, and vehicle contour extraction, and obtains the point cloud set L at the calibration time. t .

[0086] By using feature matching and the Iterative Closest Point (ICP) algorithm, the point cloud and the vehicle's 3D structure data are compared. 3D Registration, outputting the real-time position and orientation of the vehicle relative to the robot:

[0087] Q t =(X t ,Y t Z t ,Θ t ,Ψ t ,Φ t )

[0088] Q t The vehicle's six-degree-of-freedom pose in the robot coordinate system is determined; if the ICP convergence index is less than the set threshold, the point cloud is re-acquired to avoid misregistration before proceeding to the next step.

[0089] S220. Target pose and path generation: Based on the optimal position and attitude parameters of the calibration board of the i-th camera in the scheduling table:

[0090]

[0091] Calculate the desired target pose that the robot should reach along the path:

[0092]

[0093] Where: f(·) is the transformation function used to map the optimal position and orientation of the calibration board from the vehicle coordinate system to the robot's mobile coordinate system; The robot is positioned to move into a target position; the calculation results are written into the path planning module as the control target.

[0094] S230, Path planning solution: Solving the path planning problem for the robot to move from its current pose to... Optimal solution for the execution cost of the planned path with obstacles:

[0095]

[0096] in:

[0097]

[0098]

[0099] Obtain the optimal path The rear chassis motion module is then deployed.

[0100] S240, Move along the optimal path: The robot moves along... The SLAM module is continuously updated. t And perform path tracking error calculation in real time:

[0101]

[0102] When: E t >ε r Continue moving (ε) r (For path tracking error threshold);

[0103] When: E t ≤ε r When the time comes, stop moving and proceed to the next stage to avoid overshoot oscillation.

[0104] S250, Shell Rotation Orientation and Calibration Plate Visual Alignment: The robot drives the shell rotation mechanism to make the calibration plate normal vector n b With the target camera optical axis vector v i Alignment, satisfying:

[0105] ∠(n b ,v i )≤ε α

[0106] Where: n bIt is the unit vector currently oriented towards the calibration board; v i ε is the unit vector pointing towards the i-th camera; α The visual alignment angle threshold;

[0107] When the above conditions are met, the system state is automatically set to: S=4 (entering the attitude adjustment stage); and control is handed over to the attitude adjustment steps S310–S350.

[0108]

[0109]

[0110] S3 specifically includes the following sub-steps:

[0111] S310, Real-time Attitude Acquisition and Error Vector Construction: Real-time six-DOF attitude of the calibration board is acquired through the SLAM module and calibration board sensors.

[0112] B t =(x t ,y t ,z t ,θ t ,ψ t ,φ t )

[0113] Simultaneously, the optimal pose of the calibration board corresponding to the i-th camera is read from the scheduling table:

[0114]

[0115] Construct the real-time attitude error vector:

[0116]

[0117] Used to measure the difference between the current attitude of the calibration board and the target attitude.

[0118] S320. Error Decomposition and Control Variable Solution: Decompose the error vector into spatial displacement error and angular attitude error.

[0119]

[0120] The control quantity of the rotating shaft is obtained by using a hybrid linear and nonlinear gain:

[0121] U t =K p ·ΔP t +K r ·ΔR t

[0122] in:

[0123] K pThis is the position error control gain matrix;

[0124] K r This is the angle error control gain matrix;

[0125] U t This is the control vector sent to the shaft driver.

[0126] S330, Execute shaft drive and enter closed-loop regulation: Set control quantity U t The input is sent to the shaft drive module for multi-axis linkage adjustment of the mechanical shaft; the SLAM module updates B at a fixed frequency. t This forms a closed-loop regulation.

[0127] ΔB is recalculated after each adjustment cycle. t Then, it enters the next adjustment cycle until the error meets the accuracy requirements.

[0128] S340, Attitude Convergence Determination: Set the allowable attitude error threshold:

[0129] ε b =(ε x ,ε y ,ε z ,ε θ ,ε ψ ,ε φ )

[0130] If and only if all components of the error vector satisfy: |ΔB t |≤ε b ;Right now:

[0131]

[0132] The attitude adjustment is considered complete. If any component exceeds the limit, return to S310 to continue adjustment.

[0133] S350, Attitude Lock and Phase End: When |ΔB t |≤ε b At the time of its establishment:

[0134] ① Lock all adjusting shafts to enter a mechanical steady state;

[0135] ② Record the final posture B t As the actual calibration posture;

[0136] ③ Update system status: S=5 (Attitude adjustment completed → Enter calibration data acquisition stage);

[0137] ④ Transform the locked six-DOF attitude to the target camera coordinate system on the vehicle;

[0138] Control is then transferred to the S410–S450 data acquisition steps.

[0139]

[0140]

[0141] S4 specifically includes the following sub-steps:

[0142] S410. Calibration Pattern Display and Acquisition Link Establishment: The controller drives the calibration board's LCD screen to display the calibration pattern G corresponding to the current camera number i to be calibrated. i Simultaneously, the prepared calibration pattern markers and the real-time attitude B of the calibration board are compared. t Packing to form calibration data frames:

[0143] Ω i ={G i B i ,i}

[0144] Among them B t =(x t ,y t ,z t ,θ t ,ψ t ,φ t ); then Ω will be transmitted via wireless communication link i Sending B to the vehicle controller to provide input for subsequent extrinsic parameter solving. i Let be the orientation of the calibration board during the i-th calibration.

[0145] S420, Camera Imaging and Feature Extraction Trigger: The vehicle controller receives Ω i Then, the i-th camera is activated to take a picture, and the coordinates of the calibration pattern feature points are extracted from the captured image:

[0146] Π i ={(u1,v1),(u2,v2),…,(u M ,v M )}

[0147] Where (u j ,v j ) represents the image pixel coordinates, and M represents the number of effective calibration features extracted.

[0148] If M <M min (Insufficient feature points) The vehicle sends a feature deficiency request to the robot, which will trigger the robot to repeat S410–S420, where M min This is the preset minimum threshold for the number of valid feature points.

[0149] S430, Vehicle Side Camera Extrinsic Parameter Solution: The vehicle controller uses the calibration pattern 3D model point set Γ={(X j ,Y j Z j )} and image feature point set Π i Performing PnP solvers yields extrinsic attitude estimates, where the 3D model point set can be obtained from B. t Obtained from the preset calibration pattern:

[0150] E i =PnP(Γ,Π) i )

[0151] To improve accuracy, reprojection error optimization is employed:

[0152]

[0153] in: For the final external parameter solution; P(·) is the solution given E. i The projection function from 3D points to pixel coordinates.

[0154] Finally, the extrinsic parameter matrix of the camera is recorded:

[0155] S440, External Parameter Verification and Quality Judgment: Average Error of Vehicle Calculation PnP Reprojection:

[0156]

[0157] and the external parameter quality threshold ε thr Comparison: When ε i ≤ε thr The calibration is considered successful when ε i >ε thr If calibration fails, S410–S440 must be repeated; the vehicle controller will compare the calibration status (success / failure) with the current extrinsic parameter matrix T. i Send it to the robot along with the other data.

[0158] S450, Phase End and State Transition: After receiving feedback from the vehicle, the robot performs the following actions: If the vehicle feedback is "calibration failed," the camera number remains unchanged and returns to S410; if the vehicle feedback is "calibration successful," then T... i Write the calibration result to the cache and assign the number: i←i+1; update the system status: S=6 (calibration data acquisition completed → enter the accuracy verification stage); then transfer control to the S510–S550 calibration accuracy verification steps.

[0159] S5 specifically includes the following sub-steps:

[0160] S510. The robot moves to the test target position and establishes the test field of view: based on the test target position T corresponding to the i-th camera in the scheduling table. i (Note: T here) i For "test target position", and the camera extrinsic matrix T i No conflict, the latter will be defined as To avoid ambiguity, the robot performs path planning and movement, a process equivalent to the SLAM path solving mechanism of S210–S250.

[0161] Upon reaching the target location, the robot rotates its outer shell so that the calibration plate or dummy's human body outline faces the target camera, establishing a field of view for target detection testing.

[0162] S520, Real Target Pose Transmission and Perception Baseline Establishment: The robot uses the SLAM module to obtain its real-time pose relative to the vehicle.

[0163] R t =(X t ,Y t Z t ,Θ t ,Ψ t ,Φ t )

[0164] and R t The attitude is transmitted to the vehicle via a communication link, serving as the "true reference pose" for the target detection function.

[0165] S530, Vehicle-side target detection function: The vehicle activates the i-th camera to perform target recognition and outputs the recognized pose estimate.

[0166]

[0167] in The robot target pose identified by the vehicle's visual perception model.

[0168] S540, Calibration Accuracy Calculation and Compliance Judgment: Compare the vehicle's visual recognition output with the robot's reference posture benchmark to calculate the target detection error.

[0169]

[0170] And calculate the root mean square error:

[0171]

[0172] With precision threshold Compare:

[0173] when The calibration result is valid; when The calibration is invalid and a recalibration process needs to be performed; the vehicle controller will then send the above results back to the robot.

[0174] S550, Phase End and Calibration Loop Termination Conditions: The robot executes the following logic based on vehicle feedback:

[0175]

[0176] At this point, the calibration accuracy verification process is complete and seamlessly connected with the logic of the previous and next stages.

[0177]

[0178]

[0179] S6 specifically includes the following sub-steps:

[0180] S610. Construction of calibration result set: The robot traverses all camera numbers i = 1…N c The final extrinsic parameter matrix, accuracy verification index, and recalibration count of each camera are summarized to generate a calibration result set:

[0181]

[0182] in: ε is the final extrinsic parameter matrix for the i-th camera; cam,i C represents the root mean square error calculated for the i-th camera in the accuracy verification step. i This represents the number of recalibrations performed on the i-th camera (used for after-sales diagnostics and quality control tracking).

[0183] S620, Calibration Quality Summary and Operating Condition Self-Diagnosis: Construct a quality diagnosis vector based on SLAM stability indicators, motor adjustment times, movement behavior stability, and abnormal return frequency throughout the calibration period.

[0184] Λ=(λ1,λ2,λ3,λ4)

[0185] Where: λ1 is the average SLAM convergence time index; λ2 is the total number of rotational adjustment rounds; λ3 is the cumulative movement path deviation index; and λ4 is the cumulative number of calibration failure triggers. This index is used to determine whether there is hardware fatigue, sensor offset, or abnormal calibration trend during the calibration process.

[0186] S630, Generation of Structured Calibration Result Data Packets: The robot generates standardized calibration result data packets.

[0187]

[0188] Where: Timestamp is the calibration completion time; vehicle_ID is the unique identifier of the vehicle to be calibrated; task_ID is the identifier of this calibration task; the data adopts structured encoding (JSON, ProtoBuf or vehicle manufacturer-defined format).

[0189] S640, Final Result Transmission and Upload Confirmation: Ω is transmitted via wireless communication link. final Send to the handheld calibration device or vehicle cockpit system, and wait for ACK (acknowledgment code) or NACK (rejection code):

[0190] If an ACK is received, an acknowledgment log is output and the task is terminated; if a NACK is received, the result data packet is retained for manual confirmation or forced retransmission.

[0191] S650, System enters standby or returns to charging station: After uploading the results, the robot executes according to the battery level and standby queue:

[0192] If the battery level is greater than the set threshold E thr It automatically enters standby mode to wait for the next vehicle; if the battery level is ≤E thr It automatically plans a path to return to the charging station;

[0193] Finally, the system status is set to: S=0 (end calibration task); this completes the closed loop of the camera calibration process.

[0194] Example 2: This example provides a vehicle camera self-localization and calibration robot, including:

[0195] The lidar module is used to collect environmental point clouds and perform 3D contour recognition of the vehicle. Combined with pre-stored 3D structural data of the vehicle, it determines the real-time position and attitude of the vehicle relative to the robot.

[0196] The calibration board assembly has a fixed LCD screen for displaying the calibration pattern corresponding to the camera to be calibrated, and the calibration board assembly has a six-degree-of-freedom adjustable attitude.

[0197] A multi-axis drive mechanism is connected to the calibration plate assembly and, under the control of the controller, adjusts the position and angle of the calibration plate assembly in a coordinated manner to achieve the preset optimal calibration posture.

[0198] A motorized chassis or mobile actuator is used to drive the robot to the calibration position and target detection test position corresponding to the target camera.

[0199] The controller, installed inside the robot body, receives calibration task instructions via a communication link and executes the entire calibration process control. The controller is configured as follows:

[0200] a. Perform SLAM calculations based on the point cloud obtained by the LiDAR module and the vehicle's 3D structure data.

[0201] The method involves planning the optimal path to the current camera to be calibrated and controlling the robot's movement.

[0202] b. Based on the optimal position and attitude parameters of the calibration board corresponding to the camera to be calibrated, drive the multi-axis drive mechanism to adjust the calibration board assembly so that its real-time position and attitude are close to the optimal position and attitude;

[0203] c. After the calibration board assembly is in a stable attitude, control the LCD screen to display the calibration pattern, and send the calibration pattern and the real-time position and attitude of the calibration board assembly to the vehicle, so that the vehicle can extract the pattern features and solve the extrinsic parameters of the camera;

[0204] d. After the vehicle completes the external parameter solution, control the robot to move to the test target position corresponding to the camera, and send the robot's real-time position and attitude relative to the vehicle to the vehicle for vehicle target detection function verification;

[0205] e. If the calibration accuracy does not meet the standard, trigger the robot to return to the calibration attitude adjustment process to recalibrate the current camera, and switch to the next camera to be calibrated when the calibration accuracy meets the standard;

[0206] The wireless communication module is used to transmit calibration instructions, calibration pattern data, external parameter calculation status, and calibration accuracy verification results between robots, handheld calibration devices, and vehicles.

[0207] The memory is used to store the vehicle's three-dimensional structure data, the optimal attitude parameters of the calibration board corresponding to each camera to be calibrated, the calibration pattern index, and the calibration results.

[0208] The controller, through the coordinated operation of the lidar module, the mobile actuator and the multi-axis drive mechanism, enables self-positioning calibration and automatic accuracy verification of multiple cameras on the vehicle without the need for a fixed calibration room or road conditions.

[0209] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0210] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0211] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0212] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0213] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0214] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0215] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0216] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0217] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0218] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for self-localization calibration of a vehicle camera, characterized in that, Includes the following steps: S1. The calibration robot receives calibration instructions from the operator. The calibration instructions include the three-dimensional structural data of the vehicle to be calibrated, the number of cameras to be calibrated, the optimal position and attitude data of the calibration board corresponding to each camera, as well as the calibration pattern and test target position data, and starts the camera calibration process accordingly. S2. Match the point cloud data and 3D structure data collected by the calibration robot's LiDAR to determine the real-time position of the vehicle to be calibrated. Run the SLAM algorithm to plan the optimal movement path for the robot to reach the current camera to be calibrated, and drive to the endpoint of the optimal movement path so that the calibration board faces the camera to be calibrated. S3. Drive the rotating shaft mechanism of the calibration robot to adjust the position and attitude of the calibration plate, so that the real-time position and attitude of the calibration plate gradually approach the optimal position and attitude. The adjustment is completed when the relative error between the real-time position and attitude and the optimal position and attitude is lower than the preset threshold. S4. With the calibration plate in a fixed position and orientation, the calibration pattern is displayed on the LCD screen of the calibration plate, and the calibration pattern and the real-time position and orientation of the calibration plate are sent to the vehicle to be calibrated, so that the vehicle to be calibrated can start the corresponding camera to take pictures and calculate the external parameters based on the position and orientation to complete the calibration. S5. Drive the calibration robot to the test target position, making the calibration robot the test target of the camera target detection function, and send the real-time position of the calibration robot relative to the vehicle to the vehicle for comparison and verification of calibration accuracy.

2. The vehicle camera self-localization calibration method according to claim 1, characterized in that, S5 also includes: when the calibration accuracy is not up to standard, keep the current camera number and return to step S2 to re-perform the calibration; when the calibration accuracy is up to standard, increment the camera number and continue to execute step S2 to calibrate the next camera.

3. The vehicle camera self-localization calibration method according to claim 2, characterized in that, Also includes: Once all cameras to be calibrated have been calibrated, the S6 will transmit the calibration completion status back to the handheld calibration device or vehicle cockpit system via the communication link to end the calibration process.

4. The vehicle camera self-positioning calibration method according to claim 3, characterized in that, S1 specifically includes: Establish a wireless communication link with a handheld calibration device or vehicle cockpit system through a calibration robot; receive a calibration task package containing a calibration start command; The calibration task package is used to analyze the three-dimensional structural data of the vehicle to be calibrated, the number of cameras to be calibrated, the optimal position and attitude data of the calibration board for each camera, the calibration pattern data, and the position of the test target. The data is written to the controller buffer to provide parameter input for subsequent calibration control algorithm scheduling; the integrity and legality of the received data are checked. If the check fails, a request is made to resend the task package. If the check passes, the calibration process begins.

5. The vehicle camera self-positioning calibration method according to claim 4, characterized in that, S2 specifically includes: Continuously collect point cloud data from the robot's LiDAR and perform surface segmentation to identify the vehicle's outline; perform feature matching between the vehicle outline and the 3D structural data to obtain the spatial pose of the vehicle to be calibrated relative to the robot; Based on the SLAM modeling results and the optimal position and attitude data of the calibration board corresponding to the current camera, the robot's movement trajectory is planned; the robot chassis walking mechanism is then scheduled to move along the planned trajectory to the end point of the trajectory. The robot's shell rotation mechanism rotates the calibration plate to face the camera to be calibrated, completing the spatial alignment before the camera acquires data.

6. The vehicle camera self-positioning calibration method according to claim 5, characterized in that, S3 specifically includes: Start the real-time attitude monitoring program and obtain the real-time position and attitude of the calibration board, and calculate the difference vector between the real-time attitude of the calibration board and the stored optimal attitude. Generate adjustment control quantities for the shaft drive and drive the shaft mechanism to adjust the calibration plate attitude according to the difference vector direction; The process is repeated until all components of the difference vector are less than the set threshold; the rotating shaft mechanism is locked and enters a stable holding state, providing static attitude constraints for calibration pattern acquisition.

7. The vehicle camera self-localization calibration method according to claim 6, characterized in that, S4 specifically includes: After the calibration board's attitude stabilizes, the LCD screen displays the calibration pattern corresponding to the camera; the calibration pattern generation completion mark and the real-time position and attitude of the calibration board are sent to the vehicle controller; The vehicle starts the camera to be calibrated to capture the calibration pattern and extract the calibration features; the vehicle calculates the extrinsic parameters of the camera based on the real-time position and attitude of the calibration board. The vehicle will feed back its calibrated status to the robot controller through the same communication link.

8. The vehicle camera self-localization calibration method according to claim 7, characterized in that, S5 specifically includes: Plan the robot's movement path according to the test target position of the corresponding camera in the task package; the robot moves to the test target position and adjusts the shell posture to face the current camera to be calibrated; The robot sends its real-time position to the vehicle as a visual detection reference standard; the vehicle compares the relative position parameters detected and identified by the camera with the reference standard and feeds back the comparison result to the robot. If the comparison deviation is less than the accuracy threshold, switch to the next camera and continue execution; if the deviation is greater than the accuracy threshold, keep the current camera number and return to step S2 for recalibration.

9. A vehicle camera self-positioning calibration method according to claim 8, characterized in that, S6 specifically includes: The calibration cycle terminates when the number of completed camera calibrations reaches the total number of cameras to be calibrated; the calibration status, accuracy verification results, and abnormal retest history of each camera are compiled. The calibration results are packaged into structured data to generate a calibration summary message; the calibration summary message is sent to a handheld calibration device or vehicle cockpit system via a wireless communication module. Enter standby mode or return to the charging station to complete the calibration task.

10. A vehicle camera self-localization calibration robot, employing the vehicle camera self-localization calibration method according to any one of claims 1-9, characterized in that, include: The lidar module is used to collect environmental point clouds and perform 3D contour recognition of the vehicle. Combined with pre-stored 3D structural data of the vehicle, it determines the real-time position and attitude of the vehicle relative to the robot. The calibration board assembly has a fixed LCD screen for displaying the calibration pattern corresponding to the camera to be calibrated, and the calibration board assembly has a six-degree-of-freedom adjustable attitude. A multi-axis drive mechanism is connected to the calibration plate assembly and, under the control of the controller, adjusts the position and angle of the calibration plate assembly in a coordinated manner to achieve the preset optimal calibration posture. A motorized chassis or mobile actuator is used to drive the robot to the calibration position and target detection test position corresponding to the target camera. The controller is installed inside the robot body and receives calibration task instructions through a communication link and executes the entire calibration process control. The wireless communication module is used to transmit calibration instructions, calibration pattern data, external parameter calculation status, and calibration accuracy verification results between robots, handheld calibration devices, and vehicles. The memory is used to store the vehicle's three-dimensional structure data, the optimal attitude parameters of the calibration board corresponding to each camera to be calibrated, the calibration pattern index, and the calibration results.

Images