Visual perception subject calibration method and system during advancing of autonomous vehicle

Abstract

The invention discloses a method and a system for calibrating a visual perception subject when an automatic driving vehicle advances. The method comprises the following steps: acquiring an internal reference matrix of a to-be-calibrated visual perception subject; controlling a plurality of to-be-calibrated visual perception main bodies to synchronously collect images of the same checkerboard calibration plate; calculating a homography matrix based on the acquired image, and solving an external parameter of a single visual perception subject to be calibrated by combining an internal parameter matrix; based on a coordinate conversion formula, relative external parameters among the plurality of to-be-calibrated visual perception subjects are calculated through the external parameters of the single visual perception subject; error verification is conducted on the relative external parameters, if the verification error is smaller than a preset threshold value, the relative external parameters are reserved, calibration of the visual perception subject to be calibrated is completed, real-time calibration during vehicle traveling is achieved, dependence of a traditional method on a static state and special equipment is avoided, the calibration efficiency and flexibility are remarkably improved, and the calibration accuracy is improved. And meanwhile, the precision required by L4-level automatic driving is ensured, and a lightweight solution is provided for mass production application.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to a method and system for visual perception subject calibration when an autonomous vehicle is in motion. Background Technology

[0002] In autonomous driving systems, sensor calibration is a key technology for ensuring the accuracy of perception data and the collaborative operation of multiple sensors, with camera calibration being particularly important. The core purpose of camera calibration is to eliminate lens distortion through geometric correction and to determine the camera's intrinsic and extrinsic parameters. These parameters directly affect the accuracy of subsequent visual ranging, target detection, and multi-sensor data fusion. Traditional calibration methods typically rely on calibration boards such as checkerboards or AprilTags, obtaining intrinsic parameters by capturing images from multiple angles and optimizing reprojection errors. Extrinsic calibration, on the other hand, requires aligning camera data with data from other sensors using common-view targets or feature matching algorithms.

[0003] In practical applications, the accuracy of camera calibration directly affects the perception, localization, and decision-making capabilities of autonomous driving. The calibrated camera data can be used for high-precision target detection and semantic segmentation, while accurate extrinsic parameter calibration is the key to realizing vision-LiDAR fusion. However, in existing technologies, the vehicle calibration process must be carried out in a stationary state and cannot be completed in real time while the vehicle is in motion. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for visual perception subject calibration when an autonomous vehicle is in motion, in order to solve the technical defect in the prior art that vehicle calibration must be performed in a stationary state and cannot be completed in real time during the process of movement.

[0005] To achieve the above objectives, the present invention employs the following technical solution: The first aspect of this application provides a method for visual perception subject identification when an autonomous vehicle is in motion, including: Obtain the intrinsic parameter matrix of the visual perception subject to be calibrated; Control multiple visual perception subjects to be calibrated to simultaneously acquire images of the same chessboard calibration board; The homography matrix is ​​calculated based on the acquired images, and the extrinsic parameters of a single visual perception subject to be calibrated are solved by combining the intrinsic parameter matrix. Based on the coordinate transformation formula, the relative extrinsic parameters between multiple visual perception subjects to be calibrated are obtained by calculating the extrinsic parameters of a single visual perception subject. Error verification is performed on the relative extrinsic parameters. If the verification error is less than the preset threshold, the relative extrinsic parameters are retained and the calibration of the visual perception subject to be calibrated is completed.

[0006] In one optional embodiment, controlling multiple visual perception subjects to be calibrated to simultaneously acquire images of the same chessboard calibration board specifically includes: A checkerboard calibration board is set up at a fixed spatial position directly in front of the autonomous vehicle; multiple visual perception subjects to be calibrated are controlled to simultaneously acquire images of the checkerboard calibration board as the autonomous vehicle moves.

[0007] In an optional embodiment, after synchronously acquiring images of the checkerboard calibration board, the method further includes: Quality checks are performed on the synchronously acquired images to ensure that the confidence level of corner detection is not less than 95% and the degree of motion blur is not greater than 2 pixels.

[0008] In one optional embodiment, a homography matrix is ​​calculated based on the acquired images, and the extrinsic parameters of a single visual perception subject to be calibrated are solved in conjunction with the intrinsic parameter matrix, specifically including: For each checkerboard image corresponding to the visual perception subject to be calibrated, calculate each homography matrix separately; Based on the calculated homography matrices, the extrinsic parameters of a single visual perception subject to be calibrated are obtained by combining the intrinsic parameter matrix. Among them, the external parameters are the position and attitude parameters of the visual perception subject relative to the vehicle coordinate system.

[0009] In one optional embodiment, obtaining the intrinsic parameter matrix of the visual perception subject to be calibrated specifically includes: Obtain the intrinsic parameter matrix of the visual perception subject to be calibrated after it leaves the factory or is pre-calibrated, and use it as the intrinsic parameter matrix; The effectiveness of the intrinsic parameter matrix is ​​detected by the reprojection error method.

[0010] In one alternative embodiment, the visual perception subject to be calibrated is a camera.

[0011] A second aspect of this application provides a visual perception subject identification system for autonomous vehicles in motion, comprising: The acquisition module is used to acquire the intrinsic parameter matrix of the visual perception subject to be calibrated; The acquisition module is used to control multiple visual perception subjects to be calibrated to simultaneously acquire images of the same chessboard calibration board; The solution module is used to calculate the homography matrix based on the acquired images and solve for the extrinsic parameters of a single visual perception subject to be calibrated by combining the intrinsic parameter matrix. The calculation module is used to calculate the relative extrinsic parameters between multiple visual perception subjects to be calibrated based on the coordinate transformation formula and the extrinsic parameters of a single visual perception subject. The calibration module is used to verify the error of the relative extrinsic parameters. If the verification error is less than the preset threshold, the relative extrinsic parameters are retained and the calibration of the visual perception subject to be calibrated is completed.

[0012] A third aspect of this application provides a mobile device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a visual perception subject calibration method for autonomous vehicles in motion.

[0013] A fourth aspect of this application provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of a visual perception subject calibration method for autonomous vehicles in motion.

[0014] A fifth aspect of this application provides a computer program product, including computer instructions that instruct a computing device to perform operations corresponding to a visual perception subject calibration method when an autonomous vehicle is in motion.

[0015] Compared with the prior art, the present invention has the following beneficial effects: By acquiring the intrinsic parameter matrix of the visual perception subject to be calibrated, the complex process of repeatedly calibrating intrinsic parameters during the vehicle's movement is avoided, thus simplifying the calibration steps. Controlling multiple visual perception subjects to be calibrated to simultaneously acquire images of the same checkerboard calibration board allows the vehicle to acquire spatiotemporally aligned calibration data even while in motion, thereby supporting dynamic calibration. Calculating the homography matrix based on the acquired images and combining it with the intrinsic parameter matrix to solve for the extrinsic parameters of a single visual perception subject enables rapid pose calculation from a single image, significantly improving processing speed. Calculating the relative extrinsic parameters between multiple visual perception subjects using coordinate transformation formulas ensures the accuracy of multi-sensor collaborative work. Error verification of the relative extrinsic parameters and retaining the results when the verification error is less than a preset threshold ensures the reliability of the calibration results. Finally, high-precision calibration is completed during vehicle movement. This not only solves the limitation that the calibration process must be performed in a stationary state but also replaces hardware dependence with algorithm-driven methods. While ensuring the perception accuracy requirements of L4 autonomous driving, it significantly improves calibration efficiency, reduces equipment costs, and achieves a technological breakthrough from offline to online calibration. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a visual perception subject calibration method for an autonomous vehicle during driving, provided by the present invention; Figure 2This invention provides a schematic diagram of a visual perception subject calibration system for autonomous vehicles during operation. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0019] To address the technical deficiencies mentioned in the background section, this embodiment provides a method and system for visual perception subject calibration during the movement of an autonomous vehicle. The invention will be further described in detail below with reference to the accompanying drawings: In a first aspect, embodiments of the present invention provide a method for visual perception subject calibration when an autonomous vehicle is in motion, such as... Figure 1 As shown, it includes: S101. Obtain the intrinsic parameter matrix of the visual perception subject to be calibrated.

[0020] For example, the inherent parameter matrix of the visual perception subject to be calibrated after factory manufacturing or pre-calibration is obtained and used as the intrinsic parameter matrix. Then, the validity of the intrinsic parameter matrix is ​​detected by the reprojection error method.

[0021] Among them, the visual perception subject to be calibrated refers to the optical imaging device installed on the autonomous vehicle for collecting images of the environment in front, typically an industrial-grade global shutter camera, whose lens has a fixed focal length, fixed aperture and known optical distortion characteristics.

[0022] The intrinsic parameter matrix is ​​a triangular matrix that represents the internal geometric properties of the camera imaging model. It does not change with the camera's spatial pose. The intrinsic parameter matrix can be obtained through two parallel implementation paths: the inherent parameter matrix after factory manufacturing or pre-calibration, and the initial intrinsic parameter calibration result obtained using high-precision optical calibration equipment in a standard laboratory environment after the visual sensing subject (camera) has been manufactured and assembled. This intrinsic parameter matrix is ​​stored in the device firmware or non-volatile memory (such as EEPROM or Flash) in binary or text format. It has been written and verified by the OEM before the vehicle leaves the factory. Its physical meaning is to represent the inherent optical properties of the visual sensing subject under the ideal imaging geometric model, which does not change with the installation posture.

[0023] This inherent parameter matrix can be directly read and called, eliminating the need to repeatedly perform multi-angle checkerboard acquisition and nonlinear optimization processes on-site.

[0024] As an optional embodiment, a production line batch pre-calibration method can also be adopted, which involves sampling and calibrating camera modules of the same model before assembly, establishing a batch-internal parameter mapping table, and automatically loading the average internal parameter matrix of the corresponding batch based on the serial number (SN) when the vehicle is put on the production line. At the same time, a temperature-compensated internal parameter template can also be used to calibrate and store three sets of internal parameter matrices at different temperature points. During operation, the lens housing temperature is read in real time by the vehicle temperature sensor, and the optimal internal parameter matrix under the current working condition is obtained by interpolation. The reprojection error method refers to selecting an image containing a standard chessboard calibration board under the premise of known initial values ​​of extrinsic parameters, extracting the pixel coordinates of the detected corner points in the image, mapping the known 3D points in the corresponding world coordinate system onto the image plane through the complete projection model composed of the current intrinsic parameter matrix and the current extrinsic parameters, obtaining the reprojection coordinates, calculating the Euclidean distance error of each corner point, and statistically analyzing the root mean square error of all effective corner points, which is the reprojection error value.

[0025] The detection process is triggered when the vehicle is stationary or moving at low speed. Image acquisition is completed synchronously by multiple visual perception subjects under the unified scheduling of the vehicle domain controller. The error threshold is set to δ = 0.8 pixels. When RMSE ≤ δ, the intrinsic parameter matrix is ​​considered valid. If RMSE > δ, an alarm mechanism is activated: an intrinsic parameter drift warning event code is sent to the vehicle diagnostic system, and the current ambient temperature, vibration acceleration, image signal-to-noise ratio, and corner detection confidence are recorded for subsequent root cause analysis.

[0026] As an optional embodiment, a weighted projection error can also be used to give higher weight to corner points in the edge region in order to enhance the robustness of detection in lens distortion sensitive areas; a time sliding window mechanism can also be introduced to continuously acquire M=5 frames of images, calculate RMSE respectively, and take the median as the final criterion to suppress abnormal noise interference in single frames.

[0027] Through the above steps, without recalibrating the intrinsic parameters, the intrinsic parameter matrix used is ensured to continuously meet the perception accuracy requirements of Level 4 autonomous driving, thereby ensuring the credibility of the extrinsic parameter calibration results and the functional safety of the system. The overall solution takes into account both deployment efficiency and operational reliability, and is applicable to various scenarios such as final inspection of mass-produced autonomous vehicles, after-sales quick repair, and in-operation self-inspection.

[0028] S102. Control multiple visual perception subjects to be calibrated to simultaneously acquire images of the same checkerboard calibration board; for example, a checkerboard calibration board with a fixed spatial position is set up 2m in front of the autonomous vehicle, and then multiple visual perception subjects to be calibrated are controlled to simultaneously acquire images of the checkerboard calibration board during the movement of the autonomous vehicle, and finally the quality of the synchronously acquired images is checked to ensure that the confidence level of corner detection of the image is not less than 95% and the degree of motion blur is not greater than 2 pixels.

[0029] The checkerboard calibration plate, which is set up at a fixed spatial position 2m in front of the autonomous vehicle, refers to the rigid installation of a standard checkerboard calibration plate in front of the autonomous vehicle, so that its physical position remains constant and can be accurately calibrated in the global coordinate system. The plane normal of the calibration plate is approximately perpendicular to the initial driving direction of the vehicle, the center point is located in the longitudinal symmetry plane of the vehicle, and the height is adapted to the field of view coverage of the on-board visual perception subject. Its spatial position is determined by a total station, laser tracker or high-precision RTK-GNSS system and recorded as known three-dimensional coordinates and attitude angles, which serve as a unified world reference benchmark for subsequent external parameter solutions.

[0030] This deployment method does not rely on a stationary vehicle or a movable mechanical platform, and can be reused for a long time after a single installation; optionally, the calibration plate uses an acrylic substrate with a high-contrast black and white checkered pattern screen-printed.

[0031] In alternative solutions, the calibration board can also be replaced with a metal etched board with an equidistant grid structure, an LED backlit active light-emitting checkerboard pattern (to improve the signal-to-noise ratio in low-light environments), or an embedded UWB positioning tag to provide real-time feedback on its micro-vibration displacement for dynamic compensation of calibration errors. Controlling multiple visual perception subjects to be calibrated and simultaneously acquiring images of the checkerboard calibration board during the movement of an autonomous vehicle refers to using hardware trigger signals (such as GPIO level pulses) or software timestamp alignment mechanisms (based on PTP precision time protocol or GPS pulse-per-second) to ensure that all cameras to be calibrated complete exposure and image readout within the same millisecond time window during the period when the vehicle is traveling at a constant speed of 0.5–15 km / h. Each camera is uniformly scheduled through the onboard domain controller, and the exposure time is set to 1 / 1000 s to 1 / 200 s to suppress motion blur. The synchronous acquisition process lasts for no less than 3 seconds to ensure that at least 5 valid image sequences are acquired. The acquired images need to cover multi-view observations of the calibration board under different lateral / longitudinal offsets, but all images correspond to the same calibration board entity in the same fixed world coordinate system. This synchronization mechanism is compatible with onboard high-speed image transmission interfaces such as MIPICSI-2, GMSL, or FPD-Link III, and supports broadcasting synchronization commands via the CAN FD bus.

[0032] Optionally, data acquisition can be initiated when the vehicle acceleration is less than 0.1 g and the yaw rate is less than 0.5° / s to ensure that the geometric deformation of the image is controllable.

[0033] In alternative solutions, the synchronous method can also adopt an asynchronous triggering mode based on event cameras. When the brightness change of the edge pixels of the calibration board exceeds the threshold, each camera triggers acquisition independently, and then the data is aligned to a unified time reference through event timestamp interpolation.

[0034] Through the above-described steps, this application achieves the technical effect of joint calibration of extrinsic parameters of multiple visual perception subjects under dynamic vehicle driving conditions. Because the calibration board has a fixed spatial position and can be accurately traced, it solves the problems of low efficiency and high deployment threshold caused by relying on a dedicated movable calibration platform and requiring the vehicle to be parked in the background technology. Because the synchronous acquisition mechanism ensures the temporal consistency and spatial observability of multiple images, it solves the problem of offline calibration being unable to respond to parameter drift during operation in the background technology. Because the driving process simulates real working conditions, the obtained extrinsic parameters have higher environmental adaptability, significantly improving the generalization ability and robustness of calibration parameters in actual autonomous driving tasks.

[0035] Furthermore, a corner detection confidence level of no less than 95% means that after locating corners in the checkerboard image using the Shi-Tomasi corner response function or an improved FAST-ORB feature extractor, a reliability score is output for each detected corner. This score comprehensively considers four indicators: local gray-level gradient consistency, sub-pixel interpolation convergence stability, neighborhood texture richness, and multi-scale response consistency. After normalization, a confidence value in the range of 0-100% is obtained. The confidence threshold is set at 95%, meaning that the entire image is considered to have acceptable corner quality only when the average confidence level of all valid corners is ≥95% and the lowest confidence level of a single point is ≥90%. This threshold can be dynamically adjusted according to different lighting conditions. For example, it can be lowered to 92% in low-light (<50 lux) environments, while it is increased to 96% in strong backlight scenes to balance robustness and accuracy. Motion blur is no greater than 2 pixels, meaning the full width at half maximum (FWHM) of the point spread function (PSF) along the direction of motion at the corner points of the checkerboard pattern in the image is ≤2 pixels. This achieves automated, quantitative, and closed-loop control of the quality of acquired images under dynamic vehicle conditions. Because it adopts a dual-index joint criterion with configurable thresholds, it solves the problems of "images being easily affected by motion blur during vehicle movement and unstable corner point recognition leading to divergence in external parameter calculations". It significantly improves the adaptability of the calibration process to uncertainties such as real road vibration, sudden changes in lighting, and calibration board occlusion, so that the calibration results maintain long-term consistency and reproducibility throughout the entire life cycle of L4 autonomous driving mass-produced vehicles. It supports efficient calibration operations with a production line cycle time of ≤90 seconds / vehicle and provides a high-quality initial data foundation for subsequent online self-calibration.

[0036] S103. Calculate the homography matrix based on the acquired images, and solve for the extrinsic parameters of a single visual perception subject to be calibrated by combining the intrinsic parameter matrix; for example, the homography matrix H is calculated based on the acquired images using the following formula:

[0037] In the formula and For pixel coordinates, and As a world coordinate system, For rotation matrix, It is a translation vector.

[0038] Finally, the extrinsic parameters of a single visual perception subject to be calibrated are solved by combining the intrinsic parameter matrix K.

[0039] In specific operation, for each checkerboard image corresponding to the visual perception subject to be calibrated, each homography matrix H is calculated. Then, based on the calculated homography matrix H, the extrinsic parameters of a single visual perception subject to be calibrated are obtained by combining the intrinsic parameter matrix K. The extrinsic parameters are the position and attitude parameters of the visual perception subject relative to the vehicle coordinate system.

[0040] In this embodiment, each visual perception subject to be calibrated refers to multiple cameras installed at different locations on the autonomous vehicle, and their physical installation locations do not overlap; the corresponding checkerboard image refers to a single-frame grayscale image of the checkerboard calibration board at the same fixed spatial location, independently acquired by each camera under synchronous acquisition conditions.

[0041] In the above steps, under the premise of fixed intrinsic parameters, using a single synchronously acquired checkerboard image, the complete extrinsic parameters of each camera relative to a unified vehicle coordinate system are stably solved through homography matrix analytical decomposition and rotation matrix orthogonality constraint.

[0042] S104. Based on the coordinate transformation formula, the relative extrinsic parameters between multiple visual perception subjects to be calibrated are calculated using the extrinsic parameters of a single visual perception subject; for example, the coordinate transformation formula is as follows:

[0043] In the formula Let be the rotation matrix between camera A and camera B. Let A be the rotation matrix from camera A to the origin of the calibration board. Let be the rotation matrix from camera B to the origin of the calibration board.

[0044] S105. Perform error verification on the relative extrinsic parameters. If the verification error is less than a preset threshold, retain the relative extrinsic parameters and complete the calibration of the visual perception subject to be calibrated. For example, perform reprojection verification on the calculated relative extrinsic parameter error. If the error is less than 0.5 pixels, it is considered accurate. Save the parameters and complete the calibration of the visual perception subject to be calibrated. If the verification fails, repeat the above steps for recalibration.

[0045] In this embodiment, the designated visual perception subject refers to an imaging device deployed on an autonomous vehicle for acquiring environmental images and participating in multi-sensor fusion perception tasks. Its physical form is an independent imaging module equipped with an optical lens, an image sensor (such as CMOS or CCD), an image signal processing unit, and a standard video output interface.

[0046] Specifically, a camera can be used.

[0047] Secondly, embodiments of the present invention propose a visual perception subject calibration system for autonomous vehicles in motion, such as... Figure 2 As shown, it includes: The acquisition module is used to acquire the intrinsic parameter matrix of the visual perception subject to be calibrated; The acquisition module is used to control multiple visual perception subjects to be calibrated to simultaneously acquire images of the same chessboard calibration board; The solution module is used to calculate the homography matrix based on the acquired images and solve for the extrinsic parameters of a single visual perception subject to be calibrated by combining the intrinsic parameter matrix. The calculation module is used to calculate the relative extrinsic parameters between multiple visual perception subjects to be calibrated based on the coordinate transformation formula and the extrinsic parameters of a single visual perception subject. The calibration module is used to verify the error of the relative extrinsic parameters. If the verification error is less than the preset threshold, the relative extrinsic parameters are retained and the calibration of the visual perception subject to be calibrated is completed.

[0048] The acquisition module is used to obtain the intrinsic parameter matrix of the visual perception subject to be calibrated. This module obtains the intrinsic parameter matrix K obtained from factory calibration by reading the pre-stored calibration file, calling the firmware interface, or accessing non-volatile memory. This module also integrates a reprojection error detection subunit, which is used to load a set of known 3D world points and their corresponding image pixels, substitute them into K and the default extrinsic parameters to calculate the reprojection position, and count whether the average reprojection error is less than 0.5 pixels. If it exceeds the limit, an alarm is triggered and the user is prompted to re-import the valid intrinsic parameters.

[0049] As an optional implementation, the acquisition module also supports receiving incremental intrinsic parameter update packages from the cloud calibration service platform through the on-board diagnostic interface (OBD-II) and completing version verification and secure writing; or, when there is no pre-calibration data, temporarily calling a lightweight single-frame intrinsic parameter estimation algorithm (such as a self-calibration method based on linear orthogonality constraints) to generate an initial K for subsequent extrinsic parameter solving.

[0050] The acquisition module is used to control multiple visual sensing subjects to be calibrated to simultaneously acquire images of the same checkerboard calibration board. This module coordinates the exposure time of multiple visual sensing subjects through hard synchronization signals (such as GPIO trigger pulses) or soft synchronization protocols (such as PTP precise time protocol) to ensure that their acquisition time deviation does not exceed 10ms. The synchronization trigger source can come from vehicle speed sensor pulses, IMU zero-speed detection events, or be uniformly scheduled by the system timer.

[0051] The solution module is used to calculate the homography matrix based on the acquired images and solve the extrinsic parameters of a single visual perception subject to be calibrated by combining the intrinsic parameter matrix. This module independently performs sub-pixel corner localization for each image and constructs a set of matching points of no less than 12 pairs of world coordinates and image coordinates.

[0052] With the vehicle maintaining a low speed (0–15 km / h), relying solely on a forward-fixed checkerboard calibration board, the acquisition module loads reliable intrinsic parameters, the acquisition module triggers multi-camera microsecond-level synchronous imaging, the solution module obtains the absolute extrinsic parameters of each camera based on homography analysis, the calculation module derives the relative pose topology of the entire system based on these parameters, and the calibration module verifies the reliability of the results through dual criteria. Since there is no need to move the vehicle or calibration platform, the problems of strong equipment dependence and inflexible deployment in the background technology are solved.

[0053] Thirdly, embodiments of the present invention provide a mobile device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described method for visual perception subject calibration of an autonomous vehicle in motion.

[0054] This embodiment provides a mobile device capable of carrying and executing the aforementioned visual perception subject calibration method. Its technical means lies in encapsulating all the algorithm steps defined above into a software module that can be deployed on general-purpose computing hardware. Through the coordinated operation of memory and processor, the calibration process is localized, lightweighted, and portable.

[0055] This solution does not rely on cloud servers or dedicated calibration fixtures. It only requires a mobile computing platform with basic image acquisition interfaces (such as USB, MIPI), computing power (such as ARM Cortex-A76 or higher cores, or equivalent AI acceleration units), and storage space (≥2GB RAM, ≥8GB Flash) to complete the complete calibration loop. It supports deployment in various forms such as vehicle-mounted edge computing units, handheld calibration terminals, and maintenance tablets. It avoids the transmission delay and privacy leakage risks caused by data uploads. It meets the needs of multiple scenarios such as rapid production line deployment, on-site real-time recalibration, and periodic calibration during operation. It significantly reduces the dependence of calibration on professional sites and high-precision mechanical devices.

[0056] Among them, mobile devices are portable or embedded computing platforms with image processing and real-time computing capabilities. Their physical forms include, but are not limited to, vehicle domain controllers (such as NVIDIA DRIVE Orin and Huawei MDC 610), industrial-grade tablet computers (running Linux / Android systems), ruggedized handheld terminals (integrating camera modules and GPU acceleration modules), or customized edge computing modules based on RISC-V / ARM architecture.

[0057] The memory is used to persistently store calibration programs, initial values ​​of intrinsic parameters, checkerboard template parameters, intermediate variables for homography matrix solving, extrinsic parameter optimization iteration logs, and final relative extrinsic parameter results; its types include non-volatile storage media (such as eMMC, UFS, SPI NOR Flash) and volatile running memory (such as LPDDR4X).

[0058] In one embodiment, the memory is divided into an independent security zone and an application zone, and key parameters (such as intrinsic parameter matrices and preset thresholds) are stored in an encrypted partition protected by TrustZone or Secure Enclave to prevent unauthorized tampering.

[0059] The processor is used to load and execute computer programs in memory, and to complete all operations such as image acquisition scheduling, corner detection, homography matrix solving, extrinsic parameter analytical decomposition, coordinate transformation calculation and error verification; its computing resources are dynamically allocated as needed.

[0060] The computer program is structured executable code, including an image input interface module, a calibration logic engine module, an extrinsic parameter output interface module, and an exception handling module. The calibration logic engine module strictly follows the steps and constraints defined above, namely, first loading the intrinsic parameter matrix, then triggering synchronous image acquisition, then calculating the homography matrix frame by frame based on the checkerboard image, combining the intrinsic parameters to solve the extrinsic parameters of each camera, then deriving the relative extrinsic parameters between multiple cameras, and finally using whether the reprojection error is lower than a preset threshold as the criterion for successful calibration.

[0061] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for visual perception subject calibration of an autonomous vehicle in motion. Computer-readable storage media refers to non-transitory physical storage structures that can be recognized by a processor and from which instruction data can be read. These include, but are not limited to, solid-state drives (SSDs), embedded multimedia cards (eMMCs), universal flash storage (UFS), read-only memory (ROMs), electrically erasable programmable read-only memory (EEPROMs), electrically erasable programmable read-only memory (Flash Memory), secure digital cards (SD Cards), USB flash drives, and optical discs (CD-ROMs, DVD-ROMs).

[0062] The physical form of this medium can be onboard (such as SPI NOR Flash soldered to the PCB of the vehicle domain controller), modular (such as M.2 interface NVMe SSD), pluggable (such as TF card slots that support hot-swapping in the vehicle system), or distributed (such as remote storage nodes connected via the vehicle Ethernet, whose access protocols comply with ISO 14229-5 or AUTOSAR XCPover Ethernet specifications).

[0063] In one optional embodiment, the computer-readable storage medium uses an AEC-Q100 Grade 2 certified industrial-grade eMMC chip with an operating temperature range of [temperature range missing]. It can withstand temperatures from 40℃ to 105℃, supports 100,000 erase / write cycles and has a power-off protection mechanism to adapt to harsh environments such as vehicle vibration, high and low temperatures and electromagnetic interference.

[0064] In another optional embodiment, the medium is an encrypted USB flash drive conforming to the ISO / IEC 19770-2 software asset standard, with an embedded SM4 national cryptographic algorithm hardware encryption and decryption module, used to authorize the loading and execution permissions of the calibration program.

[0065] A computer program is a software entity consisting of a series of machine-executable instructions. It is written in a high-level language (such as C++17 or Python 3.9) or an intermediate representation (such as LLVM IR) and is converted into an executable format for the target platform (such as an ARM64 ELF binary file, an AUTOSAR Adaptive Platform compatible ARA::com service description file, or a shared library that can be loaded by a ROS2 node) through cross-compilation or an interpreter.

[0066] Fifthly, embodiments of the present invention provide a computer program product, including computer instructions, which instruct a computing device to perform operations corresponding to the above-described method for calibrating a visual perception subject during the movement of an autonomous vehicle.

[0067] This program product does not rely on a specific hardware platform or operating system and can run in vehicle domain controllers, edge computing units, or cloud diagnostic servers. It receives calibration trigger signals (such as vehicle start-up events, mileage thresholds, temperature drift alarms, or remote OTA commands) through standardized interfaces, and automatically calls modular subroutines such as image acquisition, homography matrix solving, external parameter analytical calculation, relative coordinate transformation, and error verification to complete the end-to-end calibration process closed loop.

[0068] A computer program product refers to a software entity that exists in digital form, possesses independent intellectual property rights, and has the capability to be distributed. Its physical carrier includes, but is not limited to, solid-state drives (SSDs), embedded multimedia cards (eMMC), USB flash drives, or digitally signed packages distributed via wireless communication networks (such as 5G NR and C-V2X). This program product contains executable code, configuration files, calibration template data (such as checkerboard physical size parameters and default value tables), and a logging module. All components are compiled and optimized and support signature verification mechanisms to ensure runtime integrity and tamper-proof performance. As an alternative embodiment, this program product can also be packaged in a containerized form (such as a Docker image), with a built-in lightweight Python / Cpp runtime environment and necessary dependency libraries such as OpenCV and Eigen, thereby achieving "build once, deploy anywhere" on heterogeneous computing platforms.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading the present invention, they can still make various changes, modifications or equivalent substitutions to the specific implementation of the invention, but these changes, modifications or equivalent substitutions are all within the scope of protection of the pending claims of the invention.

Claims

1. A method for visual perception subject identification during the movement of an autonomous vehicle, characterized in that, include: Obtain the intrinsic parameter matrix of the visual perception subject to be calibrated; Control multiple visual perception subjects to be calibrated to simultaneously acquire images of the same chessboard calibration board; The homography matrix is ​​calculated based on the acquired images, and the extrinsic parameters of a single visual perception subject to be calibrated are solved by combining the intrinsic parameter matrix. Based on the coordinate transformation formula, the relative extrinsic parameters between multiple visual perception subjects to be calibrated are obtained by calculating the extrinsic parameters of a single visual perception subject. The relative extrinsic parameters are verified for error. If the verification error is less than a preset threshold, the relative extrinsic parameters are retained and the calibration of the visual perception subject to be calibrated is completed.

2. The method for visual perception subject identification during the movement of an autonomous vehicle according to claim 1, characterized in that, Controlling multiple visual perception subjects to be calibrated to simultaneously acquire images of the same chessboard calibration board, specifically including: A checkerboard calibration board with a fixed spatial position is set up directly in front of the autonomous vehicle; Multiple visual perception subjects to be calibrated are controlled to simultaneously acquire images of the chessboard calibration board during the movement of the autonomous vehicle.

3. The method for visual perception subject calibration during the movement of an autonomous vehicle according to claim 2, characterized in that, After synchronously acquiring images of the chessboard calibration board, the process also includes: Quality checks are performed on the synchronously acquired images to ensure that the confidence level of corner detection is not less than 95% and the degree of motion blur is not greater than 2 pixels.

4. The method for visual perception subject identification during the movement of an autonomous vehicle according to claim 1, characterized in that, The process of calculating the homography matrix based on the acquired images and solving for the extrinsic parameters of a single visual perception subject to be calibrated, in conjunction with the intrinsic parameter matrix, specifically includes: For each checkerboard image corresponding to the visual perception subject to be calibrated, calculate each homography matrix separately; Based on the calculated homography matrices, the extrinsic parameters of a single visual perception subject to be calibrated are obtained by combining the intrinsic parameter matrix. The extrinsic parameters are the position and attitude parameters of the visual perception subject relative to the vehicle coordinate system.

5. The method for visual perception subject calibration of an autonomous vehicle during travel according to claim 1, characterized in that, The acquisition of the intrinsic parameter matrix of the visual perception subject to be calibrated specifically includes: Obtain the intrinsic parameter matrix of the visual perception subject to be calibrated after it leaves the factory or is pre-calibrated, and use it as the intrinsic parameter matrix; The effectiveness of the intrinsic parameter matrix is ​​detected by the reprojection error method.

6. The method for visual perception subject identification during the movement of an autonomous vehicle according to claim 1, characterized in that, The visual perception subject to be calibrated is the camera.

7. A visual perception subject calibration system for autonomous vehicles during operation, characterized in that, include: The acquisition module is used to acquire the intrinsic parameter matrix of the visual perception subject to be calibrated; The acquisition module is used to control multiple visual perception subjects to be calibrated to simultaneously acquire images of the same chessboard calibration board; The solution module is used to calculate the homography matrix based on the acquired image and solve the extrinsic parameters of a single visual perception subject to be calibrated in combination with the intrinsic parameter matrix; The calculation module is used to calculate the relative extrinsic parameters between multiple visual perception subjects to be calibrated based on the coordinate transformation formula and the extrinsic parameters of a single visual perception subject. The calibration module is used to verify the error of the relative extrinsic parameters. If the verification error is less than a preset threshold, the relative extrinsic parameters are retained and the calibration of the visual perception subject to be calibrated is completed.

8. A mobile device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the visual perception subject calibration method for an autonomous vehicle in motion as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the visual perception subject calibration method for an autonomous vehicle in motion as described in any one of claims 1-6.

10. A computer program product comprising computer instructions, characterized in that, The computer instructions instruct the computing device to perform the operation corresponding to the visual perception subject calibration method for an autonomous vehicle in motion as described in any one of claims 1-6.

Images