High-altitude camera high-precision rapid calibration method and device and electronic equipment

Abstract

The application provides a high-precision and rapid calibration method and device for an aerial camera and electronic equipment, and relates to the field of visual calibration. The method comprises the following steps: acquiring image data of a calibration device at different poses; performing feature point extraction on each image data to obtain pixel coordinates of each feature point in each image data, wherein the feature points at least include the intersection of black squares and white squares; acquiring known actual coordinates of each feature point in a world coordinate system; performing coefficient correction on a preset initial coordinate mapping model based on the pixel coordinates of each feature point and the corresponding actual coordinates to obtain a coordinate mapping model, and the coordinate mapping model is used to realize the conversion between the pixel coordinates of the aerial camera and the world coordinates, so as to realize high-precision and rapid calibration of the aerial camera.

Description

Technical Field

[0001] This application relates to the field of visual calibration, and in particular to a high-precision and rapid calibration method, device and electronic equipment for high-altitude cameras. Background Technology

[0002] In automated port quay crane operations, aerial cameras need to convert the pixel coordinates of the acquired images into ground-based world coordinates to guide truck positioning and container handling. Traditional calibration methods strictly follow the geometric model of pinhole imaging, requiring simultaneous calibration of both intrinsic camera parameters (such as focal length, principal point, and distortion coefficients) and extrinsic parameters (such as rotation matrix and translation vector). Only through joint calculation of intrinsic and extrinsic parameters can the conversion between pixel coordinates and actual coordinates be achieved.

[0003] However, the extrinsic parameter solution process in the existing technology is extremely sensitive to feature point extraction errors. Under the conditions of a large field of view at an altitude of 35m to 60m, tiny pixel deviations will be amplified sharply after being back-calculated by extrinsic parameters, making it difficult for the ground positioning error to meet the 2-3cm operation accuracy requirements of the port.

[0004] In addition, some solutions introduce multi-sensor collaborative calibration, such as lidar, to improve robustness. This not only significantly increases hardware costs but also makes the system susceptible to interference from harsh environments such as high dust levels and strong sunlight in ports, further affecting the stability and accuracy of external parameter calibration. Summary of the Invention

[0005] In view of this, this application provides a high-precision and rapid calibration method, device and electronic equipment for high-altitude cameras, which solves the technical problems of existing port quay crane high-altitude camera calibration methods, which rely on multi-sensor collaboration and require simultaneous calibration of internal and external parameters, resulting in redundant processes, high costs and complex operations, and are difficult to adapt to large-scale scenarios at altitudes of 35m-60m, thus causing the external parameter calibration accuracy and efficiency to fail to meet the needs of automated operations.

[0006] The present application is described below from multiple aspects, and the implementation methods and beneficial effects of these aspects can be referred to each other.

[0007] In a first aspect, this application provides a high-precision and rapid calibration method for an aerial camera, applied to a calibration system consisting of an aerial camera and a calibration device. The aerial camera is positioned within a preset distance from the ground, and the calibration device is set on the ground and positioned opposite the aerial camera. The calibration device has black and white squares. The method includes:

[0008] Acquire image data of the calibration device in different poses;

[0009] Feature points are extracted from each image data to obtain the pixel coordinates of each feature point in each image data. The feature points include at least the intersection of black squares and white squares.

[0010] Obtain the known actual coordinates of each feature point in the world coordinate system;

[0011] Based on the pixel coordinates and corresponding actual coordinates of each feature point, the coefficients of the preset initial coordinate mapping model are corrected to obtain the coordinate mapping model. The coordinate mapping model is used to realize the conversion between the pixel coordinates of the high-altitude camera and the world coordinates, so as to achieve high-precision and rapid calibration of the high-altitude camera.

[0012] According to this scheme, the introduction of multiple sensors such as lidar is abandoned at the hardware level. It relies solely on the cooperation of high-altitude cameras and ground calibration devices, which reduces hardware costs and system complexity, and avoids additional errors introduced by the harsh environment of port high dust and strong light interference in multi-sensor collaborative calibration, thus significantly improving the robustness of the calibration process.

[0013] At the calibration logic level, this application breaks away from the fixed paradigm of traditional methods that require simultaneous calibration of intrinsic and extrinsic parameters. It does not require explicit solution of the camera's extrinsic parameters (rotation matrix and translation vector). Instead, it corrects the coefficients of the initial coordinate mapping model through the direct correspondence between the feature point pixel coordinates and their known world coordinates. This bypasses the defect in the existing technology where the extrinsic parameter solution process is extremely sensitive to feature point extraction errors. It effectively avoids the problem that small pixel deviations are amplified drastically after being back-calculated by extrinsic parameters under high-altitude and wide-field conditions. While ensuring that the ground positioning accuracy meets the port's 2-3cm high-precision operation requirements, it significantly shortens the calibration time and simplifies on-site operations. It has the comprehensive advantages of high precision, speed, low cost, and strong environmental adaptability.

[0014] In one possible implementation of the first aspect above, the initial coordinate mapping model includes a physical foundation model and a residual correction model;

[0015] Based on the pixel coordinates and corresponding actual coordinates of each feature point, the coefficients of the preset initial coordinate mapping model are corrected to obtain the coordinate mapping model, including:

[0016] S41, Based on the pixel coordinates of each feature point and its corresponding actual coordinates, determine the equivalent focal length and principal point coordinates of the physical base model;

[0017] S42, based on the pixel coordinates of each feature point, its corresponding actual coordinates, equivalent focal length and principal point coordinates, determine the residual correction coefficients of the residual correction model;

[0018] S43, Substitute the equivalent focal length, principal point coordinates and residual correction coefficients into the initial coordinate mapping model to obtain the basic coordinate mapping model;

[0019] S44 uses pixel coordinates as input data for the basic coordinate mapping model to obtain the predicted actual coordinates;

[0020] S45, calculate the actual coordinates of each feature point and the deviation between them and the predicted actual coordinates;

[0021] S46, if the deviation is not greater than the preset threshold, output the basic coordinate mapping model as the coordinate mapping model.

[0022] According to this scheme, the physical foundation model and the residual correction model work together in a collaborative manner. The former provides benchmark constraints that conform to imaging principles to avoid overfitting, while the latter specifically absorbs system residuals such as lens distortion and feature point extraction. A closed-loop verification mechanism based on a deviation threshold is introduced to ensure that only mapping models that meet the accuracy requirements can be output. This provides an implementation architecture for coordinate mapping in high-altitude, wide-field-of-view scenarios that combines physical interpretability and high accuracy, effectively improving the reliability of calibration results and the quantifiability of quality control.

[0023] In one possible implementation of the first aspect above, if the deviation is greater than a preset threshold, new feature points are extracted from each image data and replaced with the new feature points.

[0024] Repeat steps S41-S45 iteratively until the difference between the actual coordinates of each feature point and its corresponding predicted actual coordinates is no greater than a preset threshold. Then, output the basic coordinate mapping model as the coordinate mapping model.

[0025] According to this scheme, by re-extracting feature points from the image data and replacing the original feature points, the physical parameter solving and residual fitting steps are iteratively repeated until the accuracy requirements are met before outputting the coordinate mapping model. This mechanism effectively avoids the risk of overfitting caused by blindly adjusting the model structure or increasing the model order when the feature point extraction quality is poor. It replaces the more costly model reconstruction or image re-acquisition with feature point reselection, significantly reducing the implementation cost and complexity of iterative optimization while ensuring calibration accuracy. It also enhances the adaptability of the calibration process and the reliability of the final model, ensuring that the coordinate mapping model output under any conditions has passed the quantized accuracy verification.

[0026] In one possible implementation of the first aspect above, the residual correction coefficients of the residual correction model are determined based on the pixel coordinates of each feature point, its corresponding actual coordinates, equivalent focal length, and principal point coordinates, including:

[0027] Based on each pixel coordinate, its corresponding actual coordinate, equivalent focal length, and principal point coordinate, determine the coordinate residual of each feature point;

[0028] Based on each coordinate residual, the coefficients of the residual correction model are solved using the least squares method to obtain the residual correction coefficients.

[0029] According to this scheme, the coordinate residuals of each feature point are first calculated based on the physical model parameters. Then, the least squares method is used to perform a global optimal fit on these residuals, thereby determining all the correction coefficients of the second-order polynomial in one go. This scheme, on the one hand, utilizes the statistical optimality of the least squares method to effectively suppress the interference of noise from individual feature point extraction on the correction coefficients, improving the model's noise resistance. On the other hand, it uses explicit mathematical methods to solve for the parameters in the residual correction stage, making the entire two-stage mapping model construction process logically rigorous, computationally efficient, and easy to implement in industrial settings.

[0030] In one possible implementation of the first aspect described above, the method further includes:

[0031] Image data is preprocessed, wherein the preprocessing includes at least one of grayscale conversion and binarization.

[0032] According to this scheme, by performing grayscale and / or binarization preprocessing on the image data before feature point extraction, the redundancy of color and illumination information in the original image is effectively reduced, and the contrast between the black and white checkerboard squares is enhanced, thereby improving the accuracy and repeatability of corner point (i.e. feature point) detection and providing more reliable pixel coordinate input for subsequent physical model parameter solving and accurate fitting of residual correction coefficients.

[0033] In one possible implementation of the first aspect described above, feature point extraction is performed on each image data, including:

[0034] Feature points are extracted from each image data using a corner detection algorithm.

[0035] According to this scheme, by employing a corner detection algorithm for feature point extraction, the edge pixels at the intersection of black and white squares can be accurately located, and then the corner coordinates with sub-pixel precision can be calculated, significantly improving the accuracy and noise resistance of feature point extraction. Meanwhile, edge detection, as a mature and computationally efficient image processing method, can work stably in the complex lighting and dusty environment of a port, providing high-quality input data for subsequent physical model parameter solving and residual correction, further ensuring the robustness and engineering applicability of the entire calibration method.

[0036] Secondly, this application provides a high-precision and rapid calibration device for an aerial camera, comprising:

[0037] The acquisition module is used to acquire image data of the calibration device at different poses;

[0038] The feature point extraction module is used to extract feature points from each image data to obtain the pixel coordinates of each feature point in each image data. The feature points include at least the intersection of black squares and white squares.

[0039] The acquisition module is also used to acquire the known actual coordinates of each feature point in the world coordinate system;

[0040] The correction module is used to correct the coefficients of the preset initial coordinate mapping model based on the pixel coordinates and the corresponding actual coordinates of each feature point, so as to obtain the coordinate mapping model. The coordinate mapping model is used to realize the conversion between the pixel coordinates of the high-altitude camera and the world coordinates, so as to achieve high-precision and rapid calibration of the high-altitude camera.

[0041] Thirdly, this application provides an electronic device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the high-precision rapid calibration method for high-altitude cameras disclosed in the first aspect and any possible implementation thereof.

[0042] Fourthly, this application provides a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the high-precision rapid calibration method for high-altitude cameras disclosed in the first aspect and any possible implementation thereof.

[0043] Fifthly, this application provides a computer program product comprising: computer instructions that, when executed on an electronic device, cause the electronic device to perform the high-precision rapid calibration method for an aerial camera disclosed in the first aspect and any possible implementation thereof.

[0044] The beneficial effects of the second to fifth aspects can be found in the first aspect and the beneficial effects of any possible implementation of the first aspect, and will not be repeated here. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the calibration system in the embodiments of this application;

[0046] Figure 2 This is a flowchart of the high-precision and rapid calibration process for high-altitude cameras in the embodiments of this application;

[0047] Figure 3 A flowchart illustrating the correction of the initial coordinate mapping model in the embodiments of this application;

[0048] Figure 4 This is a flowchart illustrating the calculation of residual correction coefficients in the embodiments of this application;

[0049] Figure 5 This is a flowchart illustrating the calibration process of the high-altitude camera in an embodiment of this application.

[0050] Figure 6 This is a structural block diagram of the high-precision rapid calibration device for high-altitude cameras in the embodiments of this application;

[0051] Figure 7 A block diagram of an electronic device provided in an embodiment of this application;

[0052] Figure 8 This is a block diagram of a SoC (system on chip) provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] The technical problems to be solved by the embodiments of this application will be described below.

[0055] As described in the background technology section above, the existing calibration method for high-altitude cameras on port quay cranes introduces redundant calibration procedures and error amplification problems due to the strict adherence to the traditional paradigm of joint calculation of internal and external parameters. That is, this method requires simultaneous calibration of internal and external parameters. Under the conditions of a large field of view at an altitude of 35m to 60m, the tiny pixel deviation of feature points will be amplified sharply after being back-calculated by the external parameters, making it difficult for the ground positioning error to meet the high-precision operation requirements of 2-3cm in ports.

[0056] Therefore, to address the aforementioned problems, this application provides a high-precision, rapid calibration method, apparatus, and electronic device for high-altitude cameras. This method is applied to a system consisting of a high-altitude camera and a ground-based calibration device. The method first acquires multiple image data from the calibration device at different poses and extracts the pixel coordinates of the intersections of black and white squares in each image. Simultaneously, it obtains the known actual coordinates of these feature points in the world coordinate system. Then, based on the pixel coordinates of all feature points and their corresponding actual coordinates, a preset initial coordinate mapping model is modified using coefficients to obtain a coordinate mapping model that can directly convert pixel coordinates to world coordinates, thereby completing the calibration.

[0057] This method eliminates the cumbersome process of simultaneously calibrating camera intrinsic parameters and explicitly solving extrinsic parameters. It directly constructs the mapping relationship between pixel coordinates and ground world coordinates. While simplifying calibration operations, shortening calibration time, and reducing hardware costs, it effectively avoids the problem that tiny pixel deviations are amplified by back-calculation of extrinsic parameters under high-altitude, wide-field-of-view conditions. This provides reliable technical support for achieving 2-3cm high-precision operation positioning in ports.

[0058] To better understand the high-precision and rapid calibration method for high-altitude cameras in the embodiments of this application, the following is combined with... Figure 1 First, the calibration system of the embodiments of this application on which this method depends will be described in detail.

[0059] refer to Figure 1 , Figure 1 A schematic diagram of the calibration system in an embodiment of this application is shown.

[0060] like Figure 1 As shown in this embodiment, the calibration system includes a high-altitude camera 1 and a calibration device 2. The high-altitude camera 1 is mounted on a quay crane 3, and its height is between 35m and 60m. The size of the calibration device 2 can be adjusted according to the actual calibration scenario. For example, the preferred size of the calibration device 2 is 1800mm × 2200mm, which includes black squares 4 and white squares 5, arranged alternately. The number of black squares 4 and white squares 5 can be 9 rows × 8 columns, with each square measuring 250mm × 250mm, ensuring that feature points are clearly distinguishable at a height of 35m-60m, without the need for additional internal parameter calibration equipment or targets.

[0061] Before calibrating the aerial camera 1, fix the calibration device 2 within the working area of ​​the quay crane 3 (it can be fixed to a ground support, the top of a container, etc.), ensuring that the calibration device 2 is fully visible within the camera's field of view. Simultaneously, check the installation status of the aerial camera 1 to ensure it is secure and the lens is unobstructed. Adjust the exposure parameters of the aerial camera 1 (such as exposure time and aperture) to eliminate the influence of complex port lighting and dust on imaging, ensuring clear image acquisition by the calibration device 2. No internal parameter calibration-related camera parameter presets or adjustments are required.

[0062] The high-precision and rapid calibration method for high-altitude cameras in the embodiments of this application will be described in detail below.

[0063] refer to Figure 2 , Figure 2 A flowchart illustrating the high-precision and rapid calibration process of an aerial camera in an embodiment of this application is shown.

[0064] like Figure 2 As shown in the embodiments of this application, the high-precision rapid calibration method for high-altitude cameras includes steps S10-S40.

[0065] S10 involves acquiring image data of the calibration device in different poses.

[0066] It should be noted that during the acquisition process, a polynomial fitting error correction mechanism and lens distortion correction are used to resist environmental interference, ensuring that the feature points of the calibration device (i.e., the intersections of black and white squares) are clearly distinguishable in each image, without blurry or severely distorted images, thus avoiding affecting the accuracy of subsequent feature point extraction and mapping models. Furthermore, it eliminates the need to acquire images in a specific pose or number required for external parameter calibration, and eliminates the need to adjust the angle between the calibration plate and the ground.

[0067] In some embodiments, the above method further includes:

[0068] Image data is preprocessed, wherein the preprocessing includes at least one of grayscale conversion and binarization.

[0069] Understandably, since the original image contains a large amount of redundant color and lighting information, directly converting or binarizing the image before feature point extraction can significantly enhance the contrast of the black and white checkerboard squares. This not only improves the accuracy and repeatability of corner detection, but also provides more reliable pixel-level input data for subsequent physical model parameter solving and accurate fitting of residual correction coefficients.

[0070] S20: Extract feature points from each image data to obtain the pixel coordinates of each feature point in each image data.

[0071] It is understandable that the aforementioned feature points include at least the intersections of the black and white squares.

[0072] In some embodiments, feature point extraction is performed on each image data, including:

[0073] Feature points are extracted from each image data using a corner detection algorithm.

[0074] For example, feature points can be extracted from each image data using the Harris corner detection algorithm.

[0075] S30: Obtain the known actual coordinates of each feature point in the world coordinate system.

[0076] It should be noted that in actual operation, the calibration device is laid flat on the ground, and a world coordinate system is established with a certain point of the calibration device as the origin, and the vertical coordinate of the actual coordinate of each feature point on the calibration device is 0.

[0077] S40, based on the pixel coordinates and corresponding actual coordinates of each feature point, performs coefficient correction on the preset initial coordinate mapping model to obtain the coordinate mapping model.

[0078] For example, the initial coordinate mapping model is as follows:

[0079]

[0080] Among them, (X) i Y i Z i ) represents the actual coordinates of the i-th feature point, (u i v i ) represents the pixel coordinates of the i-th feature point, (c_x, c_y) represents the principal point coordinates, f represents the equivalent focal length, and p0, p1, p2, p3, p4, p5, q0, q1, q2, q3, q4 and q5 are all residual correction coefficients.

[0081] Understandably, the coordinate mapping model is used to realize the conversion between pixel coordinates and world coordinates of the high-altitude camera. This mapping model replaces the traditional step-by-step calculation process of intrinsic and extrinsic parameters for pinhole imaging. It no longer completes the conversion from pixel to ground coordinates through the extrinsic parameter matrix, eliminating the pixel error amplification defect from the root of calculation, so as to achieve high-precision and rapid calibration of the high-altitude camera.

[0082] It should be noted that this application only uses an aerial camera and calibration device, without the need to introduce multiple sensors such as lidar, which reduces hardware costs and system complexity, and avoids the additional errors caused by multi-sensor collaborative calibration under harsh port environments (such as high dust and strong light), resulting in stronger stability.

[0083] Instead of directly solving for camera extrinsic parameters during calibration, the initial coordinate model is corrected using the pixel coordinates of feature points and their known world coordinates to obtain a coordinate mapping model that accurately maps pixel coordinates to actual coordinates. This avoids the sensitivity to feature point extraction errors during extrinsic parameter solving and prevents the drastic amplification of small pixel deviations under high altitude and wide field of view conditions. While maintaining a ground positioning accuracy of 2-3 cm, it significantly shortens calibration time, simplifies on-site operations, and combines high precision, speed, low cost, and strong environmental adaptability.

[0084] The following is a detailed explanation of step S40.

[0085] refer to Figures 3 to 4 , Figure 3 A flowchart illustrating the initial coordinate mapping model correction in an embodiment of this application is shown. Figure 4 A flowchart illustrating the calculation of residual correction coefficients in an embodiment of this application is shown.

[0086] In some embodiments, the initial coordinate mapping model includes a physical basis model and a residual correction model.

[0087] Continuing with the example above, the expression for the fundamental physical model is:

[0088]

[0089] The residual correction model expression is:

[0090]

[0091] Among them, (X_base i Y_base i ) is the estimated actual coordinate value of the i-th feature point, ( , ) represents the coordinate residual between the predicted value and the actual value of the physical model for the i-th feature point.

[0092] like Figure 3 As shown, in some embodiments, based on the pixel coordinates and corresponding actual coordinates of each feature point, the preset initial coordinate mapping model is modified by coefficients to obtain the coordinate mapping model, including steps S41-S46.

[0093] S41 determines the equivalent focal length and principal point coordinates of the physical model based on the pixel coordinates of each feature point and its corresponding actual coordinates.

[0094] For example,

[0095] Where f_x is the equivalent focal length estimate obtained by solving the feature points in the X direction of the image separately, and f_y is the equivalent focal length estimate obtained by solving the feature points in the Y direction of the image separately.

[0096] S42, based on the pixel coordinates of each feature point, its corresponding actual coordinates, equivalent focal length, and principal point coordinates, determine the residual correction coefficients of the residual correction model.

[0097] like Figure 4 As shown, in some embodiments, the residual correction coefficients of the residual correction model are determined based on the pixel coordinates of each feature point, its corresponding actual coordinates, equivalent focal length, and principal point coordinates, including steps S421-S422.

[0098] S421 determines the coordinate residual of each feature point based on each pixel coordinate, its corresponding actual coordinate, equivalent focal length, and principal point coordinate.

[0099] For example,

[0100] S422, based on each coordinate residual, the coefficients of the residual correction model are solved by the least squares method to obtain the residual correction coefficients.

[0101] Understandably, the approach involves first calculating the coordinate residuals based on the physical model, and then using the least squares method to perform a one-time global optimal fit on the second-order polynomial correction coefficients. This leverages the statistical optimality of least squares to effectively suppress the interference of noise from individual feature point extraction on the correction coefficients, thereby improving the model's noise resistance and robustness. Furthermore, this solution path has clear mathematical logic and high computational efficiency, facilitating engineering implementation in industrial settings and ensuring the reliability and operability of the entire two-stage mapping model construction process.

[0102] S43. Substitute the equivalent focal length, principal point coordinates, and residual correction coefficients into the initial coordinate mapping model to obtain the basic coordinate mapping model.

[0103] S44 uses pixel coordinates as input data for the basic coordinate mapping model to obtain the predicted actual coordinates.

[0104] S45, calculate the actual coordinates of each feature point and the deviation between them and its corresponding predicted actual coordinates.

[0105] S46, if the deviation is not greater than the preset threshold, output the basic coordinate mapping model as the coordinate mapping model.

[0106] For example, the preset threshold (physical spatial location threshold) can be set to 15mm.

[0107] Understandably, the physical foundation model and the residual correction model work together. The physical foundation model establishes benchmark constraints based on the imaging mechanism, avoiding the risk of overfitting. The residual correction model is used to fit various system residuals such as lens distortion and feature point extraction errors. The algorithm introduces a bias threshold closed-loop verification mechanism, only using mapping models that meet the accuracy requirements as valid outputs. This forms a coordinate mapping architecture suitable for high-altitude, large-field-of-view scenarios, significantly enhancing the reliability of calibration results.

[0108] In some embodiments, if the deviation is greater than a preset threshold, new feature points are extracted from each image data and replaced with the new feature points.

[0109] Repeat steps S41-S45 iteratively until the difference between the actual coordinates of each new feature point and its corresponding predicted actual coordinates is no greater than a preset threshold. Then, output the basic coordinate mapping model as the coordinate mapping model.

[0110] It should be noted that this application adopts a strategy of re-extracting new feature points from the original image and iteratively replacing the original feature points, rather than continuing to adjust model parameters or increase the model order based on the original feature points. The purpose is that if the feature point extraction itself has a large error (such as corner point positioning offset), no matter how the model parameters are optimized subsequently, it will be based on data with errors, which will not only fail to improve accuracy, but will also easily lead to overfitting, causing the model to have a memory effect on the original feature points and lose its generalization ability.

[0111] By re-extracting feature points and replacing more costly model reconstruction or image re-acquisition with feature point-level reselection, the implementation cost and complexity of iterative optimization are significantly reduced while ensuring calibration accuracy.

[0112] After training the coordinate mapping model described above, the following accuracy verification needs to be performed on the coordinate mapping model:

[0113] The trained coordinate mapping model is substituted into the camera imaging model, and a new image of the calibration device, laid flat on the ground, is acquired. Feature points are extracted from this image. The pixel coordinates of these feature points are then substituted into the coordinate mapping model to calculate estimated actual coordinates. Finally, based on these estimated actual coordinates and the actual coordinates, the positioning accuracy of the feature points is verified. Simultaneously, in the actual operation scenario of the port quay crane, images of targets such as containers and spreader gear are acquired to verify whether the target positioning error is within a preset threshold. If the error is ≤15mm, the calibration is considered successful. If the error is greater than 15mm, the coordinate mapping model is retrained.

[0114] The entire execution process of this application is described in detail below.

[0115] refer to Figure 5 , Figure 5 A flowchart illustrating the calibration process of an aerial camera in an embodiment of this application is shown.

[0116] like Figure 5 As shown in the embodiments of this application, the calibration of the high-altitude camera includes steps S100-S500.

[0117] For S100 calibration, the following steps are required: A high-precision black and white checkerboard calibration board is selected. The checkerboard squares are 250mm x 250mm, with a total of 8 x 7 squares. The calibration board is laid flat and fixed on the ground in the quay crane operating area, ensuring complete contact with the ground without tilting, and full visibility within the camera's 35m height field of view. The installation status of the aerial camera is checked to ensure it is not loose, the lens is unobstructed, and glare from port sunlight is eliminated to ensure clear imaging. No external parameter calibration-related camera adjustments are required, nor is the angle between the calibration board and the ground or the camera's optical axis adjusted.

[0118] S200, Image Acquisition: Controls the camera to acquire images of the calibration device laid flat on the ground from five positions, such as the center, upper left edge, upper right edge, lower left edge, and lower right edge of the camera's field of view. Maintaining the calibration plate flat, no tilt adjustment is required. Images are acquired in a single batch, and all feature points of the calibration plate are clearly distinguishable without blurring or severe distortion. There is no need to acquire specific images required for external parameter calibration, nor is it necessary to adjust the angle between the calibration plate and the ground.

[0119] S300, Feature Point Extraction: The calibration board image is preprocessed by converting the color image to grayscale through grayscale conversion. Then, a binarization algorithm is used to segment the black and white squares. Finally, the intersection points of the black and white squares are extracted using a corner detection algorithm or by key point detection using an AI model. A total of 840 feature points (u, v) are extracted for each image. Based on the size of the calibration device's squares and its placement on the ground (with the ground as the Z=0 reference plane), the world coordinates (X, Y, Z) corresponding to each feature point are determined, and an initial coordinate mapping model between pixel coordinates and world coordinates is established.

[0120] S400, Model Optimization: Substitute the coordinates of 840 feature points from the acquired image into the initial coordinate mapping model described above. First, estimate the physical parameters f = 2850 pixels, c_x = 2048, c_y = 1536. Then, solve for the residual correction coefficients {p0-p5} and {q0-q5}.

[0121] The entire process does not calibrate extrinsic parameters, directly constructing a mapping model between pixel coordinates and world coordinates. Simultaneously, it utilizes inherent lens distortion parameters to correct distortion in the feature point pixel coordinates, eliminating errors caused by distortion without needing to solve for camera extrinsic parameters. Finally, the deviation between the predicted world coordinates and the actual coordinates is calculated, with an average deviation of 0.85 pixels and a maximum deviation of 1.2 pixels, both less than the preset threshold of 1.5 pixels, indicating that the mapping model optimization is complete.

[0122] S500, Accuracy Verification: A new image of the calibration device, laid flat on the ground, was acquired. The feature point positioning deviation was 1.5 pixels, corresponding to an actual spatial deviation of 12mm. In the actual operation scenario of the quay crane, a container image was acquired. The conversion between pixel coordinates and world coordinates was achieved solely through a mapping model. The container positioning error was 12mm, less than 15mm, indicating that the calibration was qualified.

[0123] refer to Figure 6 , Figure 6 A structural block diagram of a high-precision rapid calibration device for an aerial camera, as shown in an embodiment of this application, is illustrated.

[0124] like Figure 6As shown in the embodiment of this application, the high-precision rapid calibration device 10000 for high-altitude cameras includes an acquisition module 1000, a feature point extraction module 2000, and a correction module 3000.

[0125] Among them, the acquisition module 1000 is used to acquire image data of different poses of the calibration device;

[0126] The feature point extraction module 2000 is used to extract feature points from each image data to obtain the pixel coordinates of each feature point in each image data. The feature points include at least the intersection of black squares and white squares.

[0127] The acquisition module 1000 is also used to acquire the known actual coordinates of each feature point in the world coordinate system;

[0128] The correction module 3000 is used to correct the coefficients of the preset initial coordinate mapping model based on the pixel coordinates and the corresponding actual coordinates of each feature point, so as to obtain the coordinate mapping model. The coordinate mapping model is used to realize the conversion between the pixel coordinates of the high-altitude camera and the world coordinates, so as to realize the high-precision and rapid calibration of the high-altitude camera.

[0129] The various modules in the apparatus of this application execute the methods of the above embodiments, and their specific functions can be referred to the description of the above embodiments, which will not be repeated here.

[0130] Now for reference Figure 7 , Figure 7 This is a block diagram of the electronic device 1200 in an embodiment of this application. The electronic device 1200 may include one or more processors (corresponding to) coupled to the controller hub 1203. Figure 7 The first processor 1201 is described above. In at least one embodiment, the controller hub 1203 communicates with the first processor 1201 via a multi-branch bus such as a front-side bus (FSB), a point-to-point interface such as a quick-path interconnect (QPI), or a similar connection. The first processor 1201 executes instructions that control general types of data processing operations. In one embodiment, the controller hub 1203 includes, but is not limited to, a graphics memory controller hub (GMCH) (not shown) and an input / output hub (IOH) (which may be on separate chips) (not shown), wherein the GMCH includes memory and a graphics controller and is coupled to the IOH.

[0131] Electronic device 1200 may also include a coprocessor coupled to controller hub 1203 (corresponding to...) Figure 7The processor 1202 and memory 1204 are integrated within the processor (as described in this application). Alternatively, one or both of the memory and GMCH can be integrated within the processor (as described in this application), with memory 1204 and the first coprocessor 1202 directly coupled to the first processor 1201 and the controller hub 1203, which is located on a single chip with the IOH. Memory 1204 can be, for example, dynamic random access memory (DRAM), phase change memory (PCM), or a combination of both. In one embodiment, the first coprocessor 1202 is a dedicated processor, such as a high-throughput MIC processor (many integerized core, MIC), a network or communication processor, a compression engine, a graphics processor, a general-purpose computing on GPU (GPGPU), or an embedded processor, etc. Optional properties of the first coprocessor 1202 are indicated by dashed lines. Figure 7 middle.

[0132] As a computer-readable storage medium, memory 1204 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. For example, memory 1204 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device such as one or more hard-disk drives (HDDs), one or more compact disc (CD) drives, and / or one or more digital versatile disc (DVD) drives.

[0133] In one embodiment, electronic device 1200 may further include a network interface controller (NIC) 1206. Network interface 1206 may include a transceiver for providing a radio interface for electronic device 1200 to communicate with any other suitable device, such as a front-end module, antenna, etc. In various embodiments, network interface 1206 may be integrated with other components of electronic device 1200. Network interface 1206 can implement the functions of the communication unit in the above embodiments.

[0134] Electronic device 1200 may further include input / output (I / O) device 1205. I / O device 1205 may include: a user interface designed to enable a user to interact with electronic device 1200; a peripheral component interface designed to enable peripheral components to also interact with electronic device 1200; and / or sensors designed to determine environmental conditions and / or location information related to electronic device 1200.

[0135] It is worth noting that, Figure 7 This is merely an example. That is, although... Figure 7 The electronic device 1200 is shown to include multiple devices such as a first processor 1201, a first coprocessor 1202, a controller hub 1203, and a memory 1204. However, in actual applications, devices using the methods of this application may include only a portion of the devices in the electronic device 1200. For example, it may include only the first processor 1201 and the network interface 1206. Figure 7 The properties of the optional devices are shown in dashed lines. According to some embodiments of this application, the memory 1204, which is a computer-readable storage medium, stores instructions that, when executed on a computer, cause the electronic device 1200 to perform the high-precision rapid calibration method for high-altitude cameras according to the above embodiments. Specific details can be found in the methods described in the above embodiments, and will not be repeated here.

[0136] Now for reference Figure 8 , Figure 8 A block diagram of the SoC (system on chip) 1300 in an embodiment of this application is shown. Figure 8 In the diagram, similar components share the same reference numerals. Additionally, dashed boxes are an optional feature for more advanced SoCs. Figure 8 In the SoC1300, interconnect unit 1350 is coupled to the processor (corresponding to...). Figure 8 The system includes a second processor 1310, a system agent unit 1380, a bus controller unit 1390, an integrated memory controller unit 1340, and one or more coprocessors (corresponding to...). Figure 8 The second coprocessor 1320 may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 1330; and a direct memory access (DMA) unit 1360. In one embodiment, the second coprocessor 1320 includes a dedicated processor, such as a network or communication processor, a compression engine, a GPGPU, a high-throughput MIC processor, or an embedded processor.

[0137] The static random access memory (SRAM) cell 1330 may include one or more computer-readable media for storing data and / or instructions. The computer-readable storage medium may store instructions, specifically, temporary and permanent copies of those instructions. These instructions may include, when executed by at least one unit in the processor, causing the SoC 1300 to perform the high-precision rapid calibration method for an aerial camera according to the above embodiments, as detailed in the methods described above, which will not be repeated here.

[0138] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0139] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as a digital signal processor (DSP), microcontroller, application-specific integrated circuit (ASIC), or microprocessor.

[0140] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0141] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc read-only memory (CD-ROMs), magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0142] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0143] This application also provides a computer-readable storage medium, the specific functions and corresponding technical effects of which can be referred to the above embodiments. Figures 1-5 The methods explained will not be elaborated here.

[0144] This application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are loaded and executed by a processor, they implement the high-precision and rapid calibration method for high-altitude cameras described in the above embodiments. The specific functions and corresponding technical effects can be found in the above embodiments. Figures 1-5 The methods explained will not be elaborated here.

[0145] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0146] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0147] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0148] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A high-precision and rapid calibration method for high-altitude cameras, characterized in that, A method applicable to a system consisting of an aerial camera and a calibration device, wherein the aerial camera is positioned within a preset distance from the ground, the calibration device is mounted on the ground and positioned opposite the aerial camera, and the calibration device has black and white squares; the method includes: Acquire image data of the calibration device in different poses; Feature points are extracted from each of the image data to obtain the pixel coordinates of each feature point in each of the image data, wherein the feature points include at least the intersection of the black square and the white square; Obtain the known actual coordinates of each feature point in the world coordinate system; Based on the pixel coordinates and corresponding actual coordinates of each feature point, the preset initial coordinate mapping model is modified by coefficients to obtain a coordinate mapping model. The coordinate mapping model is used to realize the conversion between the pixel coordinates of the high-altitude camera and the world coordinates, so as to realize the high-precision and fast calibration of the high-altitude camera.

2. The method according to claim 1, characterized in that, The initial coordinate mapping model includes a physical foundation model and a residual correction model; The step of correcting the coefficients of a preset initial coordinate mapping model based on the pixel coordinates and corresponding actual coordinates of each feature point to obtain a coordinate mapping model includes: S41, Based on the pixel coordinates of each feature point and its corresponding actual coordinates, determine the equivalent focal length and principal point coordinates of the physical base model; S42, based on the pixel coordinates of each feature point, its corresponding actual coordinates, the equivalent focal length, and the principal point coordinates, determine the residual correction coefficients of the residual correction model; S43, Substitute the equivalent focal length, the principal point coordinates and the residual correction coefficient into the initial coordinate mapping model to obtain the basic coordinate mapping model; S44, The pixel coordinates are used as input data for the basic coordinate mapping model to obtain the predicted actual coordinates; S45, calculate the deviation between the actual coordinates of each feature point and its corresponding predicted actual coordinates; S46, if the deviation is not greater than a preset threshold, the basic coordinate mapping model is output as the coordinate mapping model.

3. The method according to claim 2, characterized in that, If the deviation is greater than the preset threshold, new feature points are extracted from each of the image data and replaced with the new feature points. Repeat steps S41-S45 iteratively until the difference between the actual coordinates of each feature point and its corresponding predicted actual coordinates is no greater than the preset threshold, and then output the basic coordinate mapping model as the coordinate mapping model.

4. The method according to claim 2, characterized in that, The determination of the residual correction coefficients of the residual correction model based on the pixel coordinates of each feature point, its corresponding actual coordinates, the equivalent focal length, and the principal point coordinates includes: Based on each pixel coordinate, its corresponding actual coordinate, the equivalent focal length, and the principal point coordinate, determine the coordinate residual of each feature point; Based on each coordinate residual, the coefficients of the residual correction model are solved by the least squares method to obtain the residual correction coefficients.

5. The method according to claim 1, characterized in that, The method further includes: The image data is preprocessed, wherein the preprocessing includes at least one of grayscale conversion and binarization.

6. The method according to claim 1, characterized in that, The step of extracting feature points from each of the image data includes: Feature points are extracted from each of the image data using a corner detection algorithm.

7. A high-precision and rapid calibration device for an aerial camera, characterized in that, include: The acquisition module is used to acquire image data of the calibration device at different poses; The feature point extraction module is used to extract feature points from each of the image data to obtain the pixel coordinates of each feature point in each of the image data, wherein the feature point includes at least the intersection of the black square and the white square; The acquisition module is also used to acquire the known actual coordinates of each feature point in the world coordinate system; The correction module is used to correct the coefficients of the preset initial coordinate mapping model based on the pixel coordinates and the corresponding actual coordinates of each feature point to obtain a coordinate mapping model. The coordinate mapping model is used to realize the conversion between the pixel coordinates of the high-altitude camera and the world coordinates, so as to realize the high-precision and fast calibration of the high-altitude camera.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the high-precision and rapid calibration method for high-altitude cameras as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the high-precision and rapid calibration method for high-altitude cameras as described in any one of claims 1-6.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the high-precision and rapid calibration method for high-altitude cameras as described in any one of claims 1-6.

Images