Panoramic camera and turntable mobile measurement calibration method and system

Abstract

The present application relates to the field of mobile measurement technology, and particularly relates to a panoramic camera and rotating disc mobile measurement calibration method and system, comprising: initialization calibration, obtaining the camera internal parameter and the initial rotating center of the rotating disc as the initial reference of the deviation model; real-time dynamic deviation sensing, collecting the position deviation between the rotating shaft of the rotating disc and the optical center of the camera, the camera attitude angle deviation and the environment data in real time through the bimodal dynamic deviation sensing module. The present application obtains the camera internal parameter and the initial rotating center of the rotating disc as the reference through initialization calibration, laying a foundation for subsequent dynamic deviation calculation; then the position deviation, the attitude angle deviation and the environment data are collected in real time through the bimodal dynamic deviation sensing module, a deep learning model is constructed combining adaptive deviation modeling, the change law of the deviation with the environment and the running time is dynamically learned and the prediction value is output, solving the limitation that the traditional static calibration cannot capture dynamic deviation in real time.

Description

Technical Field

[0001] This invention relates to the field of mobile measurement technology, specifically to a method and system for calibrating a panoramic camera and a turntable for mobile measurement. Background Technology

[0002] Panoramic cameras and turntable motion measurement technology are commonly used in 3D modeling, virtual reality, and other fields. The panoramic camera is responsible for acquiring image information of the surrounding environment, while the turntable controls the camera's horizontal rotation. Working together, they can achieve full-view scene coverage. Precise calibration is crucial to ensuring the quality of the acquired data; it involves a high degree of matching between the position, orientation, and motion parameters of the camera and the turntable. Inaccurate calibration can lead to problems such as misaligned stitched images and distorted 3D point clouds.

[0003] However, existing technologies face a subtle but far-reaching problem during calibration: due to a non-ideal alignment deviation between the turntable's rotation axis and the camera's optical center, and this deviation dynamically changes due to factors such as ambient temperature and mechanical wear, traditional static calibration methods cannot correct this error in real time. This dynamic deviation gradually accumulates during long-term, large-scale scene scanning, leading to a decrease in the geometric consistency of panoramic image stitching. Existing solutions mostly rely on preset rigid body transformation parameters and do not consider online compensation for dynamic alignment deviations, resulting in uncorrectable accuracy degradation in measurement results at medium to long distances. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a panoramic camera and turntable motion measurement calibration method and system, which solves the problems of static calibration being unable to correct dynamic deviations in real time, deviation accumulation causing image stitching misalignment and 3D point cloud distortion, and lack of online compensation compared to existing technologies.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for calibrating a panoramic camera and a turntable for motion measurement, comprising:

[0006] Initialize the calibration, obtain the camera's intrinsic parameters and the turntable's initial rotation center, as the initial reference for the deviation model;

[0007] Real-time dynamic deviation sensing: The dual-modal dynamic deviation sensing module collects in real time the positional deviation between the turntable rotation axis and the camera optical center, the camera attitude angle deviation, and environmental data, including ambient temperature change data and turntable vibration data.

[0008] Adaptive deviation modeling is used to construct a deep learning deviation correlation model based on the position deviation, attitude angle deviation, ambient temperature change data and turntable operation vibration data. The model dynamically learns the variation law of deviation with environmental factors and running time, and outputs dynamic deviation prediction values ​​in real time.

[0009] The dual-dimensional real-time compensation transforms the predicted dynamic deviation value into mechanical control commands, and adjusts the rotation center of the turntable and the angle of the camera mounting bracket in real time through the servo motor to correct the mechanical alignment error; and before image stitching, the coordinate transformation of the single-frame panoramic image is corrected according to the predicted dynamic deviation value to eliminate the image pixel misalignment caused by mechanical deviation, and then feature matching and stitching are performed.

[0010] The accuracy closed-loop feedback optimization quantifies the current calibration accuracy by calculating the overlap of stitched image edges and analyzing the reprojection error of 3D point cloud. If the accuracy is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation perception module and the weight coefficient of the deep learning deviation association model are adjusted.

[0011] Furthermore, the dual-modal dynamic deviation sensing module includes:

[0012] A deviation monitoring unit is used to integrate a laser displacement sensor and a micro inertial measurement unit. The laser displacement sensor is used to collect the radial position deviation between the turntable rotation axis and the optical center of the camera in real time, and the micro inertial measurement unit is used to capture the camera attitude angle deviation.

[0013] The environmental correlation unit is used to add temperature sensors and mechanical vibration sensors to collect the environmental temperature change data and the turntable operation vibration data, and to provide environmental impact factor input for the deviation correlation model.

[0014] Furthermore, the adaptive deviation modeling includes:

[0015] A deep learning bias correlation model based on temporal convolutional networks is constructed. The model uses position bias, pose angle bias and environmental data as training samples to dynamically learn the variation law of the bias with environmental factors and runtime, and outputs the predicted value of the dynamic bias in real time.

[0016] A model iteration mechanism is set up so that the parameters of the deviation correlation model are automatically updated with newly collected deviation data at preset intervals.

[0017] Furthermore, the mechanical control compensation subunit in the dual-dimensional real-time compensation includes:

[0018] The radial position deviation output by the dynamic deviation prediction value is converted into a turntable control command, and the rotation center of the turntable is adjusted in real time by a servo motor.

[0019] The attitude angle deviation output by the dynamic deviation prediction value is converted into a camera mounting command, and the angle of the camera mounting bracket is adjusted in real time by the servo motor to correct the alignment error at the mechanical motion level.

[0020] Furthermore, the image processing compensation subunit in the dual-dimensional real-time compensation includes:

[0021] Before performing panoramic image stitching, coordinate transformation is performed on a single frame panoramic image based on the dynamic deviation prediction value. The coordinate transformation includes translation correction and rotation correction to eliminate image pixel misalignment caused by the mechanical deviation.

[0022] After performing the coordinate transformation correction, feature matching and stitching are performed on the single-frame panoramic image to ensure the geometric consistency of the panoramic image.

[0023] Furthermore, the accuracy closed-loop feedback optimization includes:

[0024] By calculating the edge overlap of stitched images, the pixel deviation at the stitching point of adjacent images is detected, and the current calibration accuracy is quantified.

[0025] By analyzing the reprojection error of the 3D point cloud, the correspondence between the 3D point cloud and the image is compared to quantify the current calibration accuracy.

[0026] If the current calibration accuracy is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation perception module will be automatically adjusted; or, the weight coefficients of the deep learning deviation association model will be automatically adjusted.

[0027] Furthermore, the sampling frequency of the bimodal dynamic deviation perception module and the weight coefficients of the deep learning deviation correlation model are adjusted, including:

[0028] If the calibration accuracy is lower than a preset threshold, the sampling frequency of the dual-modal dynamic deviation sensing module is increased to acquire deviation data more frequently.

[0029] If the calibration accuracy is lower than a preset threshold, the weight coefficients of the deep learning bias correlation model are adjusted, including adjusting the calculation proportion that enhances the impact of environmental temperature changes on the bias.

[0030] Furthermore, the initialization calibration includes:

[0031] Under standard environmental conditions, the camera's intrinsic parameters and the initial rotation center of the turntable are obtained through traditional static calibration methods.

[0032] The acquired camera intrinsic parameters and the initial rotation center of the turntable are used as the initial benchmark for the deep learning bias correlation model.

[0033] Furthermore, the method steps include:

[0034] After the panoramic measurement system is started, the real-time dynamic deviation sensing step is executed repeatedly to continuously acquire the latest deviation data and environmental data;

[0035] Next, the adaptive bias modeling step is executed repeatedly to iteratively update the deep learning bias association model and output the dynamic bias value at the current time.

[0036] Next, the two-dimensional real-time compensation steps are executed repeatedly to simultaneously complete the deviation compensation at the mechanical and data levels;

[0037] Next, the accuracy closed-loop feedback optimization step is executed repeatedly. If the calibration accuracy does not meet the standard, the parameters of the sensing, modeling and compensation links are adjusted until the measurement is completed.

[0038] This invention also provides a panoramic camera and turntable motion measurement calibration system, applied to the panoramic camera and turntable motion measurement calibration method described in any one of the above claims, characterized in that it includes:

[0039] The dual-modal dynamic deviation sensing module is used to collect in real time the positional deviation between the turntable rotation axis and the camera optical center, the camera attitude angle deviation, and environmental data, including ambient temperature change data and turntable operation vibration data.

[0040] The adaptive deviation modeling unit is used to build a deep learning deviation correlation model based on position deviation, attitude angle deviation, ambient temperature change data and turntable operation vibration data. It dynamically learns the variation law of deviation with environmental factors and running time, and outputs dynamic deviation prediction values ​​in real time.

[0041] The dual-dimensional real-time compensation unit is used to convert the dynamic deviation prediction value into mechanical control commands, and adjust the rotation center of the turntable and the angle of the camera mounting bracket in real time through the servo motor to correct the mechanical alignment error; and before image stitching, it performs coordinate transformation correction on the single-frame panoramic image based on the dynamic deviation prediction value to eliminate the image pixel misalignment caused by mechanical deviation, and then performs feature matching and stitching.

[0042] The accuracy closed-loop feedback unit is used to quantify the current calibration accuracy by calculating the overlap of stitched image edges and analyzing the reprojection error of 3D point cloud. If the accuracy is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation perception module and the weight coefficient of the deep learning deviation association model are adjusted.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] This invention obtains the initial rotation center of the turntable within the camera through initial calibration as a reference, laying the foundation for subsequent dynamic deviation calculation. Then, a dual-modal dynamic deviation sensing module collects position deviation, attitude angle deviation, and environmental data in real time. Combined with adaptive deviation modeling, a deep learning model is constructed to dynamically learn the variation of deviation with environment and runtime, outputting predicted values. This overcomes the limitation of traditional static calibration in capturing dynamic deviations in real time. Through dual-dimensional real-time compensation, the system corrects alignment errors by adjusting the turntable rotation center and camera support angle at the mechanical level, and corrects single-frame image coordinates to eliminate pixel misalignment at the image processing level, achieving online compensation for dynamic deviations. Finally, through precision closed-loop feedback, the sensing frequency and model weights are optimized to ensure stable calibration accuracy. This effectively solves the problems of traditional technologies, such as the inability to correct dynamic deviations in real time, deviation accumulation leading to image stitching misalignment and 3D point cloud distortion, and lack of online compensation, improving the accuracy and geometric consistency of panoramic measurement data. Attached Figure Description

[0045] Figure 1 This is a flowchart of the method of the present invention;

[0046] Figure 2 This is a structural diagram of the dual-modal dynamic deviation sensing module of the present invention;

[0047] Figure 3 This is a schematic diagram of the adaptive deviation modeling principle of the present invention;

[0048] Figure 4 This is a schematic diagram of the two-dimensional real-time compensation of the present invention;

[0049] Figure 5 This is the accuracy closed-loop feedback logic diagram of the present invention;

[0050] Figure 6 This is a diagram of the system hardware composition of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Please see Figures 1-6 This invention provides a method for calibrating a panoramic camera with a turntable for motion measurement, comprising:

[0053] Initialize the calibration, obtain the camera's intrinsic parameters and the turntable's initial rotation center, as the initial reference for the deviation model;

[0054] Real-time dynamic deviation sensing is achieved by using a dual-modal dynamic deviation sensing module to collect in real time the positional deviation between the turntable rotation axis and the camera optical center, the camera attitude angle deviation, and environmental data. In this embodiment, the environmental data includes ambient temperature change data and turntable vibration data.

[0055] Adaptive deviation modeling is based on position deviation, attitude angle deviation, ambient temperature change data and turntable operation vibration data in this embodiment to construct a deep learning deviation correlation model, dynamically learn the variation law of deviation with environmental factors and running time, and output dynamic deviation prediction value in real time.

[0056] The dual-dimensional real-time compensation transforms the dynamic deviation prediction value in this embodiment into mechanical control commands, and adjusts the rotation center of the turntable and the angle of the camera mounting bracket in real time through the servo motor to correct the mechanical alignment error; and before image stitching, the coordinate transformation correction of the single-frame panoramic image is performed according to the dynamic deviation prediction value in this embodiment to eliminate the image pixel misalignment caused by mechanical deviation, and then feature matching and stitching are performed.

[0057] Accuracy closed-loop feedback optimization quantifies the current calibration accuracy by calculating the overlap of stitched image edges and analyzing the reprojection error of 3D point cloud. If the accuracy is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation perception module and the weight coefficient of the deep learning deviation association model are adjusted in this embodiment.

[0058] Specifically, initial calibration is first performed under standard environmental conditions, such as a vibration-free laboratory environment with temperature controlled at 25℃ and humidity maintained at 50%. The Zhang Zhengyou calibration method is used to acquire the camera's intrinsic parameters, thus obtaining the camera's focal length. , Principal point coordinates and distortion coefficient Simultaneously, a laser positioning device is used to determine the initial rotation center of the turntable. These parameters are used as the initial benchmark for the deviation model, providing a basic reference for subsequent dynamic deviation calculations and avoiding the cumulative effect of initial parameter deviations on subsequent calibration.

[0059] Next, real-time dynamic deviation sensing is performed. The deviation monitoring unit in the dual-modal dynamic deviation sensing module starts working. The integrated laser displacement sensor is the KEYENCE IL-600 model, which has a measurement accuracy of ±0.5μm. It collects the radial position deviation between the turntable rotation axis and the camera optical center in real time. Including x-axis direction deviation Deviation in the y-axis direction The micro inertial measurement unit uses the ADI ADIS16488 model, with a sampling rate set to 100Hz, to capture camera attitude angle deviations. Covering pitch angle deviation Roll angle deviation and yaw angle deviation The environmental correlation unit is equipped with a DS18B20 temperature sensor to collect ambient temperature change data. (Unit: °C), and the vibration data of the turntable collected by the PCB352C33 vibration sensor. (Unit: m / s²), these environmental data will be used as inputs to the environmental impact factor of the deviation correlation model to ensure that the model can take into account the impact of the external environment on the deviation.

[0060] Subsequently, adaptive bias modeling is performed, constructing a deep learning bias correlation model based on a temporal convolutional network (TCN). The model's input layer receives normalized positional biases. Attitude angle deviation Ambient temperature change data Vibration data of turntable operation The normalization formula is as follows:

[0061] Position deviation normalization: In the formula This is the minimum measured value of the positional deviation. The maximum measured value of the positional deviation was obtained through statistical analysis of multiple previous experiments, for example... , This process normalizes the positional deviation to the [0,1] interval, making it consistent with the dimensions of other normalized data.

[0062] Attitude angle deviation normalization: In the formula This is the minimum measured value of the attitude angle deviation. The maximum measured value of the attitude angle deviation, for example , After normalization, the attitude angle deviation is in the range of [0,1].

[0063] Normalization of ambient temperature change data: In the formula This is the lowest measured value of the ambient temperature. The highest measured value of ambient temperature, for example , After normalization, the temperature data are in the range [0,1].

[0064] Vibration data normalization: In the formula This is the minimum measured value of vibration acceleration. The maximum measured value of vibration acceleration, for example , After normalization, the vibration data fall within the [0,1] interval.

[0065] The model's hidden layers consist of three convolutional layers and two pooling layers, with a 3×3 kernel size and ReLU activation function. Temporal convolution captures the variation of bias over time. The output layer outputs the predicted dynamic bias values. Including the prediction of radial position deviation and predicted attitude angle deviation The prediction formula is In the formula This is the output layer weight matrix. For output layer bias terms, This is the feature mapping function for the hidden layer. A model iteration mechanism is also set, automatically updating the model parameters with newly collected bias data at preset intervals (e.g., a preset interval of 300 seconds). Each update uses stochastic gradient descent to optimize the loss function. In the formula To update the number of samples in the dataset, for The actual measured deviation value at each moment is used to ensure that the model can adapt to the dynamic changes in the deviation variation pattern through iterative updates.

[0066] Subsequently, two-dimensional real-time compensation is implemented. In terms of mechanical control compensation, the radial position deviation in the dynamic deviation prediction value is... The signals are converted into turntable control commands, which are transmitted to the Panasonic A6 series servo motors via the RS485 communication protocol. The servo motors then drive the turntable's fine-tuning mechanism to adjust the turntable's rotation center in real time, reducing the alignment deviation between the rotation center and the camera's optical center; this also reduces the predicted attitude angle deviation. The commands are converted into camera mounting instructions and transmitted to the stepper motor on the camera mounting bracket (for example, using a Sanyo PK series stepper motor with a reduction ratio of 1:100). The stepper motor drives the bracket to rotate, adjusts the camera mounting angle, corrects the alignment error at the mechanical motion level, and reduces the impact of mechanical deviation on the measurement data.

[0067] In terms of image processing compensation, before panoramic image stitching, coordinate transformation is performed on the single-frame panoramic image based on the predicted dynamic deviation value. The translation correction formula is as follows: , In the formula These are the original coordinates of the image pixels. These are the pixel coordinates after translation correction. The camera pixel scale factor (unit: pixels / μm) is used to convert the length units of radial position deviation to pixel units, ensuring dimensional consistency. pixels / μm; rotation correction is achieved using a rotation matrix, the formula is as follows: In the formula These are the rotated and corrected pixel coordinates. To predict yaw angle deviation (unit: radians; if the original value is in degrees, it needs to be converted to radians first; the conversion formula is as follows): After coordinate transformation correction, the SIFT algorithm is used to perform feature matching on the single-frame panoramic image, and then the Poisson fusion algorithm is used to complete the image stitching, ensuring the geometric consistency of the panoramic image and avoiding stitching misalignment.

[0068] Finally, a closed-loop feedback optimization of accuracy is performed. The alignment accuracy is quantified by calculating the edge overlap of the stitched images, using the mean square error formula. In the formula This represents the number of pixels in the overlapping area of ​​the stitched edges. Let be the gray value of the j-th pixel in the overlapping region of the first image. Let MSE be the grayscale value of the j-th pixel in the overlapping region of the second image. A smaller MSE value indicates a higher degree of edge overlap and better alignment accuracy. Simultaneously, the accuracy is quantified through reprojection error analysis of the 3D point cloud. The reprojection error formula is: In the formula These are the actual pixel coordinates of the 3D point cloud projected onto the image. The theoretical projected pixel coordinates are calculated using camera intrinsic and extrinsic parameters; the smaller the error value, the higher the accuracy. If the current calibration accuracy obtained by quantization is lower than a preset threshold, such as MSE greater than 50 or reprojection error greater than 1.2 pixels, the sampling frequency of the dual-modal dynamic deviation sensing module is adjusted, for example, from 10Hz to 20Hz, to acquire deviation data more frequently; or the weight coefficients of the deep learning deviation correlation model are adjusted, for example, to enhance the calculation proportion of the impact of ambient temperature changes on deviation, adjusting the temperature-related weight coefficient from 0.3 to 0.5. Through feedback optimization, the calibration accuracy is continuously improved, ensuring stable data quality during long-term measurement.

[0069] This implementation method can effectively solve the problem that traditional static calibration cannot correct dynamic deviations in real time. Through dynamic sensing, modeling and compensation, it reduces splicing misalignment and point cloud distortion caused by the accumulation of deviations, thereby improving the accuracy of measurement data.

[0070] In this embodiment, the dual-modal dynamic deviation sensing module includes:

[0071] The deviation monitoring unit integrates a laser displacement sensor and a micro inertial measurement unit. In this embodiment, the laser displacement sensor is used to collect the radial position deviation between the turntable rotation axis and the camera optical center in real time. In this embodiment, the micro inertial measurement unit is used to capture the camera attitude angle deviation.

[0072] The environmental correlation unit is used to add temperature sensors and mechanical vibration sensors to collect ambient temperature change data and turntable operation vibration data in this embodiment, and to provide environmental impact factor input for the deviation correlation model in this embodiment.

[0073] Specifically, the dual-modal dynamic deviation sensing module includes a deviation monitoring unit and an environmental correlation unit. The deviation monitoring unit integrates a laser displacement sensor and a micro inertial measurement unit. The laser displacement sensor is a KEYENCE IL-600 model, with a measurement range of 0 to 300 mm and a resolution of 0.1 μm. It collects the radial position deviation between the turntable's rotation axis and the corresponding marked point of the camera's optical center in real time by illuminating the turntable's rotation axis with a laser beam. For example, if a reflective target is attached to the turntable's rotation axis and a reference point is set directly below the camera's optical center, the laser displacement sensor is installed on the side of the turntable and collects position deviation data in the x-axis and y-axis directions every 10 milliseconds to ensure timely capture of dynamic changes in radial deviation.

[0074] The micro inertial measurement unit (IMU) uses the ADI ADIS16488 model, which includes a three-axis gyroscope, a three-axis accelerometer, and a three-axis magnetometer. The measurement range is ±450° / s for the gyroscope and ±10g for the accelerometer. The sampling rate can be set to 100Hz. It is fixed to the camera housing and rigidly connected to the camera to capture the camera's attitude angle deviations in real time, including pitch, roll, and yaw angle deviations. For example, when the turntable shakes slightly during operation, the micro inertial measurement unit can quickly detect the change in the camera's attitude angle and output the corresponding deviation value, providing attitude data support for subsequent deviation compensation.

[0075] The environmental correlation unit is equipped with a temperature sensor and a mechanical vibration sensor. The temperature sensor is a DS18B20 model, which has a measurement range of -55℃ to 125℃ and an accuracy of ±0.5℃. It is installed near the turntable motor and the camera lens to collect ambient temperature change data in the turntable motor area and the camera working area, respectively. For example, when the ambient temperature rises from 25℃ to 30℃, the sensor can record the temperature change value in real time, reflecting the impact of temperature on the mechanical structure.

[0076] The mechanical vibration sensor uses PCB 352C33, with a sensitivity of 100mV / g and a measurement range of ±50g. It is mounted on the turntable base, maintaining rigid contact with the turntable, and collects vibration data during the turntable's operation, including the magnitude and direction of vibration acceleration. For example, when the turntable speed increases from 60r / min to 120r / min, the sensor can capture the change in vibration intensity, providing vibration influence factor input for the deviation correlation model. This allows the model to comprehensively consider the effect of environmental factors on the deviation, improving the accuracy of deviation prediction.

[0077] In this embodiment, adaptive deviation modeling includes:

[0078] A deep learning bias correlation model based on temporal convolutional networks is constructed. The model uses position bias, pose angle bias and environmental data as training samples to dynamically learn the variation law of bias with environmental factors and runtime in this embodiment, and outputs the dynamic bias prediction value in this embodiment in real time.

[0079] A model iteration mechanism is set up so that the parameters of the deviation correlation model in this embodiment are automatically updated with newly collected deviation data at preset intervals.

[0080] Specifically, adaptive bias modeling first constructs a deep learning bias correlation model based on a temporal convolutional network. The model structure consists of an input layer, hidden layers, and an output layer. The input layer receives four types of data, namely, normalized positional biases. Attitude angle deviation Ambient temperature change data Vibration data of turntable operation Normalization ensures that all data units are consistent and fall within the [0,1] range. The specific normalization formula is the same as that in the implementation of weight 1, and will not be repeated here.

[0081] The hidden layers consist of three temporal convolutional layers and two pooling layers. The first convolutional layer has 64 kernels, a kernel size of 3×3, and a stride of 1. Zero padding is used to ensure that the feature map size remains unchanged. The activation function is ReLU, used to extract local temporal features of the data. The first pooling layer uses max pooling with a kernel size of 2×2 and a stride of 2 to reduce the dimension of the feature map and improve the model's computation speed. The second convolutional layer has 128 kernels, with the other parameters being the same as the first layer, to further extract more complex temporal features. The parameters of the second pooling layer are the same as the first layer. The third convolutional layer has 256 kernels, with the parameters being the same as the first two convolutional layers, to delve deeper into the correlation between bias, environmental factors, and runtime.

[0082] The output layer is a fully connected layer with a dimension of 2, corresponding to the radial position deviation in the dynamic deviation prediction values. and attitude angle deviation The output value is converted into an actual physical quantity through inverse normalization. An example of the inverse normalization formula is the inverse normalization of radial position deviation: In the formula The normalized prediction value of the output layer is denormalized to obtain the actual radial position deviation (unit: μm). The denormalization method of attitude angle deviation, temperature and vibration data is similar to ensure that the output deviation prediction value conforms to the actual physical meaning.

[0083] During model training, 1000 sets of bias and environmental data collected in the early stage were used as the training set. Each set of data contained continuous sample values ​​with a time series length of 50. The loss function was minimized using the Adam optimizer, with the loss function being the mean squared error, as shown in the above embodiment. Simultaneously, a model iteration mechanism is set up to automatically update parameters at preset intervals. For example, the preset interval is set to 300 seconds. Each update selects 200 new samples from the real-time collected data and uses incremental training to update the model's weights and biases. This avoids a decrease in prediction accuracy due to changes in the pattern of bias changes, ensuring that the model can dynamically learn the pattern of bias changes with environmental factors and runtime, thereby improving the real-time performance and accuracy of bias prediction.

[0084] In this embodiment, the mechanical control compensation subunit in the two-dimensional real-time compensation includes:

[0085] The radial position deviation output by the dynamic deviation prediction value in this embodiment is converted into a turntable control command, and the rotation center of the turntable in this embodiment is adjusted in real time by the servo motor.

[0086] In this embodiment, the attitude angle deviation output by the dynamic deviation prediction value is converted into a camera mounting command. The angle of the camera mounting bracket in this embodiment is adjusted in real time by the servo motor to correct the alignment error at the mechanical motion level.

[0087] Specifically, the mechanical control compensation subunit in the two-dimensional real-time compensation mainly realizes the real-time adjustment of the turntable rotation center and the camera mounting bracket angle. First, the radial position deviation in the dynamic deviation prediction value output by the adaptive deviation model is... The conversion process into turntable control commands requires consideration of the servo motor's control precision and the turntable's fine-tuning mechanism's transmission ratio. For example, if the turntable's fine-tuning mechanism has a transmission ratio of 1:500, meaning the turntable's rotation center moves 500μm for every one revolution of the servo motor, then the radial position deviation needs to be considered. Then the angle that the servo motor needs to rotate is the transmission ratio. In the formula For example, the radius of the servo motor output shaft. Calculations yielded Converting this angle into the number of pulses for the servo motor (e.g., 10,000 pulses per motor revolution), the control pulse count is obtained as follows: The result is rounded down to 6 pulses, which are used to generate the corresponding turntable control command.

[0088] Control commands are transmitted to the Panasonic A6 series servo motor via RS485 communication protocol. After receiving the commands, the servo motor drives the fine-tuning mechanism of the turntable. The fine-tuning mechanism adopts a ball screw structure, which has high-precision transmission characteristics and can convert the rotational motion of the motor into the linear motion of the turntable's rotation center. It adjusts the position of the turntable's rotation center in the x and y axes in real time, thereby reducing the radial alignment deviation between the turntable's rotation axis and the camera's optical center and correcting the radial alignment error at the mechanical motion level.

[0089] For the adjustment of attitude angle deviation, the attitude angle deviation in the dynamic deviation prediction value is... (including pitch angle deviation) Roll angle deviation The conversion process involves adjusting the camera mounting bracket and the stepper motor's step angle. For example, using a Sanyo PK series stepper motor with a step angle of 1.8° and a 1:100 reduction gear, the actual adjustment angle accuracy is... If the pitch angle deviation is predicted... Then the stepper motor needs to rotate by the following angle: The corresponding number of pulses is the number of pulses. Rounded to 6 pulses, the camera installation command is generated.

[0090] The command is transmitted to the stepper motor on the camera mounting bracket. The stepper motor drives the rotation mechanism of the bracket to adjust the mounting angle of the camera in the pitch and roll directions respectively, so that the camera attitude angle is restored to the ideal state, corrects the attitude alignment error at the mechanical motion level, reduces the image acquisition offset problem caused by camera attitude deviation, and provides more accurate raw image data for subsequent image stitching.

[0091] In this embodiment, the image processing compensation subunit in the two-dimensional real-time compensation includes:

[0092] Before performing panoramic image stitching, coordinate transformation is performed on a single frame panoramic image based on the dynamic deviation prediction value in this embodiment. In this embodiment, the coordinate transformation includes translation correction and rotation correction to eliminate image pixel misalignment caused by mechanical deviation in this embodiment.

[0093] After performing coordinate transformation correction in this embodiment, feature matching and stitching are performed on the single-frame panoramic image to ensure the geometric consistency of the panoramic image.

[0094] Specifically, the image processing compensation subunit in the two-dimensional real-time compensation mainly performs coordinate transformation correction on the single-frame panoramic image before panoramic image stitching to eliminate image pixel misalignment caused by mechanical deviation. First, it obtains the dynamic deviation prediction value output by adaptive deviation modeling, including the predicted radial position deviation. (x-axis direction) y-axis direction ) and predicted attitude angle deviation (Mainly yaw angle deviation) ).

[0095] When performing translation correction, the radial position deviation is converted into a pixel offset based on the correspondence between the radial position deviation and image pixels, and then the translation correction formula is applied. , In the formula These are the original coordinates (in pixels) of the pixels in a single frame of the panoramic image. These are the pixel coordinates after translation correction (unit: pixels). The camera pixel scale factor (unit: pixels / μm) is used to convert the length unit (μm) of radial position deviation into pixel units, ensuring dimensional consistency, such as camera focal length. Pixel size Then the pixel scale factor Pixels / mm = 1.333 pixels / μm, if Then the pixel offset is Pixels, rounded to 7 pixels, are substituted into the formula to obtain the translated x-axis coordinates. .

[0096] After translation correction, rotation correction is performed. To address image rotation misalignment caused by yaw angle deviation, a rotation matrix is ​​used for correction. The formula is as follows: In the formula These are the rotated and corrected pixel coordinates (unit: pixels). To predict yaw angle deviation (unit: radians; if the original value is degrees, it needs to be determined first), (converted to radians) These are the coordinates of the camera's principal point (in pixels), which are the coordinates of the image's center. For example... (For 2K resolution images), during rotation correction, the pixel coordinates are first translated to a coordinate system with the principal point as the origin, then rotated and translated back to the original coordinate system to ensure that the rotation center is consistent with the camera's optical center.

[0097] After coordinate transformation correction, feature matching and stitching are performed on the single-frame panoramic images. Feature matching uses the SIFT algorithm to extract key feature points (including position, scale, orientation, etc.) from each image. Matching pairs are selected by calculating the Euclidean distance between feature points, and RANSAC algorithm is used to remove incorrect matching pairs to improve matching accuracy. Image stitching uses the Poisson fusion algorithm to determine the transformation relationship between images based on the matched feature points. The pixel grayscale values ​​of the stitching area are smoothly transitioned to avoid stitching gaps, ensure the geometric consistency of the panoramic image, and enable the stitched panoramic image to accurately reflect the actual scene, reducing image distortion caused by mechanical deviations.

[0098] In this embodiment, the accuracy closed-loop feedback optimization includes:

[0099] By calculating the edge overlap of stitched images, the pixel deviation at the stitching point of adjacent images is detected, and the current calibration accuracy is quantified.

[0100] By analyzing the reprojection error of the 3D point cloud, the correspondence between the 3D point cloud and the image in this embodiment is compared to quantify the current calibration accuracy.

[0101] If the current calibration accuracy is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation perception module will be automatically adjusted; or, the weight coefficients of the deep learning deviation association model will be automatically adjusted.

[0102] Specifically, the accuracy closed-loop feedback optimization first quantifies the current calibration accuracy by calculating the edge overlap of the stitched images. It selects the overlapping area between two adjacent images in the stitched panoramic image, for example, a rectangular area with a width of 200 pixels and a height of 1080 pixels (corresponding to a 2K resolution image), and counts the number of pixels within this area. (here) (), obtain the grayscale value of each pixel in the overlapping region in the first image. and grayscale values ​​in the second image ( The edge overlap degree is calculated using the mean square error formula: The smaller the MSE value, the smaller the grayscale difference between the edges of adjacent images, the higher the edge overlap, and the better the calibration accuracy; if the MSE value is large, it indicates that there is obvious splicing misalignment and insufficient calibration accuracy.

[0103] Simultaneously, the calibration accuracy is quantified through reprojection error analysis of the 3D point cloud. N points (e.g., N = 500) are randomly selected from the 3D point cloud generated by the panoramic measurement system, and the spatial coordinates of each 3D point are obtained. (Unit: mm), based on camera intrinsic parameters (focal length) Principal point coordinates ) and extrinsic parameters (rotation matrix) Translation vector The theoretical projected pixel coordinates of the 3D point on the image are calculated using a camera projection model. The projection formula is Then obtain the actual projected pixel coordinates of the 3D point on the image. (Determined through image feature matching), the reprojection error is calculated using the Euclidean distance formula: Take the average error of all points. As a quantitative indicator, the smaller the average error, the more accurate the correspondence between the 3D point cloud and the image, and the higher the calibration accuracy.

[0104] If the MSE calculated by edge overlap is greater than a preset threshold (e.g., the preset threshold is 50), or the average error obtained by reprojection error analysis is... If the deviation exceeds a preset threshold (e.g., 1.2 pixels), the current calibration accuracy is determined to be below the requirement, necessitating feedback adjustment. Two adjustment methods exist: First, automatically adjusting the sampling frequency of the dual-modal dynamic deviation sensing module, for example, increasing the sampling frequency from 10Hz to 20Hz, allowing the module to acquire deviation and environmental data more frequently, providing denser samples for deviation modeling, and improving the model's ability to capture dynamic changes in deviation. Second, automatically adjusting the weight coefficients of the deep learning deviation correlation model, for example, analyzing the impact of environmental factors on deviation; if temperature change is found to be the main cause of the current deviation increase, then increasing the calculation proportion of the impact of environmental temperature change on deviation, adjusting the weight coefficient of temperature-related features in the model from 0.3 to 0.5, enabling the model to more accurately predict deviations caused by temperature changes. Through feedback optimization, the calibration effect is continuously improved, ensuring that the calibration accuracy always meets the requirements during the measurement process.

[0105] In this embodiment, adjusting the sampling frequency of the bimodal dynamic deviation perception module and the weight coefficients of the deep learning deviation correlation model includes:

[0106] If the calibration accuracy in this embodiment is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation sensing module in this embodiment is increased to acquire deviation data more frequently.

[0107] If the calibration accuracy in this embodiment is lower than the preset threshold, the weight coefficients of the deep learning bias correlation model are adjusted, including the calculation proportion that enhances the impact of environmental temperature changes on the bias.

[0108] Specifically, when the calibration accuracy is determined to be lower than a preset threshold through closed-loop feedback optimization, the sampling frequency of the dual-modal dynamic deviation sensing module is first adjusted. The severity of the current deviation change is analyzed. If the deviation changes rapidly over time, for example, if the radial position deviation changes by more than 5 μm within 10 seconds, it indicates that the existing sampling frequency (e.g., 10 Hz) cannot adequately capture the dynamic changes in deviation. In this case, the sampling frequency is increased, for example, from 10 Hz to 20 Hz, and the sampling interval is shortened from 100 milliseconds to 50 milliseconds. The sampling frequency adjustment is achieved through the module's control chip (e.g., STM32F407). The control chip sends instructions to the laser displacement sensor, micro-inertial measurement unit, temperature sensor, and vibration sensor to modify the sensor's sampling period parameters, causing each sensor to collect data at the new sampling frequency. The collected data is then transmitted more frequently to the adaptive deviation modeling unit, providing the model with denser training samples. This helps the model learn the deviation change patterns more promptly and improves deviation prediction accuracy.

[0109] If the calibration accuracy still fails to meet the standard after adjusting the sampling frequency, or if analysis reveals that the deviation change is mainly driven by specific environmental factors (e.g., large ambient temperature fluctuations, with hourly temperature changes exceeding 5°C, and a significant positive correlation between temperature changes and deviation changes), then the weight coefficients of the deep learning deviation correlation model are adjusted. First, correlation analysis is used to calculate the correlation coefficients between various input features (historical data of position deviation, historical data of attitude angle deviation, temperature change data, and vibration data) and the predicted output deviation value. For example, the correlation coefficient between temperature change data and the predicted radial position deviation value is calculated. ,like (If the value is close to 1, it indicates a strong correlation), which increases the proportion of the calculation of the impact of environmental temperature changes on the deviation.

[0110] The weighting coefficients are adjusted using the gradient ascent method, aiming to reduce the model's prediction error. This involves adjusting the weighting coefficients corresponding to temperature-related features in the model; for example, adjusting the original weighting coefficients of the temperature features. Adjusted to At the same time, the weight coefficients of other less correlated features should be appropriately reduced, for example, the weight coefficient of vibration data should be reduced. Adjusted to Ensure that the sum of all feature weight coefficients in the model remains 1 (achieved through normalization). In the formula The adjusted feature weights, (These are the normalized weights). After adjusting the weight coefficients, the model can focus more on the impact of temperature changes on the deviation, more accurately predict the dynamic deviation caused by temperature fluctuations, and thus improve the effect of subsequent two-dimensional real-time compensation, restoring the calibration accuracy to above the preset threshold.

[0111] In this embodiment, the initialization calibration includes:

[0112] Under standard environmental conditions, the camera's intrinsic parameters and the initial rotation center of the turntable are obtained through traditional static calibration methods.

[0113] The acquired camera intrinsic parameters and the initial rotation center of the turntable are used as the initial benchmark for the deep learning bias correlation model.

[0114] Specifically, the initial calibration is carried out under standard environmental conditions, which must meet the requirements of stable temperature, suitable humidity and no obvious vibration. For example, the temperature is controlled at 25℃±1℃ and the humidity is maintained at 50%±5%. At the same time, the panoramic camera and the turntable moving measurement system are placed on a shockproof platform to avoid external vibration from interfering with the calibration results and to provide a stable basic environment for calibration.

[0115] Traditional static calibration methods were used to obtain camera intrinsic parameters. Zhang Zhengyou's calibration method was selected, and a checkerboard calibration board (e.g., a checkerboard size of 10×8, with each square having a side length of 20mm) was prepared. The calibration board was fixed on a high-precision translation stage, which could precisely control the position of the calibration board in three-dimensional space. 15 to 20 images of the calibration board were acquired under different orientations, such as adjusting the distance between the calibration board and the camera (from 500mm to 1500mm, in 100mm intervals), the pitch angle of the calibration board (from -15° to 15°, in 5° intervals), and the roll angle (from -15° to 15°, in 5° intervals), to ensure that the acquired images covered the entire field of view of the camera.

[0116] The acquired calibration board image is input into MATLAB's Camera Calibrator toolbox. The toolbox calculates camera intrinsic parameters, including the focal length along the x-axis, by identifying the checkerboard corners in the image. focal length along the y-axis (Unit: pixels), principal point coordinates (Unit: pixels), and radial distortion coefficients and tangential distortion coefficient For example, to obtain Pixels Pixels Pixels , , , These parameters reflect the camera's optical characteristics and provide a basis for subsequent image coordinate transformations.

[0117] To determine the initial rotation center of the turntable, a laser positioning device (e.g., Keyence LK-G80 model, with a measurement accuracy of ±1μm) is used. The laser positioning device is fixed on a tripod, and the laser beam direction is adjusted so that it perpendicularly illuminates the rotating surface of the turntable. The turntable is then started and rotated at a low, uniform speed (e.g., 10 r / min). The laser positioning device collects the position data of the turntable's rotating surface in real time. Since the rotation center position is fixed during rotation, the collected position data will form a circular trajectory. By fitting the center of this circular trajectory, the initial rotation center of the turntable is obtained. (Unit: mm), for example mm.

[0118] The acquired camera intrinsic parameters and the initial rotation center parameters of the turntable are stored in the system's database as the initial benchmark for the deep learning bias correlation model. Subsequent dynamic bias calculations of the model are all based on this initial benchmark to ensure the accuracy of the bias calculation and avoid adverse effects of initial parameter errors on the calibration results.

[0119] In this embodiment, the method steps include:

[0120] After the panoramic measurement system is started, the real-time dynamic deviation sensing step is executed repeatedly to continuously acquire the latest deviation data and environmental data;

[0121] Next, the adaptive bias modeling step is executed repeatedly to iteratively update the deep learning bias association model and output the dynamic bias value at the current time.

[0122] Next, the two-dimensional real-time compensation steps are executed repeatedly to simultaneously complete the deviation compensation at the mechanical and data levels;

[0123] Next, the accuracy closed-loop feedback optimization step is executed repeatedly. If the calibration accuracy does not meet the standard, the parameters of the sensing, modeling and compensation links are adjusted according to feedback until the measurement is completed.

[0124] Specifically, after the panoramic measurement system is started, it enters a cyclical working mode, first repeatedly executing the real-time dynamic deviation sensing step. The system sets the sensing interval via a timer, for example, triggering a sensing operation every 100 milliseconds. The laser displacement sensor, micro-inertial measurement unit, temperature sensor, and vibration sensor in the dual-modal dynamic deviation sensing module simultaneously collect data. The laser displacement sensor acquires the radial position deviation. Micro inertial measurement unit acquires attitude angle deviation Temperature sensors acquire ambient temperature change data. Vibration sensors acquire vibration data of the turntable during operation. The collected data is transmitted in real time to the system's processor (e.g., NVIDIA Jetson Xavier) via a data bus (e.g., CAN bus) to ensure that the processor can continuously acquire the latest deviation data and environmental data, providing real-time input for subsequent modeling steps.

[0125] Next, the adaptive bias modeling step is executed cyclically. The processor initiates a model update and prediction operation every preset time interval (e.g., 300 seconds). First, the newly acquired bias data and environmental data are preprocessed, including data filtering (using Kalman filtering to eliminate noise; the filtering formula is...). In the formula The filtered data is at time k. Here is the state transition matrix. For the control matrix, To control the input, For Kalman gain, The data is measured at time k and normalized. The preprocessed data is then input into a deep learning bias correlation model, which calculates and outputs the predicted dynamic bias value for the current time step through forward propagation. Meanwhile, using newly collected preprocessed data as training samples, the model's weights and bias parameters are updated using incremental training, and the model is iteratively optimized to ensure that the model can adapt to the dynamic changes in the deviation pattern and improve the accuracy of deviation prediction.

[0126] The system then iteratively executes a two-dimensional real-time compensation step, with the compensation operation and deviation prediction performed simultaneously. After acquiring the dynamic deviation prediction value, the processor immediately generates corresponding mechanical control and image processing instructions. The mechanical control instructions are transmitted to the servo motors and stepper motors to adjust the turntable rotation center and camera mounting bracket angle in real time, correcting mechanical alignment errors. The image processing instructions are transmitted to the image processing unit (e.g., an FPGA chip, Xilinx Zynq-7000). The image processing unit performs coordinate transformation correction on the single-frame panoramic image according to the instructions, then performs feature matching and stitching, simultaneously completing deviation compensation at both the mechanical and data levels. This ensures that every acquired image and generated 3D point cloud undergoes deviation correction, reducing the accumulation of deviations.

[0127] Finally, the accuracy closed-loop feedback optimization step is executed repeatedly. The system initiates an accuracy detection and adjustment operation every preset interval (e.g., 5 minutes). First, the edge overlap degree (MSE) of the stitched image and the average reprojection error of the 3D point cloud are calculated. The calculation results are compared with the preset threshold; if the calibration accuracy meets the standard (MSE≤50 and...), the result is correct. If the error is ≤1.2 pixels, the current sensing frequency and model parameters are maintained. If the calibration accuracy is not up to standard, the sampling frequency of the real-time dynamic deviation sensing module is adjusted (e.g., increased from 10Hz to 20Hz) or the weight coefficients of the deep learning deviation association model are adjusted based on the error cause feedback (e.g., increased weighting for temperature influence). After adjustment, the system re-enters the next cycle until the panoramic measurement task ends. By repeatedly executing the above steps, the system ensures that it maintains high calibration accuracy throughout the entire measurement process, thus improving the quality of the measurement data.

[0128] The present invention also provides a panoramic camera and turntable motion measurement calibration system, applied to any of the above-mentioned panoramic camera and turntable motion measurement calibration methods in this embodiment, characterized in that it includes:

[0129] The dual-modal dynamic deviation sensing module is used to collect in real time the positional deviation between the turntable rotation axis and the camera optical center, the camera attitude angle deviation, and environmental data, including ambient temperature change data and turntable operation vibration data.

[0130] The adaptive deviation modeling unit is used to build a deep learning deviation correlation model based on position deviation, attitude angle deviation, ambient temperature change data and turntable operation vibration data. It dynamically learns the variation law of deviation with environmental factors and running time, and outputs dynamic deviation prediction values ​​in real time.

[0131] The dual-dimensional real-time compensation unit is used to convert the dynamic deviation prediction value into mechanical control commands, and adjust the rotation center of the turntable and the angle of the camera mounting bracket in real time through the servo motor to correct the mechanical alignment error; and before image stitching, it performs coordinate transformation correction on the single-frame panoramic image based on the dynamic deviation prediction value to eliminate the image pixel misalignment caused by mechanical deviation, and then performs feature matching and stitching.

[0132] The accuracy closed-loop feedback unit is used to quantify the current calibration accuracy by calculating the overlap of stitched image edges and analyzing the reprojection error of 3D point cloud. If the accuracy is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation perception module and the weight coefficient of the deep learning deviation association model are adjusted.

[0133] Specifically, the panoramic camera and turntable motion measurement calibration system includes a dual-modal dynamic deviation sensing module, an adaptive deviation modeling unit, a dual-dimensional real-time compensation unit, and a precision closed-loop feedback unit. The dual-modal dynamic deviation sensing module consists of a deviation monitoring unit and an environmental correlation unit. The deviation monitoring unit integrates a laser displacement sensor (KEYENCE IL-600) and a micro-inertial measurement unit (ADI ADIS16488). The laser displacement sensor is mounted on the side of the turntable, and the micro-inertial measurement unit is fixed to the camera housing. The environmental correlation unit includes a temperature sensor (DS18B20) and a mechanical vibration sensor (PCB352C33). The temperature sensor is installed near the turntable motor and the camera lens, respectively, while the vibration sensor is mounted on the turntable base. The module communicates with other units in the system via an RS485 bus, transmitting the collected deviation data and environmental data in real time.

[0134] The adaptive bias modeling unit uses an NVIDIA Jetson Xavier embedded processor as its core, equipped with the TensorFlow deep learning framework, to build and run a deep learning bias correlation model based on a temporal convolutional network. Internally, the unit stores the camera intrinsic parameters obtained from initial calibration and the initial rotation center parameters of the turntable, serving as the initial baseline for the model. It also includes a data preprocessing module to perform data filtering and normalization, ensuring the quality of the input data. The unit receives data from the dual-modal dynamic bias sensing module via the PCIe bus, performs modeling calculations, and sends the predicted dynamic bias values ​​to the dual-dimensional real-time compensation unit. Simultaneously, it receives adjustment commands from the accuracy closed-loop feedback unit to update the model parameters.

[0135] The dual-dimensional real-time compensation unit is divided into a mechanical control compensation subunit and an image processing compensation subunit. The mechanical control compensation subunit includes a Panasonic A6 series servo motor (used to adjust the turntable rotation center) and a Sanyo PK series stepper motor (used to adjust the camera mounting bracket angle). The motor controller receives the dynamic deviation prediction value from the adaptive deviation modeling unit via the CAN bus, converts it into motor control commands, and drives the motor to move. The image processing compensation subunit uses a Xilinx Zynq-7000 FPGA chip. The chip internally implements a coordinate transformation module (performing translation and rotation correction), a feature matching module (SIFT algorithm), and an image stitching module (Poisson fusion algorithm). The FPGA chip receives single-frame panoramic images captured by the camera, processes them, and outputs the stitched panoramic image. At the same time, it transmits the image data to the accuracy closed-loop feedback unit.

[0136] The accuracy closed-loop feedback unit consists of an image analysis module and a point cloud analysis module. The image analysis module uses an ARM Cortex-A9 processor and runs an edge overlap calculation algorithm (mean square error formula) to analyze the grayscale differences in overlapping areas of the stitched image. The point cloud analysis module also uses an ARM Cortex-A9 processor and runs a reprojection error calculation algorithm (Euclidean distance formula) to compare the projection correspondence between the 3D point cloud and the image. The two modules comprehensively judge the quantified calibration accuracy results. If the accuracy is lower than a preset threshold, an adjustment command is generated and sent via UART bus to the dual-modal dynamic deviation sensing module (adjusting the sampling frequency) and the adaptive deviation modeling unit (adjusting the model weight coefficients) to achieve accuracy closed-loop optimization. All units of the system work together to ensure that the calibration accuracy of the panoramic camera and the turntable movement measurement meets the requirements of practical applications.

[0137] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0138] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for calibrating a panoramic camera with a turntable for motion measurement, characterized in that, include: Initialize the calibration, obtain the camera's intrinsic parameters and the turntable's initial rotation center, as the initial benchmark for the deep learning bias correlation model; Real-time dynamic deviation sensing: The dual-modal dynamic deviation sensing module collects in real time the radial position deviation between the turntable rotation axis and the camera optical center, the camera attitude angle deviation, and environmental data, including ambient temperature change data and turntable operation vibration data. Adaptive deviation modeling is used to construct a deep learning deviation correlation model based on the radial position deviation, attitude angle deviation, ambient temperature change data and turntable operation vibration data. The model dynamically learns the variation law of deviation with environmental factors and running time, and outputs dynamic deviation prediction values ​​in real time. The dual-dimensional real-time compensation includes a mechanical control compensation step and an image processing compensation step, wherein: the dynamic deviation prediction value is converted into a mechanical control command, and the rotation center of the turntable and the angle of the camera mounting bracket are adjusted in real time by a servo motor to correct the mechanical alignment error; and before image stitching, the coordinate transformation correction of the single-frame panoramic image is performed according to the dynamic deviation prediction value to eliminate the image pixel misalignment caused by mechanical deviation, and then feature matching and stitching are performed. The accuracy closed-loop feedback optimization quantifies the current calibration accuracy by calculating the overlap of stitched image edges and analyzing the reprojection error of 3D point cloud. If the accuracy is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation perception module and the weight coefficient of the deep learning deviation association model are adjusted.

2. The panoramic camera and turntable motion measurement calibration method according to claim 1, characterized in that, The dual-modal dynamic deviation sensing module includes: A deviation monitoring unit is used to integrate a laser displacement sensor and a micro inertial measurement unit. The laser displacement sensor is used to collect the radial position deviation between the turntable rotation axis and the optical center of the camera in real time, and the micro inertial measurement unit is used to capture the camera attitude angle deviation. The environmental correlation unit is used to add temperature sensors and mechanical vibration sensors to collect the environmental temperature change data and the turntable operation vibration data, and to provide environmental impact factor input for the deviation correlation model.

3. The panoramic camera and turntable motion measurement calibration method according to claim 1, characterized in that, The adaptive deviation modeling includes: A deep learning deviation correlation model based on temporal convolutional network is constructed. The model uses radial position deviation, pose angle deviation and environmental data as training samples to dynamically learn the variation law of the deviation with environmental factors and runtime, and outputs the predicted value of the dynamic deviation in real time. A model iteration mechanism is set up so that the parameters of the deviation correlation model are automatically updated with newly collected deviation data at preset intervals.

4. The panoramic camera and turntable motion measurement calibration method according to claim 1, characterized in that, The mechanical control compensation steps in the dual-dimensional real-time compensation are as follows: The radial position deviation output by the dynamic deviation prediction value is converted into a turntable control command, and the rotation center of the turntable is adjusted in real time by a servo motor. The attitude angle deviation output by the dynamic deviation prediction value is converted into a camera mounting command, and the angle of the camera mounting bracket is adjusted in real time by the servo motor to correct the alignment error at the mechanical motion level.

5. The panoramic camera and turntable motion measurement calibration method according to claim 1, characterized in that, The image processing compensation steps in the dual-dimensional real-time compensation are as follows: Before performing panoramic image stitching, coordinate transformation is performed on a single frame panoramic image based on the dynamic deviation prediction value. The coordinate transformation includes translation correction and rotation correction to eliminate image pixel misalignment caused by the mechanical deviation. After performing the coordinate transformation correction, feature matching and stitching are performed on the single-frame panoramic image to ensure the geometric consistency of the panoramic image.

6. The panoramic camera and turntable motion measurement calibration method according to claim 1, characterized in that, The accuracy closed-loop feedback optimization includes: By calculating the edge overlap of stitched images, the pixel deviation at the stitching point of adjacent images is detected, and the current calibration accuracy is quantified. By analyzing the reprojection error of the 3D point cloud, the correspondence between the 3D point cloud and the image is compared to quantify the current calibration accuracy. If the current calibration accuracy is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation perception module will be automatically adjusted; or, the weight coefficients of the deep learning deviation association model will be automatically adjusted.

7. The panoramic camera and turntable motion measurement calibration method according to claim 6, characterized in that, Adjust the sampling frequency of the bimodal dynamic deviation perception module and the weight coefficients of the deep learning deviation correlation model, including: If the calibration accuracy is lower than a preset threshold, the sampling frequency of the dual-modal dynamic deviation sensing module is increased to acquire deviation data more frequently. If the calibration accuracy is lower than a preset threshold, the weight coefficients of the deep learning bias correlation model are adjusted, including adjusting the calculation proportion that enhances the impact of environmental temperature changes on the bias.

8. The panoramic camera and turntable motion measurement calibration method according to claim 1, characterized in that, The initialization calibration includes: Under standard environmental conditions, the camera's intrinsic parameters and the initial rotation center of the turntable are obtained through traditional static calibration methods. The acquired camera intrinsic parameters and the initial rotation center of the turntable are used as the initial benchmark for the deep learning bias correlation model.

9. The panoramic camera and turntable motion measurement calibration method according to claim 1, characterized in that, The method includes: After the panoramic measurement system is started, the real-time dynamic deviation sensing step is executed repeatedly to continuously acquire the latest deviation data and environmental data; Next, the adaptive bias modeling step is executed repeatedly to iteratively update the deep learning bias association model and output the dynamic bias value at the current time. Next, the two-dimensional real-time compensation steps are executed repeatedly to simultaneously complete the deviation compensation at the mechanical and data levels; Next, the accuracy closed-loop feedback optimization step is executed repeatedly. If the calibration accuracy does not meet the standard, the parameters of the sensing, modeling and compensation links are adjusted according to feedback until the measurement is completed.

10. A panoramic camera and turntable motion measurement calibration system, applied to the panoramic camera and turntable motion measurement calibration method according to any one of claims 1-9, characterized in that, include: The dual-modal dynamic deviation sensing module is used to collect in real time the radial position deviation between the turntable rotation axis and the camera optical center, the camera attitude angle deviation, and environmental data, including ambient temperature change data and turntable operation vibration data. The adaptive deviation modeling unit is used to build a deep learning deviation correlation model based on radial position deviation, attitude angle deviation, ambient temperature change data and turntable operation vibration data. It dynamically learns the variation law of deviation with environmental factors and running time, and outputs dynamic deviation prediction values ​​in real time. The dual-dimensional real-time compensation unit is used to convert the dynamic deviation prediction value into mechanical control commands, and adjust the rotation center of the turntable and the angle of the camera mounting bracket in real time through the servo motor to correct the mechanical alignment error; and before image stitching, it performs coordinate transformation correction on the single-frame panoramic image based on the dynamic deviation prediction value to eliminate the image pixel misalignment caused by mechanical deviation, and then performs feature matching and stitching. The accuracy closed-loop feedback unit is used to quantify the current calibration accuracy by calculating the overlap of stitched image edges and analyzing the reprojection error of 3D point cloud. If the accuracy is lower than the preset threshold, the sampling frequency of the dual-modal dynamic deviation perception module and the weight coefficient of the deep learning deviation association model are adjusted.

Images