An image center offset correction system based on image processing

Abstract

The application discloses an image center offset correction system based on image processing, comprising the following modules: collecting calibration images under a reference magnification and recognizing the center of a calibration plate to generate image center reference coordinates; collecting calibration images under multiple magnifications, calculating center offset samples caused by the change of the magnification of a cloud platform lens, and constructing an offset sample table; acquiring cloud platform mechanical error and lens assembly deviation data to generate mechanical error calibration features; inputting the magnification data, the offset sample table and the mechanical error calibration features into an improved multiple self-adaptive regression spline model to generate a horizontal offset correction model and a vertical offset correction model; generating offset correction data according to the current magnification, and compensating for the pointing deviation of the cloud platform lens corresponding to the target center coordinates. The application is based on image calibration and magnification mechanical coupling modeling technology, and realizes dynamic calibration of the image center offset caused by the deviation of the cloud platform lens.

Description

Technical Field

[0001] This invention relates to the field of gimbal camera calibration and lens magnification calibration technology, specifically to an image center offset correction system based on image processing. Background Technology

[0002] Pan-tilt cameras are widely used in video surveillance, target tracking, video radar, edge vision analysis, and long-range target localization. These cameras typically require magnification adjustment based on target distance and monitoring range. During continuous magnification changes, lens assembly movement, optical axis deviation, rotation axis concentricity error, lens installation deviation, and pan-tilt clearance can all contribute to horizontal and vertical image center shifts. Higher magnification amplifies angular and spatial positioning errors per unit pixel, leading to cumulative image center shifts during long-range detection, pan-tilt linkage, and target coordinate conversion. When the backend target detection module still calculates target coordinates based on the uncorrected image center, a chain reaction of errors occurs in the target bounding box center point, pixel coordinates, angle conversion results, and latitude / longitude calculation results, ultimately affecting target tracking stability and positioning accuracy.

[0003] Existing solutions mostly employ manual calibration, fixed-magnification lookup tables, single linear compensation, or global polynomial fitting for correction. Manual calibration is labor-intensive, fixed-magnification lookup tables struggle to cover non-integer magnifications, linear compensation fails to accommodate nonlinear drift at high magnifications, and global polynomial fitting is insufficient to represent abrupt changes and mechanical error coupling within local magnification ranges. Furthermore, PTZ cameras from different brands, batches, and years of use exhibit significant differences in mechanical structure errors and lens assembly deviations. Simply relying on image offset data to establish a unified model cannot adequately account for the complex states of gradual shifts at low magnifications, rapid changes at medium magnifications, and amplified errors at high magnifications.

[0004] Therefore, how to provide an image center offset correction system based on image processing is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an image center offset correction system based on image processing. This invention fully utilizes image center benchmark calibration, multi-magnification calibration image acquisition, construction of gimbal lens mechanical error features, and an improved multivariate adaptive regression spline model. It describes in detail the system architecture for quantifying the image center offset caused by lens assembly deviation, optical axis deviation, gimbal rotation clearance, and gimbal pitch clearance during magnification changes by using the center identification results in the calibration image. It also performs model-based calibration and coordinate compensation of the image center offset. This system has the advantages of clear calibration object, strong magnification adaptability, high offset calibration accuracy, and convenient integration of edge algorithm boxes.

[0006] An image center offset correction system based on image processing according to an embodiment of the present invention includes: The benchmark calibration module acquires calibration images of the PTZ camera at a benchmark magnification, performs calibration board center identification and image center benchmark calculation on the calibration images, and generates image center benchmark coordinates. The multi-magnification sample generation module controls the PTZ camera to acquire calibration images at multiple calibration magnifications. It calculates the center offset sample caused by the PTZ lens magnification change based on the change in the calibration plate center relative to the image center reference coordinate, and generates an offset sample table. The mechanical error feature construction module acquires mechanical structure error data, lens assembly deviation data, and magnification change data of the PTZ camera, and performs normalization, orientation encoding, and feature combination processing on the above data to generate mechanical error calibration features. The improved model building module inputs the magnification data, offset sample table and mechanical error calibration features into the improved multivariate adaptive regression spline model. After magnification segment node correction, mechanical error interaction term construction and horizontal and vertical dual-channel fitting processing, a horizontal offset correction model and a vertical offset correction model are generated. The current magnification offset calculation module obtains the current magnification value and the direction of magnification change, and combines the mechanical error calibration characteristics to call the horizontal offset correction model and the vertical offset correction model to generate the current magnification offset correction data; The coordinate compensation module performs horizontal and vertical compensation processing on the gimbal lens pointing deviation corresponding to the target center coordinates based on the current magnification offset correction data, and generates the compensated target center coordinates. The compensation verification feedback module performs offset residual detection and compensation result verification processing on the target center coordinates after compensation, generates compensation verification results, and updates the offset sample table and the improved multivariate adaptive regression spline model based on the compensation verification results.

[0007] Optionally, the reference calibration module specifically includes: The reference magnification setting unit acquires the initial magnification, image resolution, and calibration board position parameters of the PTZ camera, sets the initial magnification as the reference magnification, and generates reference acquisition control data. The reference image acquisition unit controls the pan-tilt camera to acquire images of the calibration board based on the reference acquisition control data, thereby obtaining the reference calibration image. The calibration board center recognition unit performs grayscale conversion, edge enhancement, and calibration board corner point recognition processing on the reference calibration image to obtain the actual coordinates of the calibration board center. The image center calculation unit calculates the geometric center coordinates of the image based on the image resolution, and uses the geometric center coordinates of the image as the reference coordinates of the image center; The reference data generation unit associates and stores the reference magnification, reference calibration image, actual coordinates of the calibration board center, and reference coordinates of the image center to generate reference calibration data.

[0008] Optionally, the multi-fold sample generation module specifically includes: The magnification sequence generation unit obtains the minimum magnification, maximum magnification, and magnification sampling interval of the PTZ camera, and discretizes the magnification range to obtain multiple calibrated magnifications; The multi-magnification image acquisition unit sequentially controls the PTZ camera to acquire calibration images according to multiple calibration magnifications, and records the magnification value, acquisition order and magnification change direction of the calibration images to obtain a set of multi-magnification calibration images; The actual center identification unit performs calibration board center feature identification on the calibration images in the multi-magnification calibration image set to obtain the actual coordinates of the calibration board center corresponding to each calibration magnification. The offset difference calculation unit performs horizontal and vertical difference calculations between the actual coordinates of the calibration board center and the reference coordinates of the image center to obtain multi-magnification horizontal offset samples and multi-magnification vertical offset samples generated by the gimbal lens magnification change. The offset sample table construction unit arranges the calibration magnification, acquisition order, magnification change direction, actual coordinates of the calibration plate center, multi-magnification horizontal offset samples, and multi-magnification vertical offset samples to obtain the offset sample table.

[0009] Optionally, the mechanical error feature construction module specifically includes: The mechanical parameter acquisition unit acquires the concentricity error of the rotation axis of the pan-tilt camera, the horizontal rotation clearance of the pan-tilt, and the vertical pitch clearance of the pan-tilt, thus obtaining mechanical structure error data. The lens deviation acquisition unit acquires the lens installation deviation, lens optical axis deviation, and zoom lens group movement deviation of the PTZ camera to obtain lens assembly deviation data. The magnification direction generation unit obtains the magnification at the current sampling time and the magnification at the previous sampling time, performs a difference judgment between the two, and obtains the magnification increase state, magnification decrease state, or magnification hold state. The error scale unification unit performs dimensional unification and numerical range compression processing on mechanical structure error data and lens assembly deviation data to obtain normalized error data. The calibration feature combination unit combines normalized error data, magnification direction encoded data, and corresponding magnification data to obtain mechanical error calibration features that include magnification, rotation axis error, lens assembly deviation, optical axis deviation, rotation clearance, pitch clearance, and magnification direction.

[0010] Optionally, the improved model building module specifically includes: The training sample matrix generation unit acquires the magnification data, offset sample table, and mechanical error calibration features. It takes the magnification data and mechanical error calibration features as input columns and the multi-magnification horizontal offset samples and multi-magnification vertical offset samples as output columns to obtain the model training sample matrix. The basic spline modeling unit generates candidate segmentation nodes based on the scalar data, multi-scalar horizontal offset samples, and multi-scalar vertical offset samples in the model training sample matrix, and constructs a set of candidate spline basis functions based on the candidate segmentation nodes to obtain the basic multivariate adaptive regression spline model. The improved unit for magnification-mechanical coupling acquires the candidate spline basis function set, mechanical error calibration features, and magnification change direction data. It then performs magnification segmentation node correction, mechanical error interaction term construction, directional branch configuration, and horizontal and vertical channel allocation on the candidate spline basis function set to obtain the improved spline basis function set. The dual-channel model generation unit performs fitting processing on the multi-level horizontal offset samples based on the horizontal channel improved spline basis function set in the improved spline basis function set to obtain the horizontal offset correction model, and performs fitting processing on the multi-level vertical offset samples based on the vertical channel improved spline basis function set to obtain the vertical offset correction model. The model call data generation unit associates the horizontal offset correction model, the vertical offset correction model, the improved spline basis function set, the mechanical error calibration features, and the magnification change direction data to obtain the model call data.

[0011] Optionally, the improved rate-coupling mechanical coupling unit specifically includes: The multiplier segment node correction sub-unit obtains the multiplier segment nodes in the candidate spline basis function set, as well as the horizontal and vertical offset changes corresponding to adjacent multipliers. The multiplier segment nodes are retained, merged, or deleted to obtain the corrected multiplier segment nodes. The mechanical error interaction term construction sub-unit obtains the rotation axis error term, lens assembly deviation term, optical axis deviation term, rotation clearance term, pitch clearance term, and magnification direction term. The above error terms are combined with the corrected magnification segment node to obtain the magnification mechanical coupling interaction term. The horizontal and vertical channel allocation sub-units are used to add the magnification mechanical coupling interaction term to the horizontal offset spline channel and the vertical offset spline channel according to the horizontal offset contribution and the vertical offset contribution, respectively, to obtain the improved spline basis function set of the horizontal channel and the improved spline basis function set of the vertical channel. The scaling direction branching processing subunit configures the branching relationship of the horizontal channel improved spline basis function set and the vertical channel improved spline basis function set according to the scaling increase state, scaling decrease state and scaling hold state, so as to obtain the branch spline relationship corresponding to the scaling change direction; The improved spline output sub-unit combines the improved spline basis function set of the horizontal channel, the improved spline basis function set of the vertical channel, the scaling mechanical coupling interaction term, and the branch spline relationship to obtain the improved spline basis function set.

[0012] Optionally, the current magnification offset calculation module specifically includes: The magnification signal parsing unit acquires the current magnification signal output by the PTZ camera, performs format parsing, value verification, and magnification range matching on the current magnification signal, and obtains the current magnification value. The magnification direction confirmation unit obtains the current magnification value and the magnification at the previous sampling time, performs a difference judgment on the current magnification value and the magnification at the previous sampling time, and obtains the current magnification change direction; The model calling unit obtains the current magnification value, mechanical error calibration characteristics, and current magnification change direction. It inputs the current magnification value, mechanical error calibration characteristics, and current magnification change direction into the horizontal offset correction model and the vertical offset correction model to obtain the horizontal offset model output and the vertical offset model output. The correction output unit generates the current magnification offset correction data based on the horizontal offset model output and the vertical offset model output, and sends the current magnification offset correction data to the coordinate compensation module.

[0013] Optionally, the coordinate compensation module specifically includes: The coordinate input unit acquires the target center coordinates, the image center reference coordinates, and the current magnification offset correction data in the image to be corrected. The deviation direction determination unit reads the current horizontal offset correction amount and the current vertical offset correction amount to determine the horizontal compensation direction and vertical compensation direction corresponding to the gimbal lens pointing deviation; The coordinate compensation unit compensates for the horizontal pointing deviation of the gimbal lens corresponding to the target center coordinates based on the horizontal compensation direction and the current horizontal offset correction amount, and compensates for the vertical pointing deviation of the gimbal lens corresponding to the target center coordinates based on the vertical compensation direction and the current vertical offset correction amount, thus obtaining the compensated target center coordinates. The compensation result output unit associates the compensated target center coordinates with the current magnification value, the current magnification change direction, and the image frame identifier to generate the image center offset correction result.

[0014] Optionally, the compensation verification feedback module specifically includes: The compensation center identification unit performs target center identification or calibration board center identification on the compensated image corresponding to the image center offset correction result to obtain the actual center coordinates after compensation. The residual calculation unit performs horizontal and vertical difference calculations between the compensated actual center coordinates and the image center reference coordinates to obtain the compensation residual data formed by the gimbal lens pointing deviation. The residual judgment unit compares the compensated residual data with a preset residual threshold and generates a compensation verification result. The feedback sample writing unit writes the current multiplier value, the current multiplier change direction, and the compensation residual data into the offset sample table when the compensation residual data exceeds the preset residual threshold, thus obtaining the updated offset sample table. The model update unit triggers an improved multivariate adaptive regression spline model update process based on the updated offset sample table, resulting in updated horizontal offset correction models and vertical offset correction models.

[0015] The beneficial effects of this invention are as follows: This invention determines the reference coordinates of the image center at a reference magnification using a reference calibration module, providing a unified coordinate reference for subsequent multi-magnification center offset calculations and avoiding calibration errors caused by different initial installation states of different PTZ cameras. The multi-magnification sample generation module acquires calibration images at multiple calibration magnifications, identifies the change in the calibration board center relative to the image center reference coordinates, and obtains horizontal and vertical offset samples generated by the PTZ lens magnification changes. This allows the system to record the variation pattern of the PTZ lens center deviation in the image coordinates during the zoom process.

[0016] By incorporating data on rotation axis concentricity error, lens mounting deviation, lens optical axis deviation, zoom lens group movement deviation, gimbal horizontal rotation clearance, gimbal vertical pitch clearance, and magnification change direction through a mechanical error feature construction module, offset correction no longer relies solely on empirical fitting of image coordinates but can instead incorporate calibration modeling based on the gimbal and lens's own structural errors. An improved multivariate adaptive regression spline model is used to jointly construct magnification segment nodes, mechanical error interaction terms, magnification direction branching relationships, and horizontal and vertical dual-channel spline relationships. This allows for the establishment of more refined correction relationships for different magnification ranges, different zoom directions, and different offset directions, reducing correction bias in non-integer magnification and high magnification ranges using a single linear model or fixed lookup table method.

[0017] Through the coordinated processing of the current magnification offset calculation module and the coordinate compensation module, the system can generate horizontal and vertical offset corrections based on the current magnification signal, the direction of magnification change, and mechanical error calibration characteristics. It also performs horizontal and vertical compensation for the gimbal lens pointing deviation corresponding to the target center coordinates, reducing the target center coordinate drift caused by magnification adjustment and improving the stability of coordinate calculation in video monitoring, target tracking, gimbal linkage, and edge positioning scenarios.

[0018] By performing residual detection and verification feedback on the compensated target center coordinates through the compensation verification feedback module, the offset sample table can be updated when the compensation residual exceeds the preset threshold, and the improved multivariate adaptive regression spline model update processing can be triggered. This enables the system to adapt to the changes in offset patterns caused by lens aging, mechanical clearance changes and assembly status changes during long-term operation, thereby improving the continuous calibration capability of the image center offset correction system. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the structure of an image center offset correction system based on image processing proposed in this invention; Figure 2 This is a schematic diagram illustrating the process of benchmark calibration, multi-magnification gimbal lens offset sample generation, and mechanical error calibration feature construction in this invention. Figure 3 This is a schematic diagram illustrating the construction of the improved multivariate adaptive regression spline model, the calculation of the current magnification deviation, and the gimbal lens pointing compensation feedback process in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figures 1-3 An image center offset correction system based on image processing includes the following steps: The benchmark calibration module acquires calibration images of the PTZ camera at a benchmark magnification, performs calibration board center identification and image center benchmark calculation on the calibration images, and generates image center benchmark coordinates. The multi-magnification sample generation module controls the PTZ camera to acquire calibration images at multiple calibration magnifications. It calculates the center offset sample caused by the PTZ lens magnification change based on the change in the calibration plate center relative to the image center reference coordinate, and generates an offset sample table. The mechanical error feature construction module acquires mechanical structure error data, lens assembly deviation data, and magnification change data of the PTZ camera, and performs normalization, orientation encoding, and feature combination processing on the above data to generate mechanical error calibration features. The improved model building module inputs the magnification data, offset sample table and mechanical error calibration features into the improved multivariate adaptive regression spline model. After magnification segment node correction, mechanical error interaction term construction and horizontal and vertical dual-channel fitting processing, a horizontal offset correction model and a vertical offset correction model are generated. The current magnification offset calculation module obtains the current magnification value and the direction of magnification change, and combines the mechanical error calibration characteristics to call the horizontal offset correction model and the vertical offset correction model to generate the current magnification offset correction data; The coordinate compensation module performs horizontal and vertical compensation processing on the gimbal lens pointing deviation corresponding to the target center coordinates based on the current magnification offset correction data, and generates the compensated target center coordinates. The compensation verification feedback module performs offset residual detection and compensation result verification processing on the target center coordinates after compensation, generates compensation verification results, and updates the offset sample table and the improved multivariate adaptive regression spline model based on the compensation verification results.

[0022] In this embodiment, image center offset does not refer to the translational deformation of the image content itself, but rather to the change in the center imaging position of the calibration board relative to the image center reference coordinates caused by the movement of the lens group, lens installation deviation, optical axis deviation, pan-tilt horizontal rotation clearance, and pan-tilt vertical pitch clearance of the pan-tilt camera at different magnification states.

[0023] This change is obtained through the center identification result of the calibration plate in the calibration image, and is used as the measurement result of the center deviation of the gimbal lens in the image coordinates, participating in subsequent offset sample construction, model training, magnification offset calculation and coordinate compensation.

[0024] In this embodiment, the reference calibration module specifically includes: acquiring the initial magnification, image resolution, and calibration board position parameters of the PTZ camera, setting the initial magnification as the reference magnification, and generating reference acquisition control data.

[0025] The reference acquisition and control data includes the reference magnification value, pan-tilt horizontal angle, pan-tilt pitch angle, calibration board position parameters, and image acquisition parameters. The reference calibration module uses this data to control the pan-tilt camera to acquire images from the calibration board, thus obtaining the reference calibration image.

[0026] In this embodiment, the benchmark calibration module also includes stability testing of the calibration environment when performing benchmark calibration.

[0027] Stability testing includes testing the installation status of the PTZ camera, the fixing status of the calibration board, the brightness of the image acquisition, the image clarity, and the integrity of the calibration board.

[0028] The installation status of the PTZ camera is determined by reading the horizontal angle feedback value and the vertical pitch angle feedback value of the PTZ. If the angle feedback value corresponding to multiple consecutive frames of images changes beyond the preset angle threshold, it is determined that there is PTZ jitter in the current acquisition environment, and the reference calibration module suspends the acquisition of reference calibration images.

[0029] The fixed state of the calibration board is determined by identifying the positional changes of the calibration board's edge contour. If the positional changes of the calibration board's edge contour in adjacent frames exceed a preset pixel threshold, the calibration board's position is determined to be unstable.

[0030] Image acquisition brightness is determined by calculating the overall grayscale mean and local grayscale distribution of the reference calibration image; image sharpness is determined by calculating the gradient change in the edge region of the calibration plate; and calibration plate integrity is determined by detecting the number of corner points, the integrity of the circular outline, or the integrity of the cross intersection.

[0031] The benchmark calibration module will only write the acquired calibration image into the benchmark calibration data when the installation status of the PTZ camera, the fixed status of the calibration board, the image brightness, the image clarity, and the integrity of the calibration board all meet the acquisition conditions.

[0032] In this embodiment, the reference calibration module performs image preprocessing on the reference calibration image. The image preprocessing includes grayscale processing, noise suppression processing, edge enhancement processing, brightness equalization processing, and local contrast enhancement processing to obtain a reference preprocessed image.

[0033] The benchmark calibration module identifies the corner points of the calibration board in the benchmark preprocessed image, extracts the outer contour, internal corner points, center pattern or checkerboard intersection points of the calibration board, and calculates the actual coordinates of the center of the calibration board based on the positional relationship of multiple corner points.

[0034] The actual coordinates of the calibration board center are based on the top left corner of the image as the origin, the horizontal pixel direction of the image as the horizontal coordinate direction, and the vertical pixel direction of the image as the vertical coordinate direction.

[0035] For the checkerboard calibration board, the reference calibration module extracts the checkerboard corner point array and calculates the actual coordinates of the calibration board center based on the geometric center of the corner point array; For a circular target, the benchmark calibration module extracts the set of circular edge points and obtains the center coordinates by fitting the circular contour. For a cross-shaped target, the reference calibration module identifies the center lines of the horizontal and vertical line segments and uses the intersection of the two center lines as the actual coordinates of the calibration board center.

[0036] In this embodiment, the reference calibration module simultaneously calculates the geometric center coordinates of the image based on the image resolution and uses the geometric center coordinates of the image as the reference coordinates of the image center.

[0037] When generating the reference coordinates of the image center, the reference calibration module also records the initial deviation between the actual coordinates of the calibration board center and the coordinates of the geometric center of the image as the reference deviation.

[0038] The baseline deviation record includes baseline horizontal deviation, baseline vertical deviation, baseline magnification value, baseline image frame identifier, and baseline acquisition time.

[0039] If there is an initial deviation between the actual coordinates of the calibration plate center and the geometric center coordinates of the image in the benchmark calibration image, the system writes the initial deviation into the benchmark calibration data and deducts it uniformly in the subsequent offset difference calculation. This ensures that the horizontal and vertical offset samples calculated by the multi-magnification sample generation module mainly reflect the additional center offset caused by the gimbal lens magnification change, rather than mixing the initial installation deviation and the magnification change deviation in the same offset sample.

[0040] The benchmark calibration module associates and stores the benchmark magnification, benchmark calibration image, actual coordinates of the calibration board center, benchmark coordinates of the image center, and benchmark deviation records to generate benchmark calibration data. This benchmark calibration data serves as the benchmark input for subsequent multi-magnification offset sample calculations and coordinate compensation.

[0041] In this embodiment, the multi-magnification sample generation module specifically includes: obtaining the minimum magnification, maximum magnification, and magnification sampling interval of the PTZ camera, and discretizing the magnification range between the minimum and maximum magnification to obtain multiple calibration magnifications.

[0042] Multiple calibration magnifications can be arranged in ascending order, or they can be sampled in descending order to record the center offset differences during magnification increases and decreases.

[0043] When generating multiple calibration magnifications, the multi-magnification sample generation module can generate magnification sequences by means of equal interval sampling, non-equal interval sampling, or priority sampling of key magnifications.

[0044] Equal interval sampling is suitable for PTZ cameras with a narrow magnification range and relatively smooth offset changes; non-equal interval sampling is suitable for PTZ cameras with a wide magnification range and significant offset changes in the high magnification range. The key magnification priority sampling method is suitable for application scenarios where PTZ cameras have commonly used operating magnifications.

[0045] For the high magnification range, since slight gimbal lens deviations may be magnified in the image, the multi-magnification sample generation module can automatically increase the sampling density in the high magnification range. For low magnification ranges, if the image center offset between adjacent magnifications is small, the sampling density can be reduced.

[0046] After the magnification sequence is generated, the multi-magnification sample generation module writes the magnification sequence, sampling strategy, sampling interval, and magnification order into the magnification sampling configuration data.

[0047] In this embodiment, the multi-magnification sample generation module controls the PTZ camera to acquire calibration images sequentially according to multiple calibration magnifications, and records the magnification value, acquisition order, acquisition time, PTZ horizontal angle, PTZ pitch angle, magnification change direction and image frame identifier corresponding to each frame of calibration image, thereby obtaining a multi-magnification calibration image set.

[0048] To reduce the impact of jitter in a single frame image, multiple calibration images can be continuously acquired at each calibration magnification. The center identification results of the multiple calibration images are then averaged or filtered by median to obtain the stable center coordinates at the corresponding calibration magnification.

[0049] The multi-magnification sample generation module employs a magnification stabilization waiting mechanism when acquiring calibration images at various calibration magnifications.

[0050] The magnification stabilization waiting mechanism includes reading the magnification control commands from the PTZ camera, the magnification value fed back by the lens, and the changes in image sharpness in consecutive frames; When the lens feedback magnification value reaches the target calibration magnification and the change in image sharpness for multiple consecutive frames is less than a preset threshold, the magnification adjustment is considered complete, and then the calibration image at the corresponding magnification is acquired.

[0051] If there is a difference between the lens feedback magnification value and the target calibration magnification, continue to wait or resend the magnification control command.

[0052] By using a magnification stabilization waiting mechanism, the center recognition error caused by the lens group not yet stabilizing during the zoom process can be reduced.

[0053] In this embodiment, the multi-magnification sample generation module performs calibration board center feature recognition on each frame of calibration image in the multi-magnification calibration image set to obtain the actual coordinates of the calibration board center corresponding to each calibration magnification.

[0054] The calibration board center feature recognition includes grayscale conversion, edge enhancement, contour extraction, corner detection, and center coordinate calculation of the calibration image.

[0055] When the calibration board is a chessboard, the system extracts the chessboard corner point array and determines the actual coordinates of the calibration board center based on the horizontal and vertical mean values ​​of the corner point array. When the calibration plate is a circular target, the system extracts the set of circular edge points and obtains the center coordinates of the circle through contour fitting; When the calibration board is a crosshair target, the system identifies the center lines of the horizontal and vertical line segments and uses the intersection of the two center lines as the actual coordinates of the calibration board center.

[0056] If a frame of the calibration image is missing corner points, has an incomplete circular outline, or has an obscured cross intersection, the system marks that frame as a center recognition failure and selects other valid frames at the same magnification to calculate the center coordinates.

[0057] If multiple valid images exist at the same magnification, the system performs outlier removal and mean calculation on the center coordinates of the multiple frames to obtain the stable actual center coordinates at that magnification.

[0058] In this embodiment, the multi-magnification sample generation module calculates the horizontal difference between the actual coordinates of the calibration plate center and the reference coordinates of the image center for each calibration magnification, and obtains multi-magnification horizontal offset samples. The vertical difference between the actual coordinates of the calibration plate center and the reference coordinates of the image center is calculated for each calibration magnification to obtain multi-magnification vertical offset samples.

[0059] If the reference calibration module has recorded the reference horizontal deviation and the reference vertical deviation, the multi-magnification sample generation module will deduct the reference deviation when calculating the horizontal and vertical offset samples, so that the offset samples more accurately reflect the image center shift caused by the change in the magnification of the gimbal lens.

[0060] The multi-magnification sample generation module arranges the calibration magnification, acquisition order, magnification change direction, actual coordinates of the calibration plate center, multi-magnification horizontal offset samples, and multi-magnification vertical offset samples according to the magnification value to obtain the offset sample table.

[0061] Each row in the offset sample table corresponds to a calibration sample at a calibration magnification. The column fields include magnification value, magnification change direction, actual horizontal coordinate, actual vertical coordinate, horizontal offset sample, vertical offset sample, acquisition status identifier, and center identification confidence.

[0062] The offset sample table is output to the improved model building module and can be continuously updated by the compensation verification feedback module.

[0063] In this embodiment, the mechanical error feature construction module specifically includes: acquiring the concentricity error of the rotation axis of the pan-tilt camera, the horizontal rotation clearance of the pan-tilt, and the vertical pitch clearance of the pan-tilt, to obtain mechanical structure error data; The lens mounting deviation, lens optical axis deviation, and zoom lens group movement deviation of the PTZ camera are obtained to obtain lens assembly deviation data.

[0064] The concentricity error of the rotation axis is obtained from the gimbal rotation axis test data or the equipment factory calibration data. The gimbal horizontal rotation clearance and gimbal vertical pitch clearance are obtained from the gimbal repeat positioning test data. The lens installation deviation, lens optical axis deviation and zoom lens group movement deviation are obtained from the lens assembly calibration data or the lens factory test data.

[0065] When the mechanical error feature construction module performs feature processing on mechanical structure error data and lens assembly deviation data, it converts rotation axis concentricity error, lens installation deviation, lens optical axis deviation, zoom lens group movement deviation, gimbal horizontal rotation clearance, and gimbal vertical pitch clearance into numerical features that can participate in model fitting.

[0066] The concentricity error of the rotation axis reflects the degree of deviation between the rotation center of the gimbal and the optical center of the lens; the lens mounting deviation reflects the assembly deviation between the lens assembly position and the center of the image sensor; the lens optical axis deviation reflects the deviation between the center of the lens optical axis and the geometric center of the image; the zoom lens group movement deviation reflects the center offset trend caused by the movement of the lens group along the optical structure during the magnification change; the gimbal horizontal rotation clearance reflects the return clearance generated by the horizontal rotation mechanism when changing direction; and the gimbal vertical pitch clearance reflects the return clearance generated by the vertical pitch mechanism when changing direction.

[0067] In this embodiment, the mechanical error feature construction module obtains the current sampling time multiplier and the previous sampling time multiplier, performs a difference judgment on the current sampling time multiplier and the previous sampling time multiplier, and obtains the multiplier increase state, multiplier decrease state, or multiplier hold state.

[0068] The magnification increase state corresponds to the lens moving from low magnification to high magnification, the magnification decrease state corresponds to the lens moving from high magnification to low magnification, and the magnification hold state corresponds to the lens magnification not changing.

[0069] Because the zoom mechanism of a lens may have mechanical lag and return gap when the magnification increases or decreases, the same magnification value may correspond to different image center offsets under different directions of change.

[0070] The mechanical error feature construction module writes the magnification direction encoded data into the mechanical error calibration features, enabling the improved multivariate adaptive regression spline model to distinguish the offset patterns under magnification increase path and magnification decrease path.

[0071] In the rate hold state, the system associates the current rate value with the direction of the most recent effective rate change to avoid losing preceding mechanical hysteresis information in the rate hold state.

[0072] In this embodiment, the mechanical error feature construction module performs dimension unification and numerical range compression processing on the mechanical structure error data and lens assembly deviation data to obtain normalized error data.

[0073] The dimensionless processing transforms the data related to angle error, length error, and pixel offset to a unified numerical scale, while the numerical range compression processing maps different error fields to the same numerical range, avoiding an imbalance in the model fitting caused by an excessively large numerical range of a certain error field.

[0074] The mechanical error feature construction module performs symbol encoding on the magnification increase state, magnification decrease state, and magnification hold state to obtain magnification direction encoded data.

[0075] The mechanical error feature construction module combines normalized error data, magnification direction encoded data, and corresponding magnification data to obtain mechanical error calibration features that include magnification, rotation axis error, lens assembly deviation, optical axis deviation, rotation clearance, pitch clearance, and magnification direction.

[0076] The mechanical error calibration features and the offset sample table are correlated according to the magnification value, the acquisition order, and the direction of magnification change, serving as auxiliary inputs for the improved model building module.

[0077] In this embodiment, the improved model construction module specifically includes: acquiring magnification data, offset sample table and mechanical error calibration features, using magnification data and mechanical error calibration features as input columns, and using multi-magnification horizontal offset samples and multi-magnification vertical offset samples as output columns to obtain the model training sample matrix.

[0078] The model training sample matrix is ​​organized with calibration magnification samples as rows and magnification terms, mechanical error terms, lens assembly deviation terms, magnification direction terms, horizontal offset samples, and vertical offset samples as columns.

[0079] The magnification term reflects the optical zoom position, the mechanical error term and the lens assembly deviation term reflect the structural deviation between the gimbal and the lens, the magnification direction term reflects the zoom direction, and the horizontal offset sample and the vertical offset sample are used as the model fitting targets respectively.

[0080] The improved model building module performs field alignment on the offset sample table and mechanical error calibration features when constructing the model training sample matrix.

[0081] Field alignment includes alignment by multiplier value, alignment by acquisition order, and alignment by direction of multiplier change.

[0082] If a certain magnification sample lacks mechanical error calibration features, the system will interpolate and complete the feature based on the mechanical error calibration features of adjacent magnification samples. If the actual center identification result of a certain magnification sample is abnormal, the system reduces the weight of that sample based on the calibration image acquisition status and residual size.

[0083] In this embodiment, the basic spline modeling unit generates candidate segmentation nodes based on the multiplier data, multiplier horizontal offset samples, and multiplier vertical offset samples in the model training sample matrix, and constructs a candidate spline basis function set based on the candidate segmentation nodes to obtain the basic multivariate adaptive regression spline model.

[0084] The basic multivariate adaptive regression spline model uses scalar data as the main segmentation variable and constructs segmented spline basis functions based on the changing trends of horizontal and vertical offset samples within different scalar intervals.

[0085] Candidate segmentation nodes include magnification locations with significant offset changes, boundary locations of magnification sampling intervals, and locations where the offset change of adjacent magnifications exceeds a set amplitude.

[0086] When generating candidate segment nodes for the basic spline modeling unit, the horizontal and vertical offset changes between adjacent ratios are first calculated, and then the candidate segment nodes are determined based on the magnitude of the offset changes.

[0087] If the horizontal or vertical offset change between adjacent magnifications exceeds the set threshold, the corresponding magnification position will be used as a candidate segmentation node. If multiple candidate segment nodes are close to each other, the system retains the node with the larger offset change and merges the other adjacent nodes. If a candidate segment node is generated solely from the single-frame anomaly center identification result, the system deletes the candidate segment node based on the acquisition status identifier and residual distribution.

[0088] After screening, candidate segment nodes form an initial multiplier segment structure, and the basic spline modeling unit constructs a set of candidate spline basis functions based on the initial multiplier segment structure.

[0089] In this embodiment, the improved model building module improves the basic multivariate adaptive regression spline model for the gimbal lens center deviation calibration scenario, forming an improved multivariate adaptive regression spline model.

[0090] Improvements include correction of the magnification segment node, construction of mechanical error interaction terms, configuration of magnification direction branches, and horizontal and vertical dual-channel fitting.

[0091] Multiplier segment node correction refers to obtaining the multiplier segment nodes in the candidate spline basis function set, obtaining the horizontal and vertical offset changes corresponding to adjacent multipliers, and then retaining, merging, or deleting the multiplier segment nodes to obtain the corrected multiplier segment nodes.

[0092] For candidate nodes with small horizontal and vertical offset changes between adjacent magnifications, the system performs a merging process; for candidate nodes with a sudden increase in offset change between adjacent magnifications, the system performs a retention process; for isolated candidate nodes caused by single-frame recognition noise, the system performs a deletion process.

[0093] By correcting the multiplier-based segmented nodes, the model can reduce the impact of noise nodes on the fitted curve.

[0094] In this embodiment, the construction of mechanical error interaction terms refers to obtaining the rotation axis error term, lens assembly deviation term, optical axis deviation term, rotation clearance term, pitch clearance term, and magnification direction term from the mechanical error calibration features, and combining the above error terms with the corrected magnification segment nodes to obtain the magnification mechanical coupling interaction terms.

[0095] The magnification mechanical coupling interaction items include at least the interaction between the magnification item and the rotation axis error item, the interaction between the magnification item and the lens assembly deviation item, the interaction between the magnification item and the optical axis deviation item, the interaction between the magnification item and the horizontal rotation clearance item, the interaction between the magnification item and the vertical pitch clearance item, the interaction between the magnification direction item and the horizontal rotation clearance item, and the interaction between the magnification direction item and the vertical pitch clearance item.

[0096] For the horizontal offset channel, the system increases the weight of the interaction terms between the magnification term and the rotation axis error term, the lens assembly deviation term, and the horizontal rotation clearance term; for the vertical offset channel, the system increases the weight of the interaction terms between the magnification term and the optical axis deviation term, the zoom lens group movement deviation term, and the vertical pitch clearance term.

[0097] By distinguishing the error interaction terms of the horizontal and vertical channels, the model can more accurately reflect the differentiated impact of different gimbal lens errors on the offset in the two directions.

[0098] In this embodiment, horizontal and vertical dual-channel fitting refers to adding the magnification mechanical coupling interaction term to the horizontal offset spline channel and the vertical offset spline channel respectively to obtain the improved spline basis function set for the horizontal channel and the improved spline basis function set for the vertical channel.

[0099] The improved spline basis function set for the horizontal channel is fitted based on the multi-fold horizontal offset samples to obtain the horizontal offset correction model; The improved spline basis function set for the vertical channel is fitted based on the multi-fold vertical offset samples to obtain the vertical offset correction model.

[0100] The horizontal offset correction model and the vertical offset correction model receive the current magnification value, mechanical error calibration characteristics, and magnification change direction, respectively, and output the horizontal offset model output and the vertical offset model output.

[0101] Since the gimbal's horizontal rotation clearance, lens assembly deviation, and vertical pitch clearance have different effects on the horizontal and vertical directions, horizontal and vertical dual-channel fitting can avoid the single model mixing the offset changes in the two directions and improve the directional consistency of the offset correction data.

[0102] In this embodiment, the training data sources for the improved model building module include benchmark calibration data, multi-magnification calibration image sets, offset sample tables, mechanical structure error data, lens assembly deviation data, and mechanical error calibration features.

[0103] The training samples are based on magnification sampling points. Each training sample records the magnification value, the direction of magnification change, mechanical structure error characteristics, lens assembly deviation characteristics, horizontal offset samples, and vertical offset samples.

[0104] During training, the system filters out abnormal offset samples in the offset sample table, removing samples that correspond to calibration board center recognition failure, image blurring, abnormal acquisition angle, and abnormal magnification signal.

[0105] The model training objectives include reducing the horizontal and vertical offset fitting errors while maintaining a smooth change in the correction amount within adjacent ratio intervals.

[0106] The training parameters include the number of candidate segment nodes, the number of maximum spline basis functions, the order of interaction terms, the pruning threshold, and the residual threshold.

[0107] The maximum number of spline basis functions can be set according to the scaling sampling density. The order of the interaction term is preferably set to second order to simultaneously express the interaction relationship between the scaling term and the error term.

[0108] The model convergence condition is that both the horizontal offset residual and the vertical offset residual are less than the preset pixel threshold, or the change in the verification residual after two adjacent rounds of model pruning is less than the set threshold.

[0109] In this embodiment, the improved model building module also includes model validation processing during the training process. The model validation processing divides the offset sample table into training samples and validation samples. The training samples are used to build the horizontal offset correction model and the vertical offset correction model, while the validation samples are used to calculate the offset prediction error of the model at points that were not included in the training.

[0110] The validation metrics include horizontal average residual, vertical average residual, maximum horizontal residual, maximum vertical residual, and high-magnification interval residual.

[0111] If both the horizontal average residual and the vertical average residual corresponding to the validation sample are less than the preset pixel threshold, the model validation is successful. If the residual in the high-magnification interval exceeds the preset threshold, the system will add high-magnification interval segmentation nodes or increase the sample weights in the high-magnification interval and then retrain.

[0112] Model validation can reduce the prediction error of the model at unsampled magnification positions, enabling the current magnification offset calculation module to obtain a stable correction amount during continuous magnification changes.

[0113] In this embodiment, the improved model building module associates the horizontal offset correction model, the vertical offset correction model, the improved spline basis function set, the mechanical error calibration features, and the magnification change direction data to obtain model call data for the current magnification offset calculation module to call.

[0114] The model call data includes model version identifier, applicable magnification range, horizontal channel spline basis function set, vertical channel spline basis function set, mechanical error interaction term, magnification direction branch relationship, and model residual statistics.

[0115] The model call data is bound to the PTZ camera identifier, lens identifier, and calibration batch identifier to avoid mixing correction models between different devices.

[0116] In this embodiment, the current magnification offset calculation module specifically includes: acquiring the current magnification signal output by the PTZ camera, performing format parsing, numerical verification, and magnification range matching processing on the current magnification signal to obtain the current magnification value.

[0117] The magnification signal analysis process includes reading the magnification control command, the lens feedback magnification value, and the image acquisition timestamp, and then verifying the consistency between the magnification control command and the lens feedback magnification value.

[0118] The current magnification offset calculation module obtains the current magnification value and the magnification at the previous sampling time, judges the difference between the current magnification value and the magnification at the previous sampling time, obtains the current magnification change direction, and matches the current magnification change direction with the magnification increase state, magnification decrease state, or magnification hold state.

[0119] The current direction of the magnification change is used as one of the inputs to the model call to distinguish the offset differences in scenarios where the magnification increases, decreases, and remains constant.

[0120] In this embodiment, the current magnification offset calculation module can receive the magnification feedback signal from the PTZ camera in real time.

[0121] The magnification feedback signal can come from the gimbal control protocol, the lens zoom motor feedback value, or image acquisition metadata.

[0122] The current magnification offset calculation module performs format parsing, numerical range judgment, and abnormal jump detection on the magnification feedback signal.

[0123] If the current magnification value changes more than the maximum magnification change allowed by the PTZ camera within adjacent sampling periods, the system determines that the magnification feedback signal is abnormal and will not output new offset correction data for the time being. If the current magnification value is within the calibrated magnification range, the system calls the horizontal offset correction model and the vertical offset correction model to calculate the correction amount; If the current magnification value is between two calibrated magnification values, the system performs local spline calculations based on the segment node interval where the current magnification value is located; If the current magnification value exceeds the calibrated magnification range, the system outputs the correction amount corresponding to the boundary magnification and records the magnification out-of-bounds flag.

[0124] Through the above processing, the current magnification offset calculation module can adapt to real-time magnification changes and avoid erroneous compensation caused by abnormal magnification signals.

[0125] In this embodiment, the current magnification offset calculation module obtains the current magnification value, mechanical error calibration characteristics, current magnification change direction, and model call data. It inputs the current magnification value, mechanical error calibration characteristics, and current magnification change direction into the horizontal offset correction model and the vertical offset correction model to obtain the horizontal offset model output and the vertical offset model output.

[0126] If the current magnification value falls within the calibrated magnification range, the system performs interpolation calculations based on the improved spline basis function of the corresponding magnification interval; If the current magnification value is between two adjacent calibration magnifications, the system performs local spline calculation based on the interval where the corrected magnification segment node is located. If the current magnification value exceeds the calibrated magnification range, the system limits the output of the correction amount based on the magnification range matching result and generates a magnification out-of-bounds warning flag.

[0127] The current magnification offset calculation module generates the current horizontal offset correction amount based on the horizontal offset model output, and generates the current vertical offset correction amount based on the vertical offset model output. It then encapsulates the current horizontal offset correction amount, the current vertical offset correction amount, the current magnification value, and the current magnification change direction to obtain the current magnification offset correction data.

[0128] In this embodiment, the coordinate compensation module specifically includes: acquiring the target center coordinates, the image center reference coordinates, and the current magnification offset correction data in the image to be corrected.

[0129] The target center coordinates can come from image detection algorithms, target tracking algorithms, manual point selection results, or calibration board center recognition results.

[0130] The coordinate compensation module reads the current horizontal offset correction and the current vertical offset correction from the current magnification offset correction data, and performs horizontal and vertical coordinate compensation on the gimbal lens pointing deviation corresponding to the target center coordinates to obtain the compensated target center coordinates.

[0131] The horizontal coordinate compensation corrects the horizontal component of the target center coordinates based on the current horizontal offset correction amount, while the vertical coordinate compensation corrects the vertical component of the target center coordinates based on the current vertical offset correction amount. The coordinate compensation module then associates the compensated target center coordinates with the current magnification value, the current magnification change direction, the image frame identifier, and the model version identifier to generate the image center offset correction result.

[0132] The image center offset correction result can be transmitted to the display overlay module, gimbal control module, target tracking module, or subsequent image analysis module to keep the target center position stable at different magnifications.

[0133] In this embodiment, when the coordinate compensation module compensates for the target center coordinates, it also includes compensation direction judgment and compensation boundary restriction.

[0134] The compensation direction is determined based on the current horizontal offset correction and the current vertical offset correction. If the horizontal offset correction is positive, reverse compensation is performed on the horizontal component of the target center coordinates; if the horizontal offset correction is negative, corresponding direction compensation is performed on the horizontal component of the target center coordinates. The vertical direction is compensated in the same way.

[0135] The compensation boundary limit is used to prevent the center coordinates of the target after compensation from exceeding the image boundary. If the center coordinates of the target after compensation exceed the image width or image height range, the system will limit the center coordinates of the target after compensation to the effective image range and generate a boundary limit identifier.

[0136] The image center offset correction results output by the coordinate compensation module include the compensated target center coordinates, current magnification value, current magnification change direction, image frame identifier, model version identifier, and boundary limit identifier.

[0137] In this embodiment, the compensation verification feedback module specifically includes: re-acquiring or reading the compensated image based on the image center offset correction result, performing target center identification or calibration board center identification on the compensated image, and calculating the difference between the identified actual center coordinates after compensation and the image center reference coordinates to obtain the compensation residual data formed by the gimbal lens pointing deviation.

[0138] The compensation residual data includes horizontal residual, vertical residual, residual direction, and residual duration in frames.

[0139] The compensation verification feedback module judges the compensation residual data according to the preset residual threshold. When the compensation residual data does not exceed the preset residual threshold, the compensation verification result is marked as passed. When the compensation residual data exceeds the preset residual threshold, the current magnification value, the current magnification change direction, the compensation residual data, the target center coordinates, the compensated target center coordinates and the corresponding image frame identifier are written into the offset sample table, and the improved multivariate adaptive regression spline model update processing is triggered to obtain the updated horizontal offset correction model and vertical offset correction model.

[0140] In this embodiment, when the compensation verification feedback module verifies the compensation result, it adopts a combination of residual detection and continuous frame stability detection.

[0141] Residual detection obtains the compensation residual data by re-identifying the target center coordinates in the compensated image and calculating the difference between the actual center coordinates after compensation and the reference coordinates of the image center.

[0142] Continuous frame stability detection determines whether the compensation is stable by statistically analyzing the changes in the compensation residuals over multiple consecutive frames.

[0143] If the compensation residual of a single frame exceeds the threshold, but the residual of consecutive frames does not continue to increase, the system can mark the frame as a suspected jitter frame; if the compensation residual of multiple consecutive frames exceeds the threshold and the direction is consistent, the system determines that there is a deviation in the model of the current magnification range and writes the corresponding sample into the offset sample table.

[0144] By combining residual detection with continuous frame stability detection, erroneous feedback updates caused by image noise, short-term target occlusion, or recognition errors can be reduced.

[0145] In this embodiment, the compensation verification feedback module does not directly overwrite the original samples when updating the offset sample table. Instead, it adds feedback samples based on the image frame identifier, magnification value, and magnification change direction, and records the source type, residual size, and verification status of the feedback samples.

[0146] For feedback samples where residuals in the same direction appear in multiple consecutive frames, the system increases the weight of the feedback samples; for isolated residual samples caused by jitter in a single frame, the system decreases the weight of the feedback samples or postpones their inclusion in the model training sample matrix. When the compensation verification feedback module triggers a model update, incremental updates are preferred.

[0147] The incremental update method does not reconstruct all the scaling factor samples. Instead, it adds the feedback samples whose residuals exceed the threshold to the offset sample table and updates the segment nodes and spline basis functions within the affected scaling factor range.

[0148] If the feedback samples are concentrated in a certain magnification range, the system only adjusts the spline basis function and mechanical error interaction term corresponding to that magnification range; if the feedback samples cover multiple magnification ranges, the system re-executes the magnification segment node correction and dual-channel fitting.

[0149] After the model update is completed, the system writes the updated horizontal offset correction model and vertical offset correction model into the model call data, and retains the model version before the update so that it can be rolled back if the compensation effect decreases.

[0150] Example 1: To verify the feasibility of this invention in practice, it was applied to a scenario involving the calibration of the center deviation of a pan-tilt-zoom (PTZ) camera lens in a security equipment manufacturing company. The PTZ camera has an image resolution of 1920×1080 and an optical magnification range of 1x to 30x. The PTZ supports horizontal rotation and vertical tilt control, and the lens magnification information can be read in real-time via the control protocol. The system uses 1x magnification as the baseline magnification, with the image geometric center coordinates at (960, 540). Eight frames of calibration board images were acquired at the baseline magnification. After grayscale conversion, edge enhancement, and corner detection processing, the average actual coordinates of the calibration board center were (961.2, 539.4). The system recorded this initial difference as the baseline deviation and subtracted it uniformly in subsequent multi-magnification sample calculations.

[0151] In this embodiment, the reference calibration module controls the PTZ camera to maintain a fixed horizontal and vertical angle, acquires calibration images at a reference magnification, identifies the actual coordinates of the calibration board center, and generates the reference coordinates of the image center based on the image resolution. The multi-magnification sample generation module sets 40 effective calibration magnification points within the range of 1x to 30x, with a larger magnification interval for the 1x to 10x range, a medium magnification interval for the 10x to 20x range, and a denser magnification interval for the 20x to 30x range. For each magnification point, 6 to 8 frames of calibration images are continuously acquired. After discarding frames with missing corner points, blurred images, and unstable magnification feedback, the average center coordinates of the effective frames are taken as the actual coordinates of the calibration board center at that magnification.

[0152] During sample generation, the system records the magnification value, direction of magnification change, actual coordinates of the calibration board center, horizontal offset samples, and vertical offset samples. Statistical analysis shows that, without correction, the maximum center offset is 7.8 pixels horizontally and 5.6 pixels vertically in the 1x to 10x range; 18.9 pixels horizontally and 13.5 pixels vertically in the 10x to 20x range; and 32.6 pixels horizontally and 25.1 pixels vertically in the 20x to 30x range. These offsets are not due to translation of the image content itself, but rather are caused by a combination of factors including optical axis deviation, lens installation deviation, zoom lens group movement deviation, and gimbal mechanical clearance during magnification changes, and are manifested as a center offset through changes in the center imaging position of the calibration board.

[0153] The mechanical error feature construction module acquires the rotation axis concentricity error, lens mounting deviation, lens optical axis deviation, zoom lens group movement deviation, horizontal rotation clearance, and vertical pitch clearance of the PTZ camera. Specifically, the rotation axis concentricity error is 0.018 degrees, the lens horizontal mounting deviation is 0.020 degrees, the lens vertical mounting deviation is -0.015 degrees, the lens optical axis deviation is 0.022 degrees, the horizontal rotation clearance is 0.031 degrees, and the vertical pitch clearance is 0.026 degrees. The system unifies the dimensions and compresses the numerical range of the above error data, and encodes the magnification increase, magnification decrease, and magnification hold states in a directional manner to generate mechanical error calibration features.

[0154] The improved model building module inputs magnification data, offset sample tables, and mechanical error calibration features into the improved multivariate adaptive regression spline model. Model training samples use magnification sampling points as the basic unit. Input fields include magnification value, magnification change direction, rotation axis error term, lens assembly deviation term, optical axis deviation term, gyration clearance term, and pitch clearance term. Output fields include horizontal and vertical offset samples. The system divides the samples corresponding to 40 magnification points into training and validation samples. The maximum number of spline basis functions is set to 30, the interaction term order is set to second order, the pruning threshold is set to 0.016, and the residual threshold is set to 2 pixels. After eight rounds of node filtering and pruning, the validation residuals of both the horizontal and vertical offset correction models are less than the preset thresholds.

[0155] During the current magnification offset calculation, the current magnification offset calculation module reads the magnification feedback signal output by the PTZ camera and the magnification at the previous sampling time, determines the current magnification value and the direction of magnification change, and calls the horizontal offset correction model and the vertical offset correction model to generate the current magnification offset correction data. For example, when the current magnification is 24.5 times and the magnification direction is increasing, the model outputs a current horizontal offset correction of 26.4 pixels and a current vertical offset correction of -18.6 pixels; when the current magnification is also 24.5 times but the magnification direction is decreasing, the model outputs a current horizontal offset correction of 25.2 pixels and a current vertical offset correction of -17.8 pixels. The above results show that the system can distinguish the mechanical backlash difference during the magnification increase and decrease process.

[0156] The coordinate compensation module applies the current magnification offset correction data to the gimbal lens pointing deviation corresponding to the target center coordinates. Taking a 30x image frame as an example, the original target center coordinates output by the target detection module are (1029.3, 516.1). After compensation based on the current horizontal and vertical offset corrections, the system obtains the compensated target center coordinates as (998.4, 540.5). The compensation verification feedback module re-identifies the calibration board center in the compensated image, calculates the difference between the compensated actual center coordinates and the image center reference coordinates, and obtains a horizontal residual of 1.3 pixels and a vertical residual of 0.8 pixels, which are lower than the preset residual threshold.

[0157] Table 1 Comparison of Image Center Shift Correction Effects Caused by Gimbal Lens Deviation

[0158] As shown in Table 1, the PTZ camera exhibits significant center shift during magnification changes, and this shift amplifies with increasing magnification. The maximum original residual in the low magnification range is approximately 9.6 pixels, increasing to approximately 23.2 pixels in the medium magnification range, and further reaching approximately 41.1 pixels in the high magnification range. This indicates that lens magnification, optical axis deviation, lens assembly deviation, rotation axis concentricity error, and PTZ rotation clearance are all amplified at high magnification. The residuals in the table are not ordinary image content translation errors, but rather the residual manifestation of the PTZ lens center deviation in the image coordinate system.

[0159] While traditional linear compensation or lookup table compensation can reduce some offset, it typically only covers integer magnifications or a few commonly used magnification points. For non-integer magnifications, it can only use nearest-neighbor magnification substitution or simple interpolation. Furthermore, it cannot distinguish the mechanical backlash differences during magnification increases and decreases. Therefore, the average residual of traditional solutions remains 4.9 pixels across the entire magnification range, with a maximum residual of 7.4 pixels at high magnifications, and target center jitter remains around 3.2 pixels.

[0160] This invention establishes an offset sample table using 40 valid calibration magnification points and performs encrypted sampling in the high magnification range, ensuring a more comprehensive record of the nonlinear relationship between magnification and offset. By incorporating mechanical error calibration features, it introduces encodings for rotation axis concentricity error, lens assembly deviation, optical axis deviation, zoom lens group movement deviation, horizontal rotation clearance, vertical pitch clearance, and magnification direction, enabling the model to incorporate the equipment's own structural errors for offset fitting. The improved multivariate adaptive regression spline model further employs magnification segmentation node correction, mechanical error interaction term construction, and horizontal and vertical dual-channel fitting processing to model different magnification ranges and different offset directions separately, avoiding the inadequacy of a single linear model or a unified lookup table model in representing local magnification abrupt changes.

[0161] The results show that the system of this invention reduces the average residual at full magnification from 15.2 pixels to 1.08 pixels, a reduction of approximately 78.0% compared to traditional compensation methods. The maximum residual is compressed from 32.6 pixels to 1.9 pixels, the compensation error at non-integer magnification is reduced to approximately 1.4 pixels, and the target center jitter at high magnification is reduced to an average of 1.3 pixels. When the residual exceeds a threshold, the compensation verification feedback module writes the current magnification, the direction of magnification change, and the compensation residual into the offset sample table and triggers a model update, further reducing the average residual to 0.84 pixels. These data demonstrate that this invention can clearly complete the image measurement, model calibration, and continuous feedback correction of gimbal lens deviation without changing the image center offset correction theme defined in the patent title.

[0162] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An image center offset correction system based on image processing, characterized in that, Includes the following modules: The benchmark calibration module acquires calibration images of the PTZ camera at a benchmark magnification, performs calibration board center identification and image center benchmark calculation on the calibration images, and generates image center benchmark coordinates. The multi-magnification sample generation module controls the PTZ camera to acquire calibration images at multiple calibration magnifications. It calculates the center offset sample caused by the PTZ lens magnification change based on the change in the calibration plate center relative to the image center reference coordinate, and generates an offset sample table. The mechanical error feature construction module acquires mechanical structure error data, lens assembly deviation data, and magnification change data of the PTZ camera, and performs normalization, orientation encoding, and feature combination processing on the above data to generate mechanical error calibration features. The improved model building module inputs the magnification data, offset sample table and mechanical error calibration features into the improved multivariate adaptive regression spline model. After magnification segment node correction, mechanical error interaction term construction and horizontal and vertical dual-channel fitting processing, a horizontal offset correction model and a vertical offset correction model are generated. The current magnification offset calculation module obtains the current magnification value and the direction of magnification change, and combines the mechanical error calibration characteristics to call the horizontal offset correction model and the vertical offset correction model to generate the current magnification offset correction data; The coordinate compensation module performs horizontal and vertical compensation on the gimbal lens pointing deviation corresponding to the target center coordinates based on the current magnification offset correction data, and generates the compensated target center coordinates. The compensation verification feedback module performs offset residual detection and compensation result verification processing on the target center coordinates after compensation, generates compensation verification results, and updates the offset sample table and the improved multivariate adaptive regression spline model based on the compensation verification results.

2. The image center offset correction system based on image processing according to claim 1, characterized in that, The benchmark calibration module specifically includes: The reference magnification setting unit acquires the initial magnification, image resolution, and calibration board position parameters of the PTZ camera, sets the initial magnification as the reference magnification, and generates reference acquisition control data. The reference image acquisition unit controls the pan-tilt camera to acquire images of the calibration board based on the reference acquisition control data, thereby obtaining the reference calibration image. The calibration board center recognition unit performs grayscale conversion, edge enhancement, and calibration board corner point recognition processing on the reference calibration image to obtain the actual coordinates of the calibration board center. The image center calculation unit calculates the geometric center coordinates of the image based on the image resolution, and uses the geometric center coordinates of the image as the reference coordinates of the image center; The reference data generation unit associates and stores the reference magnification, reference calibration image, actual coordinates of the calibration board center, and reference coordinates of the image center to generate reference calibration data.

3. The image center offset correction system based on image processing according to claim 1, characterized in that, The multi-fold sample generation module specifically includes: The magnification sequence generation unit obtains the minimum magnification, maximum magnification, and magnification sampling interval of the PTZ camera, and discretizes the magnification range to obtain multiple calibrated magnifications; The multi-magnification image acquisition unit sequentially controls the PTZ camera to acquire calibration images according to multiple calibration magnifications, and records the magnification value, acquisition order and magnification change direction of the calibration images to obtain a set of multi-magnification calibration images; The actual center identification unit performs calibration board center feature identification on the calibration images in the multi-magnification calibration image set to obtain the actual coordinates of the calibration board center corresponding to each calibration magnification. The offset difference calculation unit performs horizontal and vertical difference calculations between the actual coordinates of the calibration board center and the reference coordinates of the image center to obtain multi-magnification horizontal offset samples and multi-magnification vertical offset samples generated by the gimbal lens magnification change. The offset sample table construction unit arranges the calibration magnification, acquisition order, magnification change direction, actual coordinates of the calibration plate center, multi-magnification horizontal offset samples, and multi-magnification vertical offset samples to obtain the offset sample table.

4. The image center offset correction system based on image processing according to claim 1, characterized in that, The mechanical error feature construction module specifically includes: The mechanical parameter acquisition unit acquires the concentricity error of the rotation axis of the pan-tilt camera, the horizontal rotation clearance of the pan-tilt, and the vertical pitch clearance of the pan-tilt, thus obtaining mechanical structure error data. The lens deviation acquisition unit acquires the lens installation deviation, lens optical axis deviation, and zoom lens group movement deviation of the PTZ camera to obtain lens assembly deviation data. The magnification direction generation unit obtains the magnification at the current sampling time and the magnification at the previous sampling time, performs a difference judgment between the two, and obtains the magnification increase state, magnification decrease state, or magnification hold state. The error scale unification unit performs dimensional unification and numerical range compression processing on mechanical structure error data and lens assembly deviation data to obtain normalized error data. The calibration feature combination unit combines normalized error data, magnification direction encoded data, and corresponding magnification data to obtain mechanical error calibration features that include magnification, rotation axis error, lens assembly deviation, optical axis deviation, rotation clearance, pitch clearance, and magnification direction.

5. The image center offset correction system based on image processing according to claim 1, characterized in that, The improved model construction module specifically includes: The training sample matrix generation unit acquires the magnification data, offset sample table, and mechanical error calibration features. It takes the magnification data and mechanical error calibration features as input columns and the multi-magnification horizontal offset samples and multi-magnification vertical offset samples as output columns to obtain the model training sample matrix. The basic spline modeling unit generates candidate segmentation nodes based on the scalar data, multi-scalar horizontal offset samples, and multi-scalar vertical offset samples in the model training sample matrix, and constructs a set of candidate spline basis functions based on the candidate segmentation nodes to obtain the basic multivariate adaptive regression spline model. The improved unit for magnification-mechanical coupling acquires the candidate spline basis function set, mechanical error calibration features, and magnification change direction data. It then performs magnification segmentation node correction, mechanical error interaction term construction, directional branch configuration, and horizontal and vertical channel allocation on the candidate spline basis function set to obtain the improved spline basis function set. The dual-channel model generation unit performs fitting processing on the multi-level horizontal offset samples based on the horizontal channel improved spline basis function set in the improved spline basis function set to obtain the horizontal offset correction model, and performs fitting processing on the multi-level vertical offset samples based on the vertical channel improved spline basis function set to obtain the vertical offset correction model. The model call data generation unit associates the horizontal offset correction model, the vertical offset correction model, the improved spline basis function set, the mechanical error calibration features, and the magnification change direction data to obtain the model call data.

6. The image center offset correction system based on image processing according to claim 5, characterized in that, The improved rate-capacity mechanical coupling unit specifically includes: The multiplier segment node correction sub-unit obtains the multiplier segment nodes in the candidate spline basis function set, as well as the horizontal and vertical offset changes corresponding to adjacent multipliers. The multiplier segment nodes are retained, merged, or deleted to obtain the corrected multiplier segment nodes. The mechanical error interaction term construction sub-unit obtains the rotation axis error term, lens assembly deviation term, optical axis deviation term, rotation clearance term, pitch clearance term, and magnification direction term. The above error terms are combined with the corrected magnification segment node to obtain the magnification mechanical coupling interaction term. The horizontal and vertical channel allocation sub-units are used to add the magnification mechanical coupling interaction term to the horizontal offset spline channel and the vertical offset spline channel according to the horizontal offset contribution and the vertical offset contribution, respectively, to obtain the improved spline basis function set of the horizontal channel and the improved spline basis function set of the vertical channel. The scaling direction branching processing subunit configures the branching relationship of the horizontal channel improved spline basis function set and the vertical channel improved spline basis function set according to the scaling increase state, scaling decrease state and scaling hold state, so as to obtain the branch spline relationship corresponding to the scaling change direction; The improved spline output sub-unit combines the improved spline basis function set of the horizontal channel, the improved spline basis function set of the vertical channel, the scaling mechanical coupling interaction term, and the branch spline relationship to obtain the improved spline basis function set.

7. The image center offset correction system based on image processing according to claim 1, characterized in that, The current magnification offset calculation module specifically includes: The magnification signal parsing unit acquires the current magnification signal output by the PTZ camera, performs format parsing, value verification, and magnification range matching on the current magnification signal, and obtains the current magnification value. The magnification direction confirmation unit obtains the current magnification value and the magnification at the previous sampling time, performs a difference judgment on the current magnification value and the magnification at the previous sampling time, and obtains the current magnification change direction; The model calling unit obtains the current magnification value, mechanical error calibration characteristics, and current magnification change direction. It inputs the current magnification value, mechanical error calibration characteristics, and current magnification change direction into the horizontal offset correction model and the vertical offset correction model to obtain the horizontal offset model output and the vertical offset model output. The correction output unit generates the current magnification offset correction data based on the horizontal offset model output and the vertical offset model output, and sends the current magnification offset correction data to the coordinate compensation module.

8. The image center offset correction system based on image processing according to claim 1, characterized in that, The coordinate compensation module specifically includes: The coordinate input unit acquires the target center coordinates, the image center reference coordinates, and the current magnification offset correction data in the image to be corrected. The deviation direction determination unit reads the current horizontal offset correction amount and the current vertical offset correction amount to determine the horizontal compensation direction and vertical compensation direction corresponding to the gimbal lens pointing deviation; The coordinate compensation unit compensates for the horizontal pointing deviation of the gimbal lens corresponding to the target center coordinates based on the horizontal compensation direction and the current horizontal offset correction amount, and compensates for the vertical pointing deviation of the gimbal lens corresponding to the target center coordinates based on the vertical compensation direction and the current vertical offset correction amount, thus obtaining the compensated target center coordinates. The compensation result output unit associates the compensated target center coordinates with the current magnification value, the current magnification change direction, and the image frame identifier to generate the image center offset correction result.

9. The image center offset correction system based on image processing according to claim 1, characterized in that, The compensation verification feedback module specifically includes: The compensation center identification unit performs target center identification or calibration board center identification on the compensated image corresponding to the image center offset correction result to obtain the actual center coordinates after compensation. The residual calculation unit performs horizontal and vertical difference calculations between the compensated actual center coordinates and the image center reference coordinates to obtain the compensation residual data formed by the gimbal lens pointing deviation. The residual judgment unit compares the compensated residual data with a preset residual threshold and generates a compensation verification result. The feedback sample writing unit writes the current multiplier value, the current multiplier change direction, and the compensation residual data into the offset sample table when the compensation residual data exceeds the preset residual threshold, thus obtaining the updated offset sample table. The model update unit triggers an improved multivariate adaptive regression spline model update process based on the updated offset sample table, resulting in updated horizontal offset correction models and vertical offset correction models.

Images