An adaptive multi-view cooperative camera offset correction method and system

Abstract

The application relates to the field of computer vision and intelligent monitoring technology, and discloses a self-adaptive multi-view cooperative camera offset correction method and system, which comprises the following steps: determining a threshold based on an offset angle and a feature point matching pair quantity mapping relationship; calculating the feature point matching pair quantity of a current image and a reference image and determining an offset level. If the offset is slight, camera pose correction is performed based on the reference image and the current image; if the offset is ordinary, multi-view joint correction of the camera pose is performed based on the reference image and a plurality of historical view images; if the offset is serious, an additional view image is collected, and multi-view joint correction of the camera pose is performed based on the reference image, the current image and the additional view image. The application establishes a self-adaptive grading determination mechanism based on the mapping relationship between the feature point matching quantity and the offset angle, improves the adaptability of the system to different working conditions, and avoids waste of computing resources.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent monitoring technology, specifically to an adaptive multi-view collaborative camera offset correction method and system. Background Technology

[0002] With the transformation of smart grid operation and maintenance models, the daily inspection of substations and transmission lines has become heavily reliant on PTZ cameras deployed on-site. To obtain instrument readings or switch status, cameras frequently need to invoke preset position commands to align with specific targets. However, the impact of wind loads in the outdoor environment, the thermal expansion and contraction of the base, and the mechanical backlash of the PTZ drive gears inevitably cause the camera's physical pointing to drift. Once the image is misaligned, the backend AI recognition algorithm cannot read the data, and may even cause the monitored target to completely disappear from the field of view.

[0003] Existing PTZ control technologies primarily employ a combination of motor encoder feedback and basic vision assistance. The encoder can read the motor's rotation steps in real time, offering a fast response and extremely high real-time execution of control commands. For vision assistance, conventional solutions often utilize global template matching or pixel grayscale difference techniques. These algorithms have simple computational logic, low hardware computing power requirements, and can complete initial image comparison with low system resource consumption. Under ideal conditions of constant illumination and minimal mechanical error, existing technologies can quickly maintain approximate image stability, ensuring continuous video stream transmission.

[0004] However, in actual, complex field operations, the aforementioned technologies revealed significant logical gaps. Encoder feedback is only a semi-closed-loop control; it can only indicate how much the motor has rotated, but cannot detect the actual pointing of the lens after transmission chain slippage, resulting in discrepancies between physical readings and the visual image. Visual algorithms relying on global pixel difference are overly sensitive to ambient light; sudden changes in light and shadow caused by cloud cover are often misinterpreted as displacement, triggering meaningless and frequent jitter. Furthermore, the high threshold set for anti-shake purposes causes the system to ignore slowly accumulating minute shifts. In addition, traditional dual-view geometric correction heavily relies on the overlap between the current frame and the reference frame. When the offset is slightly large, reducing the common field of view, the effective feature points drop sharply, and the calculated pose parameters are prone to divergence and non-convergence. Even more critically, in the event of strong winds causing large-angle shifts, the current and reference images completely fail to overlap, and the existing feature matching mechanism will directly fail due to the inability to find associated points, leading to system blindness and requiring manual reset. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an adaptive multi-view collaborative camera offset correction method and system. This addresses the technical challenges of existing power monitoring cameras, which suffer from target loss and high manual reset costs due to issues such as misjudgment caused by lighting changes, pose calculation divergence caused by sparse feature points, and feature matching failure under large-angle offsets when pointing offsets occur during long-term operation.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an adaptive multi-view collaborative camera offset correction method, comprising the following steps:

[0007] Based on the offset angle between the reference image and the image sample and the number of feature point matching pairs between the reference image and the image sample, a mapping relationship between the offset angle and the number of feature point matching pairs is constructed; the offset degree level is divided according to the preset offset angle threshold, and the number of feature point matching pairs threshold used to determine the offset degree level is calculated based on the mapping relationship; wherein, the offset degree level includes slight offset, normal offset and severe offset.

[0008] Acquire the current image captured by the camera and retrieve the pre-stored reference image corresponding to the camera;

[0009] Feature points are extracted from the reference image and the current image and matched to calculate the number of feature point matching pairs;

[0010] The number of feature point matching pairs is compared with a threshold number of feature point matching pairs to determine the offset level, and pose adjustment is performed based on the offset level.

[0011] If the offset is determined to be slight, then perform camera pose correction based on the reference image and the current image;

[0012] If it is determined to be a normal offset, then perform multi-view joint correction of the camera pose based on the reference image and multiple historical view images;

[0013] If a severe offset is determined, the camera is controlled to actively acquire additional viewpoint images, and multi-view joint correction of the camera pose is performed based on the reference image, the current image, and the additional viewpoint images.

[0014] An adaptive multi-view collaborative camera offset correction system includes:

[0015] The threshold setting module is used to construct a mapping relationship between the offset angle and the number of feature point matching pairs based on the offset angle between the reference image and the image sample and the number of feature point matching pairs between the reference image and the image sample; it divides the offset degree level according to the preset offset angle threshold, and calculates the feature point matching pair number threshold used to determine the offset degree level based on the mapping relationship; wherein, the offset degree level includes slight offset, normal offset and severe offset.

[0016] The image acquisition module is used to acquire the current image captured by the camera and retrieve a pre-stored reference image corresponding to the camera.

[0017] The feature point extraction and matching module is used to extract feature points from the reference image and the current image and perform matching to calculate the number of feature point matching pairs;

[0018] The camera offset determination module is used to compare the number of feature point matching pairs with a threshold number of feature point matching pairs to determine the offset level.

[0019] The offset correction module is used to perform pose adjustment according to the degree of offset. If it is determined to be a slight offset, it performs camera pose correction based on the reference image and the current image. If it is determined to be a normal offset, it performs multi-view joint correction of camera pose based on the reference image and multiple historical view images. If it is determined to be a severe offset, it controls the camera to actively acquire additional view images and performs multi-view joint correction of camera pose based on the reference image, the current image and the additional view images.

[0020] This invention provides an adaptive multi-view collaborative camera offset correction method and system. It has the following beneficial effects:

[0021] 1. This invention establishes an adaptive hierarchical judgment mechanism based on the mapping relationship between the number of feature point matching and the offset angle. According to real-time matching data, the offset state is dynamically divided into three levels: slight, normal and severe, and a differentiated strategy is matched. This improves the system's adaptability to different working conditions and avoids the waste of computing resources. Compared with the rigid mode of traditional technology that relies on a single fixed threshold or simple image similarity to trigger correction, this invention effectively solves the problem of frequent malfunctions caused by correction trigger delay or slight jitter due to improper threshold setting in complex power scenarios.

[0022] 2. This invention addresses common offset scenarios with reduced field-of-view overlap by introducing a multi-view collaborative optimization bundle adjustment strategy based on historical image sequences. By jointly optimizing the pose of multiple frames and the coordinates of three-dimensional feature points, the average pixel reprojection error is converged to the sub-pixel level. Compared with existing technologies that rely solely on the current frame and reference frame for dual-view geometric calculation, this invention overcomes the instability of pose estimation values ​​and the divergence of accumulated errors that are easily caused when the number of feature point pairs decreases or their distribution is uneven. This significantly ensures the high accuracy and robustness of the correction process.

[0023] 3. This invention addresses severe offset conditions caused by feature matching failure by employing active rotation scanning and intermediate transition viewpoint construction techniques. It reconstructs the geometric relationship between the current image and the reference image using a selected high-matching intermediate transition viewpoint. Unlike traditional visual servoing methods that directly determine tracking loss or require manual reset when feature matching pairs are insufficient, this invention achieves fully automatic closed-loop recovery in cases where direct matching breaks due to large-angle physical offset. This expands the effective working domain of the visual correction system and significantly reduces remote maintenance costs. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0025] Figure 2 This is a system module block diagram according to an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0027] See attached document Figure 1 and attached Figure 2 , Figure 1 This is a flowchart of an adaptive multi-view collaborative camera offset correction method according to an embodiment of the present invention. Figure 2 This is a block diagram of an adaptive multi-view collaborative camera offset correction system according to an embodiment of the present invention. The present invention provides an adaptive multi-view collaborative camera offset correction method, applied to an adaptive multi-view collaborative camera offset correction system. The adaptive multi-view collaborative camera offset correction system includes an image acquisition module, a feature point extraction and matching module, a threshold setting module, a camera offset degree judgment module, and an offset correction module. The adaptive multi-view collaborative camera offset correction includes the following steps:

[0028] S100: The threshold setting module pre-constructs a grading standard for determining the degree of offset. Based on the offset angle between the reference image and the image sample and the number of feature point matching pairs between the reference image and the image sample, a mapping relationship between the offset angle and the number of feature point matching pairs is established. The corresponding feature point matching pair number threshold is calculated according to the preset offset angle threshold.

[0029] S200 acquires the current image captured by the camera at preset time intervals through the image acquisition module, and retrieves the reference image of the corresponding camera that is stored in advance. The reference image is a standard view image of the device in a state of no offset.

[0030] S300 uses the scale-invariant feature transformation algorithm to perform feature point detection and descriptor generation on the reference image and the current image through the feature point extraction and matching module, calculates the feature point matching pairs between the reference image and the current image, and counts the number of feature point matching pairs.

[0031] S400 determines the current offset level of the camera by comparing the number of feature point matching pairs with the threshold number of feature point matching pairs through the camera offset degree judgment module. The offset degree levels include slight offset, normal offset and severe offset.

[0032] The S500 uses an offset correction module to select the corresponding correction strategy based on the offset level to perform pose calculation and adjustment. When the offset level is slight, it performs camera pose correction based on the reference image and the current image. When the offset level is normal, it performs multi-view joint correction of camera pose based on the reference image and multiple historical view images. When the offset level is severe, it controls the camera to actively acquire additional view images and performs multi-view joint correction of camera pose based on the reference image, the current image, and the additional view images.

[0033] Step S100 involves pre-constructing a grading standard for determining the degree of offset using a threshold setting module, establishing a mapping relationship between offset angle and the number of feature point matching pairs based on the power scene camera offset dataset, and calculating the corresponding feature point matching pair threshold according to a preset offset angle threshold. Specifically, this includes the following sub-steps:

[0034] S101, Set the grading angle thresholds for camera offset. Based on the field-of-view requirements for power equipment monitoring and the stability of the installation environment, a first angle threshold is preset. Second angle threshold ,and Based on the first angle threshold Second angle threshold The actual physical offset angle of the camera Divided into three levels: when When defined as a slight offset; When defined as a normal offset; when A severe offset is defined as the first angle threshold. The selection range includes, but is not limited to, 2 to 5 degrees, and the second angle threshold. The selection range includes, but is not limited to, 15 to 25 degrees, with the specific value configured according to the focal length and field of view coverage of the surveillance camera.

[0035] S102, construct a power scene camera offset dataset and extract sample image groups. Collect video data of camera rotation captures covering various power equipment scenes, including outdoor substation scenes, transmission line tower scenes, and indoor distribution room scenes. The video data records image frames of the camera rotating from its initial position to different angles. Accurate physical rotation angle values ​​are labeled for each frame using synchronously recorded high-precision pan-tilt encoder feedback data or readings from external angle sensors. For the [specific image group] in the dataset... In power-related scenarios, three images are extracted from video data as a set of samples. ,in This refers to the number of videos captured in this type of power scenario (i.e., the total number of sample groups). For sample index, The value range is 1 to In the first In the group sample, This is a reference image sample when the camera has not shifted. The camera rotates at an angle relative to the reference image sample. The first image sample at that time; The camera rotates at an angle relative to the reference image sample. The second image sample at that time.

[0036] S103, calculate the number of feature point matching pairs for each scene sample at a specific angle threshold. Then, call the feature point extraction and matching algorithm. For the sample set Each group of samples in the dataset is processed. For the first... Group samples, calculate reference image samples With the first image sample Number of first feature point matching pairs between The calculation formula is:

[0037] ;

[0038] Simultaneously, calculate reference image samples. With the second image sample Number of second feature point matching pairs between The calculation formula is:

[0039] ;

[0040] Feature point extraction and matching algorithm The Scale-Invariant Feature Transform (SIFT) algorithm is used for feature extraction, and the nearest neighbor ratio method is used to remove mismatches in order to obtain a high number of high-confidence matching pairs.

[0041] S104, constructs a global feature point matching pair threshold through statistical averaging. For all The mean of the number of matching pairs of the first feature points obtained from the group of samples is used to obtain the value corresponding to the first angle threshold. First feature point matching pair number threshold The mean of the number of matching pairs of the second feature points calculated for all groups of samples is used to obtain the value corresponding to the second angle threshold. Second feature point matching pair quantity threshold The calculation formula is as follows:

[0042] ;

[0043] ;

[0044] Through the above calculations, a mapping relationship was established between the offset angle in physical space and the number of feature point matching pairs in image space. Since the geometric differences between images increase with the offset angle, leading to a decrease in the number of effectively matching feature points, the numerical relationship satisfies... The threshold setting module calculates the threshold number of matching pairs for the first feature point. Second feature point matching pair number threshold Stored in system memory, it serves as a benchmark parameter for determining the degree of offset during subsequent online operation. When the first angle threshold is adjusted... Or the second angle threshold When this happens, the system re-executes steps S102 to S104 to update the corresponding threshold for the number of matching pairs.

[0045] Step S200 involves acquiring the current image captured by the camera at preset time intervals using the image acquisition module, and retrieving the pre-stored reference image of the corresponding camera. This includes the following sub-steps:

[0046] S201, Configure system operating parameters and reference baseline. During the deployment phase of the power equipment monitoring system, establish the standard monitoring pose of the camera. This pose must ensure that the camera's field of view completely covers the insulator, conductor, tower, or transformer assembly to be monitored. Control the camera in the standard monitoring pose to acquire images and mark these images as reference images. Reference image It is stored in the system's database or local storage unit, serving as the geometric reference for subsequent offset calculations. Simultaneously, the sampling time interval parameter is set. This parameter is set according to the real-time requirements of power inspection and the load capacity of the computing unit, for example, it can be set to any value between 5 seconds and 30 minutes.

[0047] S202 executes periodic image acquisition tasks. The timing unit built into the image acquisition module or an external trigger determines the timing based on the sampling time interval parameter. Generate a data acquisition command. In response to this command, the camera captures a monitoring image of the power equipment at the current moment, generating the current image. Current image Compared with reference image They have the same resolution and color space format, or are uniformly converted into grayscale image matrices before subsequent processing to reduce computational complexity.

[0048] S203, Construct the image pair to be processed. The image acquisition module retrieves the reference image from the storage unit. and will capture the current image in real time. Compared with reference image Composition of image pairs The image, along with the camera's internal parameter matrix data, is synchronously transmitted to the feature point extraction and matching module for subsequent feature analysis. The camera's internal parameter matrix, including focal length, principal point coordinates, and distortion coefficients, is pre-obtained using the Zhang Zhengyou calibration method or other camera calibration techniques.

[0049] Step S300 involves using the scale-invariant feature transform algorithm to perform feature point detection and descriptor generation on the reference image and the current image through the feature point extraction and matching module, calculating feature point matching pairs between the reference image and the current image, and counting the number of feature point matching pairs. Specifically, this includes the following sub-steps:

[0050] S301, Constructs a Gaussian difference pyramid to detect scale-space extrema. The feature point extraction and matching module receives the reference image. and the current image Scale-space mapping is performed on two images separately. Gaussian pyramids are generated by convolving the images with Gaussian kernels of different scales. A difference-of-Gaussian pyramid is then constructed by calculating the difference between adjacent layers within the Gaussian pyramid. (Gaussian difference function) The definition is as follows:

[0051] ;

[0052] in, Image pixel coordinates, For scale space factor, This represents the scaling factor between adjacent scale layers. For the input image, It is a scale-variable Gaussian function. This represents the convolution operation. In the constructed Gaussian difference pyramid, the extreme value relationship between each sampling point and its neighboring pixels in the same and adjacent scale layers is detected. Local extreme points are identified as candidate keypoints, and low-contrast points and edge response points are removed to obtain a stable set of SIFT keypoints.

[0053] S302, Generate rotation-invariant feature descriptors. Based on the gradient direction distribution of the detected keypoint's neighboring pixels, determine the principal direction of each keypoint to achieve rotation invariance. Define a neighborhood window of a preset size (e.g., 16×16 pixels) around the keypoint, dividing this window into multiple sub-regions (e.g., 4×4 sub-regions), and calculate the gradient histogram of each sub-region in eight directions. Combine the above gradient information to generate a 128-dimensional vector as the feature descriptor for that keypoint. Reference images are then obtained. Feature descriptor subset and the current image Feature descriptor subset .

[0054] S303: Perform feature point matching and mismatch removal, and count the number of matches. Euclidean distance is used as the similarity metric, and the feature descriptor subset of the reference image is traversed. Each feature descriptor in In the current image feature descriptor subset Find the feature descriptor with the closest Euclidean distance. and the second nearest feature descriptor Calculate the nearest distance. and the second closest distance The nearest neighbor ratio method is used to filter the matching results, and the filtering criteria satisfy the following formula:

[0055] ;

[0056] in, The preset rejection threshold ranges from 0.6 to 0.8. If the above conditions are met, then it is considered... This represents a valid feature point matching pair. To facilitate subsequent quantification of the offset, the entire process of SIFT-based dual-view feature point extraction and matching is uniformly defined as an algorithm function. The corresponding processing procedure can be formalized as follows:

[0057] ;

[0058] in, This refers to the number of feature point matching pairs output after processing by the above algorithm. As a core indicator for measuring geometric consistency between images, it is transmitted to the camera offset determination module for subsequent classification. For the specific implementation details of the SIFT algorithm and the Euclidean distance calculation method, those skilled in the art can refer to relevant computer vision literature; these will not be elaborated upon here.

[0059] Step S400, which uses a camera offset determination module to compare the number of feature point matching pairs with a threshold number of feature point matching pairs to determine the current offset level of the camera, includes the following sub-steps:

[0060] S401, acquire the judgment benchmark parameters and real-time measurement data. The camera offset judgment module reads the first feature point matching pair quantity threshold, which has been pre-calculated and stored by the threshold setting module, from the system storage unit. Second feature point matching pair number threshold Simultaneously, it receives the real-time number of feature point matching pairs between the current image and the reference image, output by the feature point extraction and matching module. Based on the mapping principle established by the preceding steps, the smaller the geometric viewpoint difference between images, the more corresponding feature points are retained. Therefore, in terms of numerical relationships, it usually satisfies... .

[0061] S402, Determine the level of slight offset. The camera offset determination module will match the number of feature point pairs in real time. Number of matching pairs with the first feature point threshold Perform a numerical comparison. If This indicates that there is a rich correspondence between the features of the current image and the reference image, a high degree of overlap in the field of view, and that the geometric transformation is within the linear approximation range. At this point, the camera's offset state is determined to be slight. This state corresponds to a physical offset angle of the camera being less than or equal to a first angle threshold. The situation.

[0062] S403, determine the normal offset level. If the comparison result shows... This indicates that the number of feature point matching pairs is between two thresholds. At this point, although the overlap area between images is reduced and some perspective distortion exists, the number of features is still sufficient to support conventional bundle adjustment calculations. Based on this, the camera offset determination module determines the camera offset state to be normal offset. This state corresponds to the camera's physical offset angle being within the first angle threshold. With the second angle threshold The situation between them.

[0063] S404, determine the severity of the offset. If the comparison result shows... This indicates that there are very few effective matching features between the current image and the reference image, and it is even impossible to establish reliable initial geometric constraints. At this point, the camera's offset state is determined to be severe offset. This state corresponds to a physical offset angle of the camera being greater than or equal to a second angle threshold. In such cases, it usually means that a large-angle rotation or significant loss of field of view has occurred, and directly using traditional dual-view geometric solutions can easily lead to calculation divergence or solution failure.

[0064] S405, Generate and transmit status control signals. Based on the above determination results, the camera offset judgment module generates a corresponding offset level status code or control command. This status code is transmitted to the offset correction module as the decision-making basis for triggering different correction algorithm branches, thereby achieving adaptive processing for different offset levels.

[0065] Step S500 involves the offset correction module selecting the corresponding correction strategy based on the offset level to perform pose calculation and adjustment, specifically including the following sub-steps:

[0066] S501 performs camera pose correction based on the reference image and the current image. When the offset level is determined to be slight, the offset correction module performs correction based on the reference image. With the current image The feature point matching pair set is used to combine the least squares method with the camera's intrinsic parameter matrix. Estimate the essential matrix describing the geometric relationship between two images. For the essential matrix Perform Singular Value Decomposition (SVD) to recover the rotation matrix of the current camera relative to the reference pose. Translation vector Rotation matrix The system decomposes the angles into Euler angles (including horizontal yaw, vertical pitch, and roll) corresponding to the camera's gimbal coordinate system, calculating the angular deviation values ​​for each axis. The offset correction module converts these angular deviation values ​​into stepping pulses or control commands for the gimbal motors, driving the camera to rotate in the opposite direction to compensate for the offset. To ensure correction accuracy, the system performs pixel distance error verification: the corrected image captured by the camera is used as the new current image, and feature matching is performed again with the reference image. The image with the highest confidence level is selected. For each feature matching pair, calculate the average pixel distance error. For the first There are 3 matching pairs, and the coordinates of the feature points in the reference image are . The coordinates of the feature points in the current image are Single-point pixel distance error The calculation formula is:

[0067] ;

[0068] Before calculation Mean error of each matching pair The mean error With the preset convergence threshold Compare. If If the correction is complete, then the correction is considered complete; if If the error is not satisfied, the iterative optimization process will begin, and the latest current image will be used as input to continue executing the above essential matrix estimation and pose recovery steps until the error meets the convergence condition.

[0069] S502 performs multi-view joint correction of camera pose based on the reference image and multiple historical view images. When the offset level is determined to be a normal offset, the offset correction module constructs a multi-view image set. This collection contains reference images. and historical image sequences captured by cameras The offset correction module does not rely solely on the latest frame for calculation; instead, it employs a bundle adjustment algorithm to adjust the set. Joint pose estimation is performed on all images. First, the initial pose between each current image and the reference image is calculated. Then, a global optimization objective function is constructed, simultaneously optimizing the camera pose parameters and spatial 3D feature point coordinates from all viewpoints to minimize the reprojection error. The optimization objective function is expressed as:

[0070] ;

[0071] in, Indicates the first Projection matrices for each viewpoint (including rotation and translation parameters). Indicates the first The three-dimensional spatial coordinates of each feature point Indicates the first The feature point at the th ... The coordinates of the observed pixels on the image. Let L2 be the square of the L2 norm. By solving the above nonlinear least squares problem, the precise pose parameters after multi-view geometric constraint correction are obtained, and the camera is corrected and controlled accordingly. This method utilizes the redundant information of historical viewpoints to compensate for the uneven feature distribution of single-frame images under large angular shifts.

[0072] S503 performs multi-view joint correction of the camera pose based on the reference image, the current image, and additional view images. When the offset level is determined to be severe, effective geometric constraints cannot be directly established because the reference image and the current image lack sufficient overlapping fields of view. The offset correction module controls the camera to perform an active rotational scanning action along a preset path (such as a spiral scan centered on the current position or a horizontal panoramic scan), acquiring a sequence of scan images containing different angles. The system calculates the relationship between each frame in the scan image sequence and the reference image. The number of feature matches is used to filter out the top images with the highest matching degree to the reference image. Frame images, serving as intermediate transitional viewpoints, together with the current severely offset image, constitute a multi-view image set. Based on this, the multi-view collaborative optimization strategy of step S502 is used to establish a geometric relationship between the severely offset image and the reference image through the transfer effect of intermediate viewpoints, jointly calculating the absolute offset of the camera and completing the correction. For the specific mathematical solution process of the bundle adjustment algorithm (such as the Levenberg-Marquardt algorithm), those skilled in the art can refer to relevant computer vision literature for implementation, and will not be elaborated here.

Claims

1. An adaptive multi-view collaborative camera offset correction method, characterized in that, Includes the following steps: Based on the offset angle between the reference image and the image sample and the number of feature point matching pairs between the reference image and the image sample, a mapping relationship between the offset angle and the number of feature point matching pairs is constructed. The degree of offset is divided into levels based on a preset offset angle threshold, and the number of feature point matching pairs used to determine the degree of offset is calculated based on the mapping relationship using the offset angle threshold; wherein, the degree of offset includes slight offset, normal offset and severe offset; Acquire the current image captured by the camera and retrieve the pre-stored reference image corresponding to the camera; Feature points are extracted from the reference image and the current image and matched to calculate the number of feature point matching pairs; The number of feature point matching pairs is compared with a threshold number of feature point matching pairs to determine the offset level, and pose adjustment is performed based on the offset level. If the offset is determined to be slight, then perform camera pose correction based on the reference image and the current image; If it is determined to be a normal offset, then perform multi-view joint correction of the camera pose based on the reference image and multiple historical view images; If a severe offset is determined, the camera is controlled to actively acquire additional viewpoint images, and multi-view joint correction of the camera pose is performed based on the reference image, the current image, and the additional viewpoint images.

2. The adaptive multi-view collaborative camera offset correction method according to claim 1, characterized in that, The process of constructing a mapping relationship between the offset angle and the number of feature point matching pairs between the reference image and the image sample, and calculating the threshold for the number of feature point matching pairs used to determine the degree of offset based on the offset angle threshold, includes: Set a first angle threshold and a second angle threshold, with the first angle threshold being less than the second angle threshold; use the first angle threshold and the second angle threshold to classify the degree of offset into three levels: slight offset, normal offset, and severe offset. Construct a sample set containing multiple sets of samples. Each set of samples includes a reference image sample when the camera has not shifted, a first image sample when the camera rotates relative to the reference image sample by a first angle threshold, and a second image sample when the camera rotates relative to the reference image sample by a second angle threshold. Calculate the number of first feature point matching pairs between the reference image sample and the first image sample in each group of samples, and the number of second feature point matching pairs between the reference image sample and the second image sample. The first matching pair threshold is obtained by averaging the number of matching pairs of the first feature points in all groups of samples, and the second matching pair threshold is obtained by averaging the number of matching pairs of the second feature points in all groups of samples.

3. The adaptive multi-view collaborative camera offset correction method according to claim 1, characterized in that, The step of acquiring the current image captured by the camera and retrieving a pre-stored reference image corresponding to the camera includes: Establish a standard monitoring pose for the camera, control the camera in the standard monitoring pose to acquire images, and mark the images as reference images; The image acquisition module generates an acquisition command based on a preset sampling time interval parameter. The camera responds to the acquisition command by capturing a monitoring image of the power equipment at the current moment and generating the current image. The reference image is retrieved from the storage unit, and the current image and the reference image are combined to form an image pair and transmitted to the feature point extraction and matching module.

4. The adaptive multi-view collaborative camera offset correction method according to claim 1, characterized in that, The step of extracting feature points from the reference image and the current image and matching them to calculate the number of feature point matching pairs includes: Construct a Gaussian difference pyramid to detect scale-space extrema and identify them as candidate key points; Generate rotation-invariant feature descriptors based on the gradient direction distribution of neighboring pixels of candidate keypoints; The similarity between the feature descriptors of the current image and the reference image is measured by Euclidean distance. The nearest neighbor ratio method is used to select feature point matching pairs that meet the preset removal threshold conditions, and the number of feature point matching pairs is counted.

5. The adaptive multi-view collaborative camera offset correction method according to claim 2, characterized in that, The comparison of the number of matching feature point pairs with a threshold number of matching feature point pairs to determine the degree of offset includes: Read the first matching pair number threshold and the second matching pair number threshold, and receive the feature point matching pair number; When the number of feature point matching pairs is greater than or equal to the threshold of the number of first matching pairs, the camera offset state is determined to be slight offset. When the number of feature point matching pairs is less than the first matching pair number threshold but greater than the second matching pair number threshold, the camera's offset state is determined to be normal offset. When the number of feature point matching pairs is less than or equal to the second matching pair number threshold, the camera's offset state is determined to be severe offset.

6. The adaptive multi-view collaborative camera offset correction method according to claim 1, characterized in that, If the deviation is determined to be slight, then performing camera pose correction based on the reference image and the current image includes: The essential matrix describing the geometric relationship between the reference image and the current image is estimated by using the least squares method combined with the camera's intrinsic parameter matrix. The essential matrix is ​​decomposed to recover the rotation matrix and translation vector of the current camera relative to the reference pose, and the camera is controlled based on the rotation matrix to counteract the offset. The corrected and adjusted camera image is used as the new current image and is used to perform feature matching with the reference image. The average pixel distance error of the feature matching pair is calculated. If the average pixel distance error is less than the preset convergence threshold, the correction is considered complete; if the average pixel distance error is greater than or equal to the preset convergence threshold, the iterative optimization process begins.

7. The adaptive multi-view collaborative camera offset correction method according to claim 1, characterized in that, If the offset is determined to be ordinary, then performing multi-view joint correction of the camera pose based on the reference image and multiple historical view images includes: Construct a multi-view image set that includes a reference image and a sequence of historical images captured by the camera; The bundle adjustment algorithm is used to perform joint pose estimation on all images in a multi-view image set; A global optimization objective function is constructed to minimize the reprojection error. At the same time, the camera pose parameters and spatial 3D feature point coordinates of all viewpoints are optimized to obtain the corrected pose parameters and to perform corrective control on the camera accordingly.

8. The adaptive multi-view collaborative camera offset correction method according to claim 7, characterized in that, If a severe offset is determined, the step of controlling the camera to actively acquire additional viewpoint images and performing multi-view joint correction of the camera pose based on the reference image, the current image, and the additional viewpoint images includes: Control the camera to perform an active rotation scanning action along a preset path to acquire a sequence of scanned images; Calculate the number of feature matches between each frame in the scanned image sequence and the reference image, and select images that meet the condition of the number of feature matches as intermediate transitional viewpoints; The intermediate transition viewpoint and the current severely offset image are combined to form a multi-view image set. A multi-view collaborative optimization correction strategy is used to establish the geometric relationship between the severely offset image and the reference image and complete the correction.

9. The adaptive multi-view collaborative camera offset correction method according to claim 6, characterized in that, The method of controlling the camera based on a rotation matrix to compensate for the offset includes: The rotation matrix is ​​decomposed into the horizontal yaw angle, vertical pitch angle, and roll angle corresponding to the camera gimbal coordinate system; Calculate the angular deviations of the horizontal yaw angle, vertical pitch angle, and roll angle, and convert the angular deviations into control commands for the gimbal motors; The camera is driven to rotate in the opposite direction to compensate for the angular deviation.

10. An adaptive multi-view collaborative camera offset correction system, applied to the adaptive multi-view collaborative camera offset correction method according to any one of claims 1-9, characterized in that, The system includes: The threshold setting module is used to construct a mapping relationship between the offset angle and the number of feature point matching pairs based on the offset angle between the reference image and the image sample and the number of feature point matching pairs between the reference image and the image sample; it divides the offset degree level according to the preset offset angle threshold, and calculates the feature point matching pair number threshold used to determine the offset degree level based on the mapping relationship; wherein, the offset degree level includes slight offset, normal offset and severe offset. The image acquisition module is used to acquire the current image captured by the camera and retrieve a pre-stored reference image corresponding to the camera. The feature point extraction and matching module is used to extract feature points from the reference image and the current image and perform matching to calculate the number of feature point matching pairs; The camera offset determination module is used to compare the number of feature point matching pairs with a threshold number of feature point matching pairs to determine the offset level. The offset correction module is used to perform pose adjustment according to the degree of offset. If it is determined to be a slight offset, it performs camera pose correction based on the reference image and the current image. If it is determined to be a normal offset, it performs multi-view joint correction of camera pose based on the reference image and multiple historical view images. If it is determined to be a severe offset, it controls the camera to actively acquire additional view images and performs multi-view joint correction of camera pose based on the reference image, the current image and the additional view images.

Images