Pan-tilt camera deviation correction method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a correction method and device for a pan-tilt camera, electronic equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a first preset position of the pan-tilt camera, a first focal length list, and shooting parameters and reference images of a plurality of preset focal lengths; under the condition that the pan-tilt camera is erected at the first preset position, selecting a current preset focal length to be processed from the first focal length list according to a sequence from small to large, and acquiring an image of the target scene by using the pan-tilt camera to obtain a to-be-detected image; comparing the to-be-detected image with the reference image to obtain a deviation angle of the PTZ camera under the preset focal length, and correcting the PTZ camera based on the deviation angle; the current to-be-processed preset focal lengths are selected from the first focal length list according to the sequence from small to large until the PTZ camera is subjected to deviation correction under all the preset focal lengths. According to the embodiment of the invention, the correction effect of the PTZ camera is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a deviation correction method, device, electronic equipment and storage medium for a pan / tilt camera. Background Art

[0002] PTZ cameras are often used in industrial inspection scenes to conduct long-term, large-scale environmental monitoring. For example, long-term monitoring of substations to ensure safe and stable operation. During the use of PTZ cameras, it is often necessary to correct the deviation of the cameras so that they remain stable during the acquisition process and can acquire images of better quality.

[0003] In the related art, although the common deflection correction processing method can achieve a certain deflection correction effect, it is easily affected by the magnification effect when shooting objects that are far away and small in size, and the deflection correction effect is poor. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device, electronic device and storage medium for correcting the deviation of a pan-tilt camera, so as to improve the correction effect of the pan-tilt camera. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for correcting a PTZ camera, comprising:

[0006] Acquire a first preset position of the pan / tilt camera, a first focal length list corresponding to the first preset position, and shooting parameters and a reference image corresponding to each of a plurality of preset focal lengths in the first focal length list;

[0007] When the pan / tilt camera is set at the first preset position, the preset focal length to be processed is selected from the first focal length list in ascending order, and an image of the target scene is captured by the pan / tilt camera based on the preset focal length and the corresponding shooting parameters to obtain an image to be detected corresponding to the preset focal length;

[0008] Comparing the image to be detected with a reference image corresponding to the preset focal length, obtaining an offset angle of the pan / tilt camera at the preset focal length, and correcting the pan / tilt camera based on the offset angle;

[0009] Return to the execution step: select the preset focal length to be processed in the first focal length list in order from small to large, use the pan-tilt camera to capture images of the target scene based on the preset focal length and the corresponding shooting parameters, and obtain the image to be detected corresponding to the preset focal length, until the pan-tilt camera has completed the deviation correction at each of the preset focal lengths in the first focal length list.

[0010] In one embodiment of the present application, the comparing the image to be detected with the reference image corresponding to the preset focal length, obtaining the offset angle of the pan / tilt camera at the preset focal length, and correcting the pan / tilt camera based on the offset angle includes:

[0011] Performing feature point matching on the image to be detected and the reference image corresponding to the preset focal length to obtain a first feature point pair, and obtaining a homography matrix between the image to be detected and the reference image based on the first feature point pair;

[0012] Calculating the offset of the pan-tilt camera in the translation dimension and the offset of the pitch dimension at the preset focal length according to the homography matrix;

[0013] Calculating the offset angle of the pan-tilt camera in the pan-tilt dimension according to the offset in the pan-tilt dimension, and calculating the offset angle of the pan-tilt camera in the pitch dimension according to the offset in the pitch dimension;

[0014] Based on the offset angle of the translation dimension, the pan-tilt camera is corrected in the translation dimension, and based on the offset angle of the pitch dimension, the pan-tilt camera is corrected in the pitch dimension.

[0015] In one embodiment of the present application, the calculating the offset angle of the pan-tilt camera in the pan-tilt dimension according to the offset in the pan-tilt dimension, and the calculating the offset angle of the pan-tilt camera in the pitch dimension according to the offset in the pitch dimension, include:

[0016] Calculating the offset angle of each pixel in the image to be detected in the translation dimension according to the offset in the translation dimension;

[0017] Based on the offset angle of each pixel in the translation dimension, obtaining the offset angle of the pan / tilt camera in the translation dimension;

[0018] Calculating the offset angle of each pixel in the image to be detected in the pitch dimension according to the offset in the pitch dimension;

[0019] Based on the offset angle of each pixel in the pitch dimension, the offset angle of the pan / tilt camera in the pitch dimension is obtained.

[0020] In one embodiment of the present application, the offset angle of the translation dimension is calculated according to the following formula:

[0021]

[0022] Wherein, ap represents the offset angle of the translation dimension, represents the offset of the translation dimension, Indicates the offset for every 1° offset, Indicates the offset of each pixel by 1°;

[0023] The offset angle of the pitch dimension is calculated according to the following formula:

[0024]

[0025] Wherein, at represents the offset angle of the pitch dimension, represents the offset of the pitch dimension, Indicates the offset for every 1° offset, Each pixel is shifted by 1°.

[0026] In one embodiment of the present application, after correcting the PTZ camera based on the offset angle, the method further includes:

[0027] Calling the shooting parameters corresponding to the preset focal length for the pan-tilt camera, and detecting whether the captured picture of the pan-tilt camera under the shooting parameters is stable;

[0028] When the captured image is stable, it is determined that the deviation correction of the PTZ camera is completed at the current preset focal length.

[0029] In one embodiment of the present application, after the PTZ camera has completed deviation correction at each of the preset focal lengths in the first focal length list, the method further includes:

[0030] Using the PTZ camera after the correction is completed to collect images of the target scene at the preset focal length to obtain an image to be analyzed;

[0031] Performing feature point matching on the image to be analyzed and the reference image corresponding to the preset focal length to obtain a second feature point pair;

[0032] Using a thin plate spline interpolation algorithm, based on the second feature point pair, a coordinate mapping relationship between the image to be analyzed and the reference image is calculated;

[0033] Obtaining an offset of the image to be analyzed according to the coordinate mapping relationship and the second feature point pair;

[0034] The image to be analyzed is deflected based on the offset of the image to be analyzed to obtain a target image.

[0035] In a second aspect, an embodiment of the present application provides a deviation correction device for a pan / tilt camera, comprising:

[0036] A data acquisition module, used to acquire a first preset position of a pan / tilt camera, a first focal length list corresponding to the first preset position, and shooting parameters and a reference image corresponding to each of a plurality of preset focal lengths in the first focal length list;

[0037] A first image acquisition module is used to select the preset focal length to be processed in the first focal length list in ascending order when the pan-tilt camera is set at the first preset position, and use the pan-tilt camera to capture an image of the target scene based on the preset focal length and the corresponding shooting parameters to obtain an image to be detected corresponding to the preset focal length;

[0038] A correction module, used for comparing the image to be detected with a reference image corresponding to the preset focal length, obtaining an offset angle of the pan-tilt camera at the preset focal length, and correcting the pan-tilt camera based on the offset angle;

[0039] Return to the execution module, used to return to the execution step: select the preset focal length currently to be processed in the first focal length list in order from small to large, use the pan-tilt camera to capture images of the target scene based on the preset focal length and the corresponding shooting parameters, and obtain the image to be detected corresponding to the preset focal length, until the pan-tilt camera completes the deviation correction at each of the preset focal lengths in the first focal length list.

[0040] In one embodiment of the present application, the correction module includes:

[0041] a homography matrix obtaining submodule, configured to perform feature point matching between the image to be detected and the reference image corresponding to the preset focal length to obtain a first feature point pair, and obtain a homography matrix between the image to be detected and the reference image based on the first feature point pair;

[0042] An offset calculation submodule, used to calculate the offset of the pan-tilt camera in the translation dimension and the offset of the pitch dimension at the preset focal length according to the homography matrix;

[0043] An offset angle calculation submodule, used to calculate the offset angle of the pan-tilt camera in the translation dimension according to the offset in the translation dimension, and to calculate the offset angle of the pan-tilt camera in the pitch dimension according to the offset in the pitch dimension;

[0044] The correction submodule is used to correct the translation dimension of the pan-tilt camera based on the offset angle of the translation dimension, and to correct the pitch dimension of the pan-tilt camera based on the offset angle of the pitch dimension.

[0045] In one embodiment of the present application, the offset angle calculation submodule is specifically used to:

[0046] Calculating the offset angle of each pixel in the image to be detected in the translation dimension according to the offset in the translation dimension;

[0047] Based on the offset angle of each pixel in the translation dimension, obtaining the offset angle of the pan / tilt camera in the translation dimension;

[0048] Calculating the offset angle of each pixel in the image to be detected in the pitch dimension according to the offset in the pitch dimension;

[0049] Based on the offset angle of each pixel in the pitch dimension, the offset angle of the pan / tilt camera in the pitch dimension is obtained.

[0050] In one embodiment of the present application, the device further includes:

[0051] A shooting parameter stability detection module, used to call the shooting parameters corresponding to the preset focal length of the pan-tilt camera, and detect whether the captured picture of the pan-tilt camera under the shooting parameters is stable;

[0052] The deflection correction completion confirmation module is used to determine whether the deflection correction of the pan / tilt camera is completed at the current preset focal length when the collected image is stable.

[0053] In one embodiment of the present application, the device further includes:

[0054] A second image acquisition module is used to acquire an image of the target scene at the preset focal length using the PTZ camera after the correction is completed to obtain an image to be analyzed;

[0055] A feature point matching module, used for matching the feature points of the image to be analyzed with the reference image corresponding to the preset focal length to obtain a second feature point pair;

[0056] An image analysis module, configured to calculate a coordinate mapping relationship between the image to be analyzed and the reference image based on the second feature point pair by using a thin plate spline interpolation algorithm;

[0057] An offset determination module, used for obtaining the offset of the image to be analyzed according to the coordinate mapping relationship and the second feature point pair;

[0058] The image correction module is used to correct the image to be analyzed based on the offset of the image to be analyzed to obtain a target image.

[0059] In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0060] Memory, used to store computer programs;

[0061] The processor is used to implement any of the above-mentioned steps of the method for correcting the deviation of the pan-tilt camera when executing the program stored in the memory.

[0062] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements any of the above-mentioned steps of the method for correcting the deviation of a pan-tilt camera.

[0063] An embodiment of the present invention further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-mentioned methods for correcting the deviation of a pan-tilt camera.

[0064] Beneficial effects of the embodiments of the present invention:

[0065] The embodiments of the present invention provide a method, device, electronic device and storage medium for correcting the deviation of a pan-tilt camera, which obtain a first preset position of the pan-tilt camera, a first focal length list corresponding to the first preset position, and shooting parameters and reference images corresponding to multiple preset focal lengths in the first focal length list; when the pan-tilt camera is set at the first preset position, the current preset focal length to be processed is selected from the first focal length list in ascending order, and the pan-tilt camera is used to capture images of a target scene based on the preset focal length and corresponding shooting parameters to obtain an image to be detected corresponding to the preset focal length; the image to be detected is compared with the reference image corresponding to the preset focal length to obtain the offset angle of the pan-tilt camera at the preset focal length, and the pan-tilt camera is corrected based on the offset angle; and the execution step is returned to: the current preset focal length to be processed is selected from the first focal length list in ascending order, and the pan-tilt camera is used to capture images of the target scene based on the preset focal length and corresponding shooting parameters to obtain an image to be detected corresponding to the preset focal length, until the deviation correction of the pan-tilt camera is completed at each preset focal length in the first focal length list.

[0066] At each preset focal length in the first focal length list of the first preset position, the PTZ camera is iteratively corrected, and the PTZ camera is precisely adjusted in sequence at each preset focal length, thereby achieving a smooth transition of the PTZ camera from a broad field of view to detailed focus. In particular, for targets with small volumes and remote locations in the scene, the accuracy of PTZ camera recognition and monitoring is improved through the gradually refined zoom of each preset focal length in the first focal length list, and the correction effect of the PTZ camera is improved, further achieving the PTZ camera's accurate capture of targets and effective monitoring of scenes.

[0067] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0069] Figure 1 A first flow chart of the method for correcting the deviation of a pan-tilt camera according to an embodiment of the present application;

[0070] Figure 2 A schematic diagram of a possible implementation of step S103 in an embodiment of the present application;

[0071] Figure 3 A schematic diagram of a possible implementation of step S203 in an embodiment of the present application;

[0072] Figure 4 A second flow chart of the method for correcting the deviation of a pan-tilt camera according to an embodiment of the present application;

[0073] Figure 5-1 A third flow chart of the method for correcting the deviation of a pan-tilt camera according to an embodiment of the present application;

[0074] Figure 5-2 A schematic diagram of the process flow of the preparation phase of the deviation correction method of the pan-tilt camera according to an embodiment of the present application;

[0075] Figure 5-3 A schematic diagram of the inspection phase flow of the deviation correction method for the pan-tilt camera of an embodiment of the present application;

[0076] Figure 6 A schematic diagram of the structure of a deviation correction device for a pan / tilt camera according to an embodiment of the present application;

[0077] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0079] Although the correction processing method in the related art can achieve a certain correction effect, the correction effect is poor when shooting objects that are far away and small in size. In order to solve this problem, a correction method for a pan-tilt camera is provided in a first aspect of an embodiment of the present application. The method can be performed by an electronic device. Specifically, the electronic device can be an industrial computer, a personal computer, a cloud server, etc. connected to the pan-tilt camera.

[0080] The following is a detailed description of the method for correcting the deviation of the pan / tilt camera of the present application embodiment. Figure 1 , Figure 1 The first flow chart of the method for correcting the deviation of the pan-tilt camera of the embodiment of the present application includes:

[0081] S101, obtaining a first preset position of a pan / tilt camera, a first focal length list corresponding to the first preset position, and shooting parameters and a reference image corresponding to each of a plurality of preset focal lengths in the first focal length list.

[0082] PTZ cameras are used for long-term, large-scale monitoring of target scenes. The target scenes can be substations, chemical plants, refineries, and other scenes that require industrial inspections.

[0083] The preset position refers to the position where the key area in the target scene can be captured. It is set in advance according to the actual situation of the target scene. Each target scene can have multiple preset positions. During the inspection process, the PTZ camera needs to quickly run to the preset position to capture the image / video of the key area so that the monitoring personnel can quickly view the current situation of the key area.

[0084] Exemplarily, after the PTZ camera is corrected, the shooting status of the PTZ camera corresponding to the preset position (including a focal length list, shooting parameters corresponding to each preset focal length, etc.) is recorded, so that when the PTZ camera runs to the preset position, it can quickly restore the shooting status after correction and collect the required images / videos for key areas.

[0085] Each preset focal length in the first focal length list enables the PTZ camera to capture clear images of objects of various sizes at various distances in the key area of ​​the target scene when the PTZ camera is in the first preset position.

[0086] Specifically, the first focal length list can be an incremental list, in which the first preset focal length is a fixed value (can be 1), the second preset focal length is 1 / 2 of the preset focal length value (for example, if the preset focal length value is 64, the second preset focal length is 32), and the third preset focal length is calculated by first obtaining 1 / 2 of the difference between the second preset focal length value and the first preset focal length value (rounded up), and then superimposing the second preset focal length value to obtain the third preset focal length value (for example, the first preset focal length is 1, the second preset focal length is 32, and the third preset focal length is 48). Starting from the third preset focal length value, each preset focal length is the previous focal length value superimposed with 1 / 2 of the difference between the previous focal length and the previous focal length, until the increment is less than or equal to the preset increment threshold (which can be 8 by default) compared with the previous value. The last preset focal length is the focal length value of the preset position (ZOOM). For example, if the focal length value of the preset position is 64, the first focal length list is [1,32,48,56,64].

[0087] For each preset focal length, obtain the corresponding shooting parameters and reference images. Specifically, the shooting parameters include PTZ (Pan / Tilt / Zoom, pan / tilt all-round (left / right / up / down) movement and lens zoom and focus parameters (focus). The reference image is the standard image captured by the sample camera at the preset focal length according to the corresponding shooting parameters of the target scene.

[0088] S102, when the pan-tilt camera is set at the first preset position, select the preset focal length to be processed in the first focal length list in ascending order, use the pan-tilt camera to capture an image of the target scene based on the preset focal length and corresponding shooting parameters, and obtain the image to be detected corresponding to the preset focal length.

[0089] When the PTZ camera is set up at the first preset position, the preset focal lengths to be processed are selected in order from small to large in the first focal length list (for example, when the first focal length list is [1, 32, 48, 56, 64], in the order of 1-64), so that the PTZ camera captures images of the target scene at the preset focal lengths according to the corresponding shooting parameters, and obtains the image to be detected at each preset focal length.

[0090] S103, comparing the image to be detected with a reference image corresponding to the preset focal length, obtaining an offset angle of the pan / tilt camera at the preset focal length, and correcting the pan / tilt camera based on the offset angle.

[0091] For the preset focal length, the deviation between the image to be detected and the corresponding reference image is compared to obtain the offset angle of the pan-tilt camera at the preset focal length, and the pan-tilt camera is corrected based on the offset angle.

[0092] S104, return to the execution step: select the preset focal length to be processed in the first focal length list in ascending order, use the pan-tilt camera to capture images of the target scene based on the preset focal length and the corresponding shooting parameters, and obtain the image to be detected corresponding to the preset focal length, until the pan-tilt camera completes the deviation correction at each of the preset focal lengths in the first focal length list.

[0093] The above-mentioned deviation correction process of the pan-tilt camera is an iterative process in the first focal length list, that is, first select the first preset focal length, collect the image to be detected of the target scene under its corresponding shooting parameters, compare the image to be detected with the reference image to obtain the offset angle of the pan-tilt camera at the first preset focal length, and correct the deviation of the pan-tilt camera based on the current offset angle. Then select the second preset focal length, collect the image to be detected of the target scene under its corresponding shooting parameters, compare the image to be detected with the reference image to obtain the offset angle of the pan-tilt camera at the second preset focal length, and correct the deviation of the pan-tilt camera based on the current offset angle. Until the deviation correction of the pan-tilt camera is completed at each preset focal length in the first focal length list, the deviation correction of the pan-tilt camera at the first preset position is completed. The deviation correction of each preset position in the target scene is completed in this iterative manner.

[0094] As can be seen from the above, the deviation correction method of the PTZ camera provided in the embodiment of the present application iteratively corrects the PTZ camera at each preset focal length in the first focal length list of the first preset position, and realizes precise control of the PTZ camera at each preset focal length in turn, thereby realizing a smooth transition of the PTZ camera from a wide field of view to detailed focus. In particular, for targets with small volumes and remote locations in the scene, the accuracy of the PTZ camera's recognition and monitoring is improved through the gradually refined zooming of each preset focal length in the first focal length list, the deviation correction effect of the PTZ camera is improved, and the PTZ camera's accurate capture of targets and effective monitoring of scenes are further realized.

[0095] In one possible implementation, see Figure 2 , the comparing the image to be detected with the reference image corresponding to the preset focal length, obtaining the offset angle of the pan-tilt camera at the preset focal length, and correcting the pan-tilt camera based on the offset angle, comprises:

[0096] S201, performing feature point matching on the image to be detected and the reference image corresponding to the preset focal length to obtain a first feature point pair, and obtaining a homography matrix between the image to be detected and the reference image based on the first feature point pair;

[0097] S202, calculating, according to the homography matrix, an offset of the pan-tilt camera in the translation dimension and an offset of the pitch dimension at the preset focal length;

[0098] S203, calculating the offset angle of the pan / tilt camera in the pan / tilt dimension according to the offset in the pan / tilt dimension, and calculating the offset angle of the pan / tilt camera in the pitch dimension according to the offset in the pitch dimension;

[0099] S204, based on the offset angle of the translation dimension, correct the pan-tilt camera in the translation dimension, and based on the offset angle of the pitch dimension, correct the pan-tilt camera in the pitch dimension.

[0100] Based on the neural network feature point matching algorithm, feature point matching is performed on the image to be detected and the reference image to obtain a first feature point pair between the image to be detected and the reference image. Specifically, any feature point matching algorithm in the relevant technology can be used to achieve this. Based on the first feature point pair, a homography matrix between the image to be detected and the reference image is calculated.

[0101] The homography matrix represents the transformation relationship between some points on the common plane between two images. Specifically, the coordinates of the first feature point pair can be normalized, and then the homography matrix can be calculated using the eight-point method (also known as the DLT algorithm). The eight-point method estimates the homography matrix by solving a system of linear equations, usually using SVD (singular value decomposition) to solve it.

[0102] ‌Map the feature points in the reference image to the image to be detected, or map the feature points in the image to be detected to the reference image through the homography matrix. Determine the correspondence between each pair of feature points through the first feature point pair, and calculate the offset between the image to be detected and the reference image based on the deviation between the mapped feature points and the corresponding feature points in the image.

[0103] Specifically, the offset includes the pan dimension and the tilt dimension. According to the offset of each dimension, the offset angle of the dimension is calculated, so that the PTZ camera is corrected from the pan dimension and the tilt dimension according to the offset angle.

[0104] Specifically, the offset angle can be set through the SDK setting (Software Development Kit, device parameter setting) of the PTZ camera, so that the PTZ camera moves a corresponding angle to complete the deviation correction.

[0105] As can be seen from the above, the method for correcting the pan-tilt camera provided in the embodiment of the present application corrects the pan-tilt camera from two dimensions, namely the translation dimension and the pitch dimension, thereby improving the accuracy and rigor of the correction of the pan-tilt camera.

[0106] In one possible implementation, see Figure 3 , calculating the offset angle of the pan-tilt camera in the pan-tilt dimension according to the offset in the pan-tilt dimension, and calculating the offset angle of the pan-tilt camera in the pitch dimension according to the offset in the pitch dimension, including:

[0107] S301, calculating the offset angle of each pixel in the image to be detected in the translation dimension according to the offset in the translation dimension;

[0108] S302, obtaining an offset angle of the pan / tilt camera in the translation dimension based on an offset angle of each pixel in the translation dimension;

[0109] S303, calculating the offset angle of each pixel in the image to be detected in the pitch dimension according to the offset in the pitch dimension;

[0110] S304: Obtain the offset angle of the pan / tilt camera in the pitch dimension based on the offset angle of each pixel in the pitch dimension.

[0111] According to the offset in the translation dimension, the offset angle of each pixel in the image to be detected in the translation dimension is first calculated, and then the offset angle of the image to be detected is calculated, so as to obtain the offset angle of the pan-tilt camera in the translation dimension. According to the offset in the pitch dimension, the offset angle of each pixel in the image to be detected in the pitch dimension is first calculated, and then the offset angle of the image to be detected is calculated, so as to obtain the offset angle of the pan-tilt camera in the pitch dimension.

[0112] In one embodiment of the present application, the offset angle of the translation dimension is calculated according to the following formula:

[0113]

[0114] Wherein, ap represents the offset angle of the translation dimension, represents the offset of the translation dimension, Indicates the offset for every 1° offset, Indicates the offset of each pixel by 1°;

[0115] The offset angle of the pitch dimension is calculated according to the following formula:

[0116]

[0117] Wherein, at represents the offset angle of the pitch dimension, represents the offset of the pitch dimension, Indicates the offset for every 1° offset, Indicates that each pixel is offset by 1°.

[0118] As can be seen from the above, the gimbal camera correction method provided in the embodiment of the present application calculates the offset angle of each pixel of the image to be detected in the two dimensions of translation dimension and pitch dimension, thereby obtaining the offset angle of the gimbal camera, thereby improving the accuracy and rigor of the gimbal camera correction.

[0119] After the PTZ camera is corrected based on the offset angle, in a possible implementation manner, see Figure 4 , the method further comprises:

[0120] S401, calling the shooting parameters corresponding to the preset focal length of the pan-tilt camera, and detecting whether the captured image of the pan-tilt camera under the shooting parameters is stable;

[0121] S402, when the captured image is stable, determining that the deviation correction of the PTZ camera is completed at the current preset focal length.

[0122] After the PTZ camera performs PTZ control, the collected image may be unstable due to mechanical movement, or the image may be blurred due to inaccurate focus. Therefore, after setting the shooting parameters of the PTZ camera (including PTZ and focus), wait for a period of time to check whether the collected image is stable. If the waiting time exceeds the preset time threshold and the parameters are still not stable, the parameters of the PTZ camera are called again. If the number of resets exceeds the preset number threshold, it means that the current correction has failed, and the PTZ camera is corrected again at the current preset focal length. After the collected image of the PTZ camera is stable, it is determined that the correction of the PTZ camera is completed at the current preset focal length.

[0123] As can be seen from the above, the deflection correction method for the pan-tilt camera provided in the embodiment of the present application performs an image stability detection after the pan-tilt camera is corrected, so that the pan-tilt camera determines that the deflection correction is completed when the captured image is stable, thereby reducing the image quality problem caused by the unstable image of the pan-tilt camera when capturing images after the deflection correction, and reduces the repeated deflection correction of the pan-tilt camera due to image quality problems after the image is captured, thereby improving the stability and reliability of image capture, and also improving the accuracy and efficiency of the deflection correction of the pan-tilt camera.

[0124] After the PTZ camera has completed the deviation correction at each of the preset focal lengths in the first focal length list, in a possible implementation manner, see Figure 5-1 , the method further comprises:

[0125] S501, using the PTZ camera after the correction to collect an image of the target scene at the preset focal length to obtain an image to be analyzed;

[0126] S502, performing feature point matching on the image to be analyzed and the reference image corresponding to the preset focal length to obtain a second feature point pair;

[0127] S503, using a thin plate spline interpolation algorithm to calculate a coordinate mapping relationship between the image to be analyzed and the reference image based on the second feature point pair;

[0128] S504, obtaining an offset of the image to be analyzed according to the coordinate mapping relationship and the second feature point pair;

[0129] S505: Correct the image to be analyzed based on the offset of the image to be analyzed to obtain a target image.

[0130] Since the mechanical movement of the PTZ camera is essentially discrete, this may cause a certain deviation in the final captured image. In order to correct this deviation, after the PTZ camera has completed the deviation correction at each preset focal length in the first focal length list, the image to be analyzed is matched with the reference image for feature points to obtain a second feature point pair. Exemplarily, the SuperPoint model algorithm (an image feature point detection algorithm based on a convolutional neural network) can be used to extract and match feature point pairs with scores exceeding a preset score threshold to obtain a second feature point pair.

[0131] Using the Thin Plate Spline (Thin Plate Spline interpolation algorithm), based on the second feature point pair, the transformation parameters between the image to be analyzed and the reference image are obtained, and the coordinate mapping relationship between the image to be analyzed and the reference image is determined.

[0132] The feature points in the reference image are mapped to the image to be analyzed, or the feature points in the image to be analyzed are mapped to the reference image through the coordinate mapping relationship. The corresponding relationship between each pair of feature points is determined through the second feature point pair, and the offset between the image to be analyzed and the reference image is calculated according to the deviation between the mapped feature points and the corresponding feature points in the image. The image to be analyzed is corrected according to the offset to obtain the target image.

[0133] In one embodiment of the present application, the correction method of the PTZ camera is divided into a preparation stage and an inspection stage, such as Figure 5-2 and Figure 5-3As shown. The preparation stage is used to determine each photographing focal length in the first focal length list, and collect the reference map and preset position information (shooting parameters) of each focal length. In the inspection stage, the PTZ corresponding to the focal length is called, and the homography matrix, offset, offset pixel, and offset angle are calculated through the algorithm. The gimbal is adjusted to correct the gimbal, and the shooting parameters of the gimbal after adjustment and correction are checked to see whether they are stable, and whether the correction is completed. Until all focal lengths in the first focal length list are corrected. Use the gimbal camera after correction to capture images, correct the offset of the image, and obtain the target image. The embodiment of the present application can improve the quality and stability of the image during the acquisition stage. In the substation scenario, there is a significant improvement in the meter reading algorithm for some smaller meters and the algorithm recognition performance of switch cabinet components.

[0134] It can be seen from the above that the deviation correction method for the pan-tilt camera provided in the embodiment of the present application corrects the image capture deviation that may be caused by discrete mechanical movement of the pan-tilt camera, and further improves the accuracy and stability of image acquisition of the pan-tilt camera.

[0135] The present application also provides a device for correcting the deviation of a pan-tilt camera, see Figure 6 ,include:

[0136] The data acquisition module 601 is used to acquire a first preset position of the PTZ camera, a first focal length list corresponding to the first preset position, and shooting parameters and a reference image corresponding to each of a plurality of preset focal lengths in the first focal length list;

[0137] The first image acquisition module 602 is used to select the preset focal length to be processed in the first focal length list in ascending order when the pan-tilt camera is set at the first preset position, and use the pan-tilt camera to capture an image of the target scene based on the preset focal length and the corresponding shooting parameters to obtain an image to be detected corresponding to the preset focal length;

[0138] A correction module 603 is used to compare the image to be detected with a reference image corresponding to the preset focal length, obtain an offset angle of the pan / tilt camera at the preset focal length, and correct the pan / tilt camera based on the offset angle;

[0139] Return to execution module 604, which is used to return to the execution step: select the preset focal length to be processed in the first focal length list in order from small to large, use the pan-tilt camera to capture images of the target scene based on the preset focal length and the corresponding shooting parameters, and obtain the image to be detected corresponding to the preset focal length, until the pan-tilt camera has completed the deviation correction at each of the preset focal lengths in the first focal length list.

[0140] As can be seen from the above, the correction device for the pan-tilt camera provided in the embodiment of the present application iteratively corrects the pan-tilt camera at each preset focal length in the first focal length list of the first preset position, and realizes precise control of the pan-tilt camera PTZ in sequence at each preset focal length, thereby realizing a smooth transition of the pan-tilt camera from a wide field of view to detailed focus. In particular, for targets with small volumes and remote locations in the scene, the accuracy of pan-tilt camera recognition and monitoring is improved through the gradually refined zooming of each preset focal length in the first focal length list, the correction effect of the pan-tilt camera is improved, and the pan-tilt camera is further realized to accurately capture the target and effectively monitor the scene.

[0141] In a possible implementation, the correction module 603 includes:

[0142] a homography matrix obtaining submodule, configured to perform feature point matching between the image to be detected and the reference image corresponding to the preset focal length to obtain a first feature point pair, and obtain a homography matrix between the image to be detected and the reference image based on the first feature point pair;

[0143] An offset calculation submodule, used to calculate the offset of the pan-tilt camera in the translation dimension and the offset of the pitch dimension at the preset focal length according to the homography matrix;

[0144] An offset angle calculation submodule, used to calculate the offset angle of the pan-tilt camera in the translation dimension according to the offset in the translation dimension, and to calculate the offset angle of the pan-tilt camera in the pitch dimension according to the offset in the pitch dimension;

[0145] The correction submodule is used to correct the translation dimension of the pan-tilt camera based on the offset angle of the translation dimension, and to correct the pitch dimension of the pan-tilt camera based on the offset angle of the pitch dimension.

[0146] As can be seen from the above, the correction device for the pan-tilt camera provided in the embodiment of the present application corrects the pan-tilt camera from two dimensions, namely the translation dimension and the pitch dimension, thereby improving the accuracy and rigor of the correction of the pan-tilt camera.

[0147] In a possible implementation manner, the offset angle calculation submodule is specifically used to:

[0148] Calculating the offset angle of each pixel in the image to be detected in the translation dimension according to the offset in the translation dimension;

[0149] Based on the offset angle of each pixel in the translation dimension, obtaining the offset angle of the pan / tilt camera in the translation dimension;

[0150] Calculating the offset angle of each pixel in the image to be detected in the pitch dimension according to the offset in the pitch dimension;

[0151] Based on the offset angle of each pixel in the pitch dimension, the offset angle of the pan / tilt camera in the pitch dimension is obtained.

[0152] As can be seen from the above, the correction device for the pan-tilt camera provided in the embodiment of the present application calculates the offset angle of each pixel of the image to be detected in two dimensions, namely the translation dimension and the pitch dimension, so as to obtain the offset angle of the pan-tilt camera, thereby improving the accuracy and rigor of the correction of the pan-tilt camera.

[0153] In a possible implementation, the device further includes:

[0154] A shooting parameter stability detection module is used to call the shooting parameters corresponding to the preset focal length to be processed for the pan-tilt camera, and detect whether the captured picture of the pan-tilt camera under the shooting parameters is stable;

[0155] The deflection correction completion confirmation module is used to determine whether the deflection correction of the pan / tilt camera is completed at the current preset focal length when the collected image is stable.

[0156] As can be seen from the above, the deflection correction device for the pan-tilt camera provided in the embodiment of the present application performs image stability detection after the pan-tilt camera is corrected, so that the pan-tilt camera determines that the deflection correction is completed when the captured image is stable, thereby reducing the image quality problem caused by the unstable image of the pan-tilt camera when capturing images after the deflection correction, and reduces the repeated deflection correction of the pan-tilt camera due to image quality problems after the image is captured, thereby improving the stability and reliability of image capture, and also improving the accuracy and efficiency of the deflection correction of the pan-tilt camera.

[0157] In a possible implementation, the device further includes:

[0158] A second image acquisition module is used to acquire an image of the target scene at the preset focal length using the PTZ camera after the correction is completed to obtain an image to be analyzed;

[0159] A feature point matching module, used for matching the feature points of the image to be analyzed with the reference image corresponding to the preset focal length to obtain a second feature point pair;

[0160] An image analysis module, configured to calculate a coordinate mapping relationship between the image to be analyzed and the reference image based on the second feature point pair by using a thin plate spline interpolation algorithm;

[0161] An offset determination module, used for obtaining the offset of the image to be analyzed according to the coordinate mapping relationship and the second feature point pair;

[0162] The image correction module is used to correct the image to be analyzed based on the offset of the image to be analyzed to obtain a target image.

[0163] It can be seen from the above that the deviation correction device for the pan-tilt camera provided in the embodiment of the present application corrects the image capture deviation that may be caused by discrete mechanical movement of the pan-tilt camera, and further improves the accuracy and stability of image acquisition of the pan-tilt camera.

[0164] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701 , a communication interface 702 , a memory 703 and a communication bus 704 , wherein the processor 701 , the communication interface 702 , and the memory 703 communicate with each other via the communication bus 704 .

[0165] Memory 703, used for storing computer programs;

[0166] The processor 701 is used to implement any of the above-mentioned methods for correcting the deviation of the pan-tilt camera when executing the program stored in the memory 703.

[0167] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0168] The communication interface is used for communication between the above electronic device and other devices.

[0169] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0170] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0171] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for correcting the deviation of any pan-tilt camera are implemented.

[0172] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the deviation correction method for a pan-tilt camera in any of the above embodiments.

[0173] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0174] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0175] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0176] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for correcting the deviation of a pan / tilt camera, characterized in that: include: Acquire a first preset position of the pan / tilt camera, a first focal length list corresponding to the first preset position, and shooting parameters and a reference image corresponding to each of a plurality of preset focal lengths in the first focal length list; When the pan / tilt camera is set at the first preset position, the preset focal length to be processed is selected from the first focal length list in ascending order, and an image of the target scene is captured by the pan / tilt camera based on the preset focal length and the corresponding shooting parameters to obtain an image to be detected corresponding to the preset focal length; Comparing the image to be detected with a reference image corresponding to the preset focal length, obtaining an offset angle of the pan / tilt camera at the preset focal length, and correcting the pan / tilt camera based on the offset angle; Return to the execution step: select the preset focal length to be processed in the first focal length list in order from small to large, use the pan-tilt camera to capture images of the target scene based on the preset focal length and the corresponding shooting parameters, and obtain the image to be detected corresponding to the preset focal length, until the pan-tilt camera has completed the deviation correction at each of the preset focal lengths in the first focal length list.

2. The method according to claim 1, characterized in that The comparing the image to be detected with the reference image corresponding to the preset focal length to obtain the offset angle of the pan / tilt camera at the preset focal length, and correcting the pan / tilt camera based on the offset angle, comprises: Performing feature point matching on the image to be detected and the reference image corresponding to the preset focal length to obtain a first feature point pair, and obtaining a homography matrix between the image to be detected and the reference image based on the first feature point pair; Calculating the offset of the pan-tilt camera in the translation dimension and the offset of the pitch dimension at the preset focal length according to the homography matrix; Calculating the offset angle of the pan-tilt camera in the pan-tilt dimension according to the offset in the pan-tilt dimension, and calculating the offset angle of the pan-tilt camera in the pitch dimension according to the offset in the pitch dimension; Based on the offset angle of the translation dimension, the pan-tilt camera is corrected in the translation dimension, and based on the offset angle of the pitch dimension, the pan-tilt camera is corrected in the pitch dimension.

3. The method according to claim 2, characterized in that The step of calculating the offset angle of the pan-tilt camera in the pan-tilt dimension according to the offset in the pan-tilt dimension, and calculating the offset angle of the pan-tilt camera in the pitch dimension according to the offset in the pitch dimension, comprises: Calculating the offset angle of each pixel in the image to be detected in the translation dimension according to the offset in the translation dimension; Based on the offset angle of each pixel in the translation dimension, obtaining the offset angle of the pan / tilt camera in the translation dimension; Calculating the offset angle of each pixel in the image to be detected in the pitch dimension according to the offset in the pitch dimension; Based on the offset angle of each pixel in the pitch dimension, the offset angle of the pan / tilt camera in the pitch dimension is obtained.

4. The method according to claim 3, characterized in that The offset angle of the translation dimension is calculated according to the following formula: ; Wherein, ap represents the offset angle of the translation dimension, represents the offset of the translation dimension, Indicates the offset for every 1° offset, Indicates the offset of each pixel by 1°; The offset angle of the pitch dimension is calculated according to the following formula: ; Wherein, at represents the offset angle of the pitch dimension, represents the offset of the pitch dimension, Indicates the offset for every 1° offset, Indicates that each pixel is offset by 1°.

5. The method according to claim 1, characterized in that After correcting the deviation of the pan / tilt camera based on the offset angle, the method further includes: Calling the shooting parameters corresponding to the preset focal length for the pan-tilt camera, and detecting whether the captured picture of the pan-tilt camera under the shooting parameters is stable; When the captured image is stable, it is determined that the deviation correction of the PTZ camera is completed at the current preset focal length.

6. The method according to claim 1, characterized in that After the PTZ camera has completed deviation correction at each of the preset focal lengths in the first focal length list, the method further includes: Using the PTZ camera after the correction is completed to collect images of the target scene to obtain images to be analyzed; Performing feature point matching on the image to be analyzed and the reference image corresponding to the preset focal length to obtain a second feature point pair; Using a thin plate spline interpolation algorithm, based on the second feature point pair, a coordinate mapping relationship between the image to be analyzed and the reference image is calculated; Obtaining an offset of the image to be analyzed according to the coordinate mapping relationship and the second feature point pair; The image to be analyzed is deflected based on the offset of the image to be analyzed to obtain a target image.

7. A deviation correction device for a pan / tilt camera, characterized in that: include: A data acquisition module, used to acquire a first preset position of a pan / tilt camera, a first focal length list corresponding to the first preset position, and shooting parameters and a reference image corresponding to each of a plurality of preset focal lengths in the first focal length list; A first image acquisition module is used to select the preset focal length to be processed in the first focal length list in ascending order when the pan-tilt camera is set at the first preset position, and use the pan-tilt camera to capture an image of the target scene based on the preset focal length and the corresponding shooting parameters to obtain an image to be detected corresponding to the preset focal length; A correction module, used for comparing the image to be detected with a reference image corresponding to the preset focal length, obtaining an offset angle of the pan-tilt camera at the preset focal length, and correcting the pan-tilt camera based on the offset angle; Return to the execution module, used to return to the execution step: select the preset focal length currently to be processed in the first focal length list in order from small to large, use the pan-tilt camera to capture images of the target scene based on the preset focal length and the corresponding shooting parameters, and obtain the image to be detected corresponding to the preset focal length, until the pan-tilt camera completes the deviation correction at each of the preset focal lengths in the first focal length list.

8. The device according to claim 7, characterized in that The correction module comprises: a homography matrix obtaining submodule, configured to perform feature point matching between the image to be detected and the reference image corresponding to the preset focal length to obtain a first feature point pair, and obtain a homography matrix between the image to be detected and the reference image based on the first feature point pair; An offset calculation submodule, used to calculate the offset of the pan-tilt camera in the translation dimension and the offset of the pitch dimension at the preset focal length according to the homography matrix; An offset angle calculation submodule, used to calculate the offset angle of the pan-tilt camera in the translation dimension according to the offset in the translation dimension, and to calculate the offset angle of the pan-tilt camera in the pitch dimension according to the offset in the pitch dimension; The correction submodule is used to correct the translation dimension of the pan-tilt camera based on the offset angle of the translation dimension, and to correct the pitch dimension of the pan-tilt camera based on the offset angle of the pitch dimension.

9. The device according to claim 8, characterized in that The offset angle calculation submodule is specifically used for: Calculating the offset angle of each pixel in the image to be detected in the translation dimension according to the offset in the translation dimension; Based on the offset angle of each pixel in the translation dimension, obtaining the offset angle of the pan / tilt camera in the translation dimension; Calculating the offset angle of each pixel in the image to be detected in the pitch dimension according to the offset in the pitch dimension; Based on the offset angle of each pixel in the pitch dimension, the offset angle of the pan / tilt camera in the pitch dimension is obtained.

10. The device according to claim 7, characterized in that The device also includes: A shooting parameter stability detection module, used to call the shooting parameters corresponding to the preset focal length of the pan-tilt camera, and detect whether the captured picture of the pan-tilt camera under the shooting parameters is stable; The correction completion confirmation module is used to determine whether the correction of the pan-tilt camera is completed at the current preset focal length when the collected image is stable.

11. The device according to claim 8, characterized in that The device also includes: A second image acquisition module is used to acquire an image of the target scene at the preset focal length using the PTZ camera after the correction is completed to obtain an image to be analyzed; A feature point matching module, used for matching the feature points of the image to be analyzed with the reference image corresponding to the preset focal length to obtain a second feature point pair; An image analysis module, configured to calculate a coordinate mapping relationship between the image to be analyzed and the reference image based on the second feature point pair by using a thin plate spline interpolation algorithm; An offset determination module, used for obtaining the offset of the image to be analyzed according to the coordinate mapping relationship and the second feature point pair; The image correction module is used to correct the image to be analyzed based on the offset of the image to be analyzed to obtain a target image.

12. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-6 when executing a program stored in a memory.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images