Ball Machine Calibration Method and Electronic Device

Abstract

The present invention relates to the technical field of image processing, and particularly to a calibration method for a dome camera and an electronic device. The method includes obtaining a first picture and a second picture collected by a target dome camera, and first virtual camera parameters corresponding to the first picture; performing pixel point matching on the first picture and the second picture to determine a first pixel point set and a second pixel point set; obtaining the actual position coordinates of each pixel point in the first pixel point set, and making one-to-one correspondence between the actual position coordinates and the second pixel coordinates of each pixel point in the second pixel point set; and determining second virtual camera parameters corresponding to the second picture based on the first virtual camera parameters and the one-to-one correspondence result. Since the actual position coordinates will not change, making one-to-one correspondence between the second pixel coordinates and the actual position coordinates and obtaining the second virtual camera parameters based on the first virtual camera parameters improves the accuracy of dome camera calibration.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a calibration method for a PTZ camera and an electronic device. Background Art

[0002] A fixed gun camera is installed at a certain height, such as 30 meters, and the installation angle is fixed. Therefore, the picture captured by the gun camera is a picture with a fixed angle. Compared with the situation where the captured picture of the gun camera is fixed at a certain angle and the calibration is performed for the single-angle picture, a PTZ camera / PTZ can rotate. During the rotation of the PTZ camera, due to reasons such as installation and machine wear, the angle during rotation deviates greatly from the actual rotation angle and cannot be measured. Therefore, this deviation will make it difficult to accurately calibrate the picture after rotating a preset angle from this angle after the PTZ camera calibrates the picture at a certain angle. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a calibration method for a PTZ camera and an electronic device to solve the problem of low calibration accuracy of the PTZ camera.

[0004] According to a first aspect, an embodiment of the present invention provides a calibration method for a PTZ camera, including:

[0005] Obtaining a first picture and a second picture captured by a target PTZ camera, and first virtual camera parameters corresponding to the first picture, where there is the same target in the first picture and the second picture;

[0006] Performing pixel point matching on the first picture and the second picture to determine a first pixel point set and a second pixel point set;

[0007] Obtaining the actual position coordinates of each pixel point in the first pixel point set, and corresponding the actual position coordinates to the second pixel coordinates of each pixel point in the second pixel point set one by one;

[0008] Based on the first virtual camera parameters and the one-to-one corresponding result, adjusting the first virtual camera parameters to determine second virtual camera parameters corresponding to the second picture.

[0009] In the method for calibrating a PTZ camera provided by an embodiment of the present invention, since there are the same targets in the first picture and the second picture, first, pixel point matching is performed on the two pictures to determine the actual position coordinates of the same targets. Since the actual position coordinates objectively exist and do not change, the second pixel coordinates in the second picture are put into one-to-one correspondence with the actual position coordinates. By using the one-to-one correspondence results and the first virtual camera parameters, the second virtual camera parameters corresponding to the second picture are determined, that is, the virtual camera parameters corresponding to each picture collected by the target PTZ camera are sequentially determined to represent the pose of the target PTZ camera in each picture. The determination of this pose is sequentially determined in combination with the actual scene, improving the accuracy of PTZ camera calibration.

[0010] Combined with the first aspect, in the first implementation manner of the first aspect, determining the second virtual camera parameters corresponding to the second picture based on the first virtual camera parameters and the one-to-one correspondence results includes:

[0011] Determine the search range of the second virtual camera parameters based on the first virtual camera parameters;

[0012] Obtain the current second virtual camera parameters within the search range, and obtain the virtual pixel coordinates corresponding to each of the actual position coordinates under the current second virtual camera parameters;

[0013] Calculate the differences between each of the virtual pixel coordinates and the corresponding second pixel coordinates according to the one-to-one correspondence results between the actual position coordinates and the second pixel coordinates;

[0014] When each of the differences meets the preset conditions, determine the current second virtual camera parameters as the second virtual camera parameters corresponding to the second picture.

[0015] In the method for calibrating a PTZ camera provided by an embodiment of the present invention, the projection of the actual position coordinates under the virtual camera is determined by using the projection transformation matrix corresponding to the virtual camera to obtain the virtual pixel coordinates, and then the second virtual camera parameters are determined by using the differences between the virtual pixel coordinates and the second pixel coordinates. That is, the second virtual camera parameters of the second picture are calibrated based on the first virtual camera parameters of the first picture, improving the accuracy of PTZ camera calibration.

[0016] Combined with the first implementation manner of the first aspect, in the second implementation manner of the first aspect, determining the search range of the second virtual camera parameters based on the first virtual camera parameters includes:

[0017] Determine the starting search range of the second virtual camera parameters based on the first virtual camera parameters;

[0018] Obtain the current search step size;

[0019] Determine the search range based on the current search step size and the search start range.

[0020] The ball machine calibration method provided by the embodiments of the present invention limits the search range of the second virtual camera parameters through the first virtual camera parameters and the search step size, avoiding the influence of large-scale search on the calculation amount and improving the search efficiency.

[0021] Combined with the second implementation manner of the first aspect, in the third implementation manner of the first aspect, when each of the differences satisfies the preset conditions, determining the current second virtual camera parameter as the second virtual camera parameter corresponding to the second picture includes:

[0022] Obtain multiple groups of virtual camera parameters from the search range according to the current search step size;

[0023] Respectively use the multiple groups of virtual camera parameters as the current second virtual camera parameters, and obtain the corresponding differences under each group of virtual camera parameters;

[0024] Compare the differences corresponding to each group of virtual camera parameters to determine the current best virtual camera parameter from the multiple groups of virtual camera parameters, and use the current best virtual camera parameter as the second virtual camera parameter corresponding to the second picture.

[0025] The ball machine calibration method provided by the embodiments of the present invention determines multiple groups of virtual camera parameters within the search range, then determines a group of current best virtual camera parameters from the multiple groups of virtual camera parameters using the differences, and then determines the second virtual camera parameter based on the current best virtual camera parameter, gradually adjusting the virtual camera parameter to improve the accuracy of the finally determined second virtual camera parameter.

[0026] Combined with the third implementation manner of the first aspect, in the fourth implementation manner of the first aspect, using the current best virtual camera parameter as the second virtual camera parameter corresponding to the second picture includes:

[0027] Judge whether the current search step size reaches the preset search accuracy;

[0028] If not, update the search start range of the second virtual camera parameter based on the current best virtual camera parameter;

[0029] Reduce the current search step size to obtain the updated search step size;

[0030] Determine the updated search range based on the updated search start range and the updated search step size;

[0031] Obtain multiple sets of virtual camera parameters from the updated search range according to the updated search step length to re-determine the second virtual camera parameters of the second picture.

[0032] The pan-tilt calibration method provided by the embodiments of the present invention limits the search step length through the search accuracy, that is, gradually reduces the adjustment range of the virtual camera parameters, and then achieves the corresponding accuracy, which can improve the search efficiency.

[0033] Combined with the first aspect, in the fifth implementation manner of the first aspect, the matching of pixel points for the first picture and the second picture to determine the first pixel point set and the second pixel point set includes:

[0034] Perform feature extraction and feature detection on the first picture and the second picture respectively to obtain a first feature point set and a second feature point set;

[0035] Filter the first feature point set and the second feature point set to determine the first pixel point set and the second pixel point set.

[0036] The pan-tilt calibration method provided by the embodiments of the present invention can remove some abnormal pixel points by filtering the feature point set, reduce the data processing amount, and improve the calibration efficiency.

[0037] Combined with the fifth implementation manner of the first aspect, in the sixth implementation manner of the first aspect, the filtering of the first feature point set and the second feature point set to determine the first pixel point set and the second pixel point set includes:

[0038] Calculate the pixel offset of the matching pixel points in the first feature point set and the second feature point set;

[0039] Filter the first feature point set and the second feature point set based on the pixel offset to obtain a first point set and a second point set;

[0040] Calculate the pixel offset statistical value of the matching pixel points in the first point set and the second point set;

[0041] Filter the first point set and the second point set based on the pixel offset statistical value to obtain a new first point set and a new second point set;

[0042] Based on the first picture and the second picture, perform uniform sampling filtering on the new first point set and the new second point set to determine the first pixel point set and the second pixel point set.

[0043] The pan-tilt calibration method provided by the embodiments of the present invention screens the feature point set from the perspectives of pixel offset and pixel offset statistical value respectively, further ensuring the accuracy of the filtered pixel point set.

[0044] Combined with the sixth embodiment of the first aspect, in the seventh embodiment of the first aspect, based on the first picture and the second picture, uniformly sampling and filtering the new first point set and the new second point set to determine the first pixel point set and the second pixel point set, including:

[0045] Divide the first picture and the second picture into equal parts respectively;

[0046] For each equal - divided area in the first picture and the second picture, determine the central coordinate of the equal - divided area, and screen the matching pixel points closest to the central coordinate from the corresponding point sets to determine the first pixel point set and the second pixel point set.

[0047] The ball machine calibration method provided by the embodiments of the present invention performs uniform sampling and filtering in the new first point set and the new second point set, which can avoid the influence of a small area in the figure on the calculation accuracy in the new point set, and improve the accuracy of ball machine calibration.

[0048] According to the second aspect, the embodiments of the present invention further provide a ball machine calibration device, including:

[0049] An acquisition module, configured to acquire a first picture and a second picture collected by a target ball machine, and first virtual camera parameters corresponding to the first picture, where there are the same targets in the first picture and the second picture;

[0050] A matching module, configured to perform pixel - point matching on the first picture and the second picture to determine a first pixel point set and a second pixel point set;

[0051] A corresponding module, configured to acquire the actual position coordinates of each pixel point in the first pixel point set, and correspond the actual position coordinates to the second pixel coordinates of each pixel point in the second pixel point set one by one;

[0052] An adjustment module, configured to determine second virtual camera parameters corresponding to the second picture based on the first virtual camera parameters and the one - to - one corresponding results.

[0053] In the ball machine calibration device provided by the embodiment of the present invention, since the same target exists in the first picture and the second picture, first, pixel point matching is performed on the two pictures to determine the actual position coordinates of the same target. Since the actual position coordinates objectively exist and do not change, the second pixel coordinates in the second picture are put into one-to-one correspondence with the actual position coordinates. Using the one-to-one correspondence result and the first virtual camera parameters, the second virtual camera parameters corresponding to the second picture are determined, that is, the virtual camera parameters corresponding to each picture collected by the target ball machine are determined in sequence to represent the attitude of the target ball machine in each picture. The determination of this attitude is determined in sequence in combination with the actual scene, which improves the accuracy of ball machine calibration.

[0054] According to a third aspect, an embodiment of the present invention provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. Computer instructions are stored in the memory, and the processor executes the computer instructions to execute the ball machine calibration method described in the first aspect or any one of the embodiments of the first aspect.

[0055] According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions for causing a computer to execute the ball machine calibration method described in the first aspect or any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0057] Figure 1 is a flowchart of the ball machine calibration method according to an embodiment of the present invention;

[0058] Figure 2 is a flowchart of the ball machine calibration method according to an embodiment of the present invention;

[0059] Figure 3 is a flowchart of the ball machine calibration method according to an embodiment of the present invention;

[0060] Figure 4 is a structural block diagram of the ball machine calibration device according to an embodiment of the present invention;

[0061] Figure 5 is a schematic hardware structure diagram of the electronic device provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0063] It should be noted that the ball machine calibration method provided by the embodiments of the present invention can be used to calibrate the target ball machine every time it rotates through a preset angle, so as to achieve a 360° rotation angle calibration. Each angle corresponds to a set of camera parameters, and this set of camera parameters is obtained by virtual camera calibration hereinafter. Therefore, it is also referred to as virtual camera parameters hereinafter.

[0064] For example, set the position point when the target ball machine rotates 0°, and collect an image at this point to obtain a picture at the initial position. Subsequently, collect one picture every 2° of rotation, so that 180 pictures can be obtained, and each picture corresponds to a set of virtual camera parameters. Of course, the above rotation angle can be clockwise rotation or counterclockwise rotation along the horizontal direction, or up and down rotation at the same horizontal position point. Multiple pictures can be collected at the same position point, and each picture is calibrated to obtain the corresponding virtual camera parameters.

[0065] Taking the collection of 180 pictures as an example, after determining the camera parameters corresponding to the starting position, starting from the starting position, collect one picture every 2° in the clockwise direction, and determine the virtual camera parameters corresponding to each picture, traversing 180°; then collect one picture every 2° in the counterclockwise direction, and calibrate the virtual camera parameters corresponding to each picture, traversing 180°, so as to obtain the virtual camera parameters of the ball machine every 2°. In other embodiments, one picture can also be collected every 2° in the clockwise or counterclockwise direction, and the virtual camera parameters corresponding to each picture are determined, traversing 360°. Since the calibration accuracy of the initial position is the highest, collecting one picture every 2° in the clockwise direction and the counterclockwise direction respectively and traversing 180° and determining the virtual camera parameters corresponding to each picture has a higher calibration accuracy than only traversing 360° in the clockwise or counterclockwise direction.

[0066] The specific calibration method will be described in detail hereinafter.

[0067] According to an embodiment of the present invention, an embodiment of a PTZ camera calibration method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0068] In this embodiment, a PTZ camera calibration method is provided, which can be used in electronic devices such as computers, tablets, etc. Figure 1 It is a flowchart of the PTZ camera calibration method according to an embodiment of the present invention, as Figure 1 shown, this process includes the following steps:

[0069] S11, obtain a first picture and a second picture collected by a target PTZ camera, and first virtual camera parameters corresponding to the first picture.

[0070] Wherein, there are the same targets in the first picture and the second picture.

[0071] The first picture and the second picture can be pictures collected by the target PTZ camera continuously twice in the same direction. For example, if the currently collected picture is the first picture, then continue to rotate a preset rotation angle in the same direction, and the picture collected by the target PTZ camera is called the second picture. Among them, the preset rotation angle of the target PTZ camera between collecting the first picture and the second picture can be set according to the actual situation, and no limitation is made on the rotation angle of the PTZ camera here. Or, the first picture and the second picture can also be pictures collected by the target PTZ camera in the same direction, and it is not limited that they are collected continuously, as long as there are the same targets in the first picture and the second picture.

[0072] Among them, the first virtual camera parameters corresponding to the first picture can be obtained by calibration or by setting. Taking calibration as an example, if the first picture is an initial picture collected at the initial position, the calibration method of the first virtual camera parameters is to obtain a preset number of calibration points in the first picture, obtain the actual position coordinates (such as GPS coordinates) and pixel coordinates of the calibration points, establish a virtual camera parameter search space corresponding to the first picture, which is composed of multiple groups of virtual camera parameters, use existing open-source tools to obtain the corresponding relationship between the actual position coordinates and pixel coordinates under each group of virtual camera parameters, obtain the mapped pixel coordinates corresponding to the actual position coordinates under each group of virtual camera parameters, compare each group of mapped pixel coordinates with the pixel coordinates respectively, and use the virtual camera parameters corresponding to the mapped pixel coordinates with the smallest difference as the first virtual camera parameters corresponding to the first picture.

[0073] If the first picture is not the initial picture collected at the initial position, the virtual camera parameters of the first picture are calibrated using the virtual camera parameters of the pictures collected before the first picture. For the specific calibration process, please refer to the following description.

[0074] In other embodiments, the first virtual camera parameters corresponding to the first image can also be obtained by other methods, such as coordinate transformation, etc.

[0075] S12. Perform pixel point matching on the first picture and the second picture to determine the first pixel point set and the second pixel point set.

[0076] Since there are the same targets in the first picture and the second picture, based on these same targets, pixel point matching can be performed on the first picture and the second picture to determine the pixel points corresponding to the same targets. Among them, the pixel point matching method can be determined by feature extraction, or can be determined by target recognition, etc., and no limitation is imposed on it here.

[0077] After the electronic device performs pixel point matching on the first picture and the second picture, a first pixel point set corresponding to the first picture and a second pixel point set corresponding to the second picture are obtained respectively. Among them, the pixel points in the first pixel point set and the pixel points in the second pixel point set are in one-to-one correspondence.

[0078] Optionally, after pixel point matching, the matching pixel points are filtered to remove the abnormal points in the matching pixel points, and the first pixel point set and the second pixel point set are obtained.

[0079] S13. Obtain the actual position coordinates of each pixel point in the first pixel point set, and correspond the actual position coordinates to the second pixel coordinates of each pixel point in the second pixel point set one by one.

[0080] The actual position coordinates of each pixel point in the first pixel point set can be calculated by the pixel coordinates of each pixel point and the first virtual camera parameters. For example, after the open-source software obtains the virtual camera parameters, the corresponding relationship between the actual position coordinates and the pixel coordinates can be obtained; or the actual position coordinates can also be obtained by manual calibration. No limitation is imposed on the method for obtaining the actual position coordinates of each pixel point in the first pixel point set here. As described above, the pixel points in the first pixel point set and the second pixel point set are in one-to-one correspondence. Therefore, the actual position coordinates of each pixel point in the first pixel point set are also the actual position coordinates of the corresponding pixel points in the second pixel point set.

[0081] The electronic device corresponds the actual position coordinates to the second pixel coordinates of the corresponding pixel points in the second pixel point set one by one. Among them, the second pixel coordinates are the pixel coordinates of each pixel point in the second pixel point set.

[0082] S14. Determine the second virtual camera parameters corresponding to the second picture based on the first virtual camera parameters and the corresponding results one by one.

[0083] Since the first picture and the second picture have the same target, after pixel point matching, the same target in the first picture and the second picture can be determined. Since the same target has the same actual position coordinates, due to the different postures of the target ball machine in different pictures, the pixel coordinates of the same target in different pictures are different. Based on the first virtual camera parameters corresponding to the first picture, combined with the one-to-one correspondence between the actual position coordinates and the pixel coordinates in the second picture, the second virtual camera parameters corresponding to the second picture are determined. Specifically, the method of using a virtual camera is used to determine the posture of the target ball machine, that is, the camera parameters of the target ball machine in different postures. Since the association relationship between the first picture and the second picture is established through pixel point matching, the second virtual camera parameters can be determined using the first virtual camera parameters.

[0084] For example, based on the first virtual camera parameters, the initial value of the second virtual camera parameters can be determined. Using this initial value, the virtual pixel coordinates of the actual position coordinates are determined, and the virtual pixel coordinates are compared with the pixel coordinates to determine the second virtual camera parameters. If the difference meets the conditions, the second virtual camera parameters can be determined; if the difference does not meet the conditions, the search for the second virtual camera parameters needs to be carried out again.

[0085] Alternatively, based on the first virtual camera parameters, the search range can be determined, and multiple groups of virtual camera parameters can be determined within this search range. Using the same principle as above, the second virtual camera parameters are determined based on multiple groups of virtual camera parameters.

[0086] The specific details of this step will be described in detail below.

[0087] For the ball machine calibration method provided in this embodiment, since the first picture and the second picture have the same target, first, pixel point matching is performed on the two pictures to determine the actual position coordinates of the same target. Since the actual position coordinates objectively exist and do not change, the second pixel coordinates in the second picture are put into one-to-one correspondence with the actual position coordinates. Using the one-to-one correspondence results and the first virtual camera parameters, the second virtual camera parameters corresponding to the second picture are determined, that is, by sequentially determining the virtual camera parameters corresponding to each picture collected by the target ball machine to represent the posture of the target ball machine in each picture. The determination of this posture is sequentially determined in combination with the actual scene, which improves the accuracy of ball machine calibration.

[0088] In this embodiment, a ball machine calibration method is provided, which can be used in electronic devices such as computers and tablet computers.Figure 2 is a flowchart of a calibration method for a PTZ camera according to an embodiment of the present invention. As Figure 2 shown, the process includes the following steps:

[0089] S21, obtaining a first picture and a second picture collected by a target PTZ camera, and first virtual camera parameters corresponding to the first picture.

[0090] Wherein, there are the same targets in the first picture and the second picture.

[0091] For details, please refer to Figure 1 S11 in the embodiment shown, which will not be elaborated here.

[0092] S22, performing pixel point matching on the first picture and the second picture to determine a first pixel point set and a second pixel point set.

[0093] For details, please refer to Figure 1 S12 in the embodiment shown, which will not be elaborated here.

[0094] S23, obtaining the actual position coordinates of each pixel point in the first pixel point set, and corresponding the actual position coordinates to the second pixel coordinates of each pixel point in the second pixel point set one by one.

[0095] For details, please refer to Figure 1 S13 in the embodiment shown, which will not be elaborated here.

[0096] S24, determining second virtual camera parameters corresponding to the second picture based on the first virtual camera parameters and the one-to-one corresponding results.

[0097] Specifically, the above S24 includes:

[0098] S241, determining a search range of the second virtual camera parameters based on the first virtual camera parameters.

[0099] Since the second picture is collected by the target PTZ camera after deflecting a preset angle based on the first picture, therefore, the electronic device randomly determines the search range of the second virtual camera parameters based on the first virtual camera parameters; or, slightly adjusts the first virtual camera parameters to determine the search starting value of the second virtual camera parameters, and then determines the search range based on the search starting value.

[0100] In some alternative embodiments of the present embodiment, the above S241 includes:

[0101] (1) Determining a search starting range of the second virtual camera parameters based on the first virtual camera parameters.

[0102] Among the parameters of the PTZ camera, there are fixed parameters, and the parameters that need to be calibrated are uncertain parameters. For example, for four parameters, assuming the parameters of the first virtual camera are (h, x, y, z), the search start range of the parameters of the second virtual camera is params = (h ± 0.1, x + 2 ± 0.1, y ± 0.1, z ± 0.1). The reason for x + 2 is that it is assumed that the second picture is horizontally rotated by 2 degrees based on the first picture; if it is rotated vertically or in other directions, then y or z is added by 2; if it is rotated by other angles, the value is the corresponding value of other angles.

[0103] (2) Obtain the current search step size.

[0104] The current search step size represents the change value for each adjustment of each parameter in the initial camera parameters, and its specific value can be set according to actual needs, and no limitation is made on it here.

[0105] (3) Based on the current search step size and the search start range, determine the search range.

[0106] Continuing with the above example, if the camera parameters to be adjusted include 4 parameters, the search start range of the parameters of the second virtual camera is params = (h ± 0.1, x + 2 ± 0.1, y ± 0.1, z ± 0.1), and the current search step size is step, then the corresponding search range is params ± c1 * step, where c1 is a constant. For example, the initial value of h ± 0.1 for the camera height is h0, then for the camera height h, its search range is: h ± c1 * step.

[0107] For example, assuming the step size is step and the constant is 10, then the search range is params ± 10 * step. Among them, the constant 10 can be set according to actual needs and is not limited to this.

[0108] S242. Obtain the current parameters of the second virtual camera within the search range, and obtain the virtual pixel coordinates corresponding to each actual position coordinate under the current parameters of the second virtual camera.

[0109] After the electronic device determines the search range, it can randomly determine multiple virtual camera parameters within the preset search range of each camera parameter, so as to determine multiple groups of current parameters of the second virtual camera. The electronic device then calculates the virtual pixel coordinates corresponding to the actual position coordinates for each group of current parameters of the second virtual camera in turn. The calculation of the virtual pixel coordinates can be implemented using open-source software, and no limitation is made on it here.

[0110] S243. According to the one-to-one correspondence result between the actual position coordinates and the second pixel coordinates, calculate the difference between each virtual pixel coordinate and the corresponding second pixel coordinate.

[0111] The virtual pixel coordinates are calculated based on the actual position coordinates. Since both the virtual pixel coordinates and the second pixel coordinates represent the coordinates of the actual position coordinates under different camera parameters, the electronic device can determine the difference between the current second virtual camera parameters and the actual camera parameters by using the difference between the virtual pixel coordinates and the second pixel coordinates.

[0112] S244, determine whether each difference satisfies a preset condition.

[0113] All the matching pixel points are used to constrain the camera parameters, that is, the determined camera parameters should ensure that the differences corresponding to all the matching pixel points satisfy the preset conditions. For example, for 10 sets of virtual camera parameters and 10 matching pixel points, the electronic device needs to calculate whether the difference between the virtual pixel coordinates and the pixel coordinates of each matching pixel point satisfies the preset condition under each set of virtual camera parameters; if the differences between the virtual pixel coordinates and the pixel coordinates of all the matching pixel points satisfy the preset condition under the current set of virtual camera parameters, it means that the current set of second virtual camera parameters can be used as the second virtual camera parameters. If there is a difference between the virtual pixel coordinates and the pixel coordinates of a matching pixel point that does not satisfy the preset condition under the current set of second virtual camera parameters, the current set of second virtual camera parameters is discarded.

[0114] When each of the differences satisfies the preset condition, execute S245; otherwise, perform other operations. Among them, the other operations can be to re-determine the search range, or to re-determine multiple sets of second virtual camera parameters based on the search range, and so on.

[0115] S245, determine the current second virtual camera parameters as the second virtual camera parameters corresponding to the second picture.

[0116] The electronic device can directly determine the current second virtual camera parameters as the second virtual camera parameters corresponding to the second picture, or use the current second virtual camera parameters as the starting camera parameters for searching for the second virtual camera parameters, and on this basis, determine the second virtual camera parameters again, or select the best virtual camera parameters from multiple sets of second virtual camera parameters that meet the conditions as the second virtual camera parameters.

[0117] In some optional implementation manners of this embodiment, the above S245 may include:

[0118] (1) Obtain multiple sets of virtual camera parameters from the search range according to the current search step.

[0119] (2) Respectively use the multiple sets of virtual camera parameters as the current second virtual camera parameters, and obtain the corresponding differences under each set of virtual camera parameters.

[0120] (3) Compare the differences corresponding to each set of virtual camera parameters to determine the current best virtual camera parameter from multiple sets of virtual camera parameters, and use the current best virtual camera parameter as the second virtual camera parameter corresponding to the second picture.

[0121] When the electronic device determines that there are multiple sets of current second virtual camera parameters for which the differences of all matching pixel points satisfy the preset conditions, it can determine a set of current second virtual camera parameters with the smallest difference from the multiple sets of current second virtual camera parameters and use it as the current best virtual camera parameter. If there is only one set of current second virtual camera parameters, then this set of current second virtual camera parameters is determined as the current best virtual camera parameter.

[0122] After determining the current best virtual camera parameter, the current best virtual camera parameter can be used as the second virtual camera parameter; or, in order to further improve the accuracy of the camera parameter, the current search step size can be reduced based on the current search step size, and the search range can be determined again.

[0123] By determining multiple sets of current second virtual camera parameters within the search range, then using the differences to determine a set of current best virtual camera parameters from the multiple sets of current second virtual camera parameters, and then determining the second virtual camera parameter based on the current best virtual camera parameter, gradually adjust the current best virtual camera parameter to improve the accuracy of the finally determined second virtual camera parameter.

[0124] As an alternative implementation manner of this embodiment, in step (3) of the above S244, the using the current best virtual camera parameter as the second virtual camera parameter corresponding to the second picture may include:

[0125] 3.1) Determine whether the current search step size reaches the preset search accuracy.

[0126] 3.2) If not, update the search start range of the second virtual camera parameter based on the current best virtual camera parameter.

[0127] 3.3) Reduce the current search step size to obtain the updated search step size.

[0128] 3.4) Determine the updated search range based on the updated search start range and the updated search step size.

[0129] 3.5) Obtain multiple sets of virtual camera parameters from the updated search range according to the updated search step size to re-determine the second virtual camera parameter of the second picture.

[0130] The electronic device can set a preset search accuracy for the search step length. Before adjusting the current search step length, it first determines whether it reaches the preset search accuracy. When the preset search accuracy is not reached, it means that the current search step length can be further reduced. Based on the current optimal camera parameters, the search range is adjusted using the search step length until the search step length reaches the search accuracy.

[0131] Specifically, the electronic device uses the optimal virtual camera parameters as the search start range for the updated second virtual camera parameters, reduces the current search step length to obtain the updated search step length, and then combines the updated search step length based on the updated second virtual camera parameters to re-determine the updated search range. After the search range is determined, the electronic device obtains multiple sets of virtual camera parameters from this search range based on the updated search step length, and then uses the above S242 - S245 to re-determine the second virtual camera parameters. Further, after re-determining the second virtual camera parameters using the updated search step length, step 3.1) is executed again to determine whether the updated search step length reaches the preset search accuracy. If not, steps 3.2) - 3.5) are executed until the updated search step length reaches the preset search accuracy; among them, the finally determined second virtual camera parameters are used as the pose of the target pan-tilt camera to collect the second picture, that is, the corresponding camera parameters.

[0132] By limiting the search step length through the search accuracy, that is, gradually reducing the adjustment range of the second virtual camera parameters, and then achieving the corresponding accuracy, the search efficiency can be improved.

[0133] The pan-tilt camera calibration method provided in this embodiment uses the projection transformation matrices corresponding to different second virtual camera parameters within the search range to determine the virtual pixel coordinates of the actual position coordinates under different second virtual camera parameters, and then determines the second virtual camera parameters using the difference between the virtual pixel coordinates and the second pixel coordinates, that is, calibrates the second virtual camera parameters of the second picture using the first virtual camera parameters of the first picture, improving the calibration accuracy of the pan-tilt camera.

[0134] In this embodiment, a pan-tilt camera calibration method is provided, which can be used in electronic devices such as computers and tablet computers. Figure 3 is a flowchart of the pan-tilt camera calibration method according to an embodiment of the present invention, as Figure 3 shown, and this process includes the following steps:

[0135] S31, obtain the first picture and the second picture collected by the target pan-tilt camera, and the first virtual camera parameters corresponding to the first picture.

[0136] For details, please refer to Figure 1 S11 in the shown embodiment, and details will not be repeated here.

[0137] S32. Perform pixel point matching on the first picture and the second picture to determine the first pixel point set and the second pixel point set.

[0138] Specifically, the above S32 includes:

[0139] S321. Respectively perform feature extraction and feature detection on the first picture and the second picture to obtain the first feature point set and the second feature point set.

[0140] The electronic device can first extract the features of the first picture and the second picture respectively, and then calculate the similarity of the extracted features, that is, the above-mentioned feature detection, and use the similarity to determine the first feature point set and the second feature point set.

[0141] Among them, the feature extraction can be carried out in the way of using a feature extraction model, or through image processing, and no limitation is made here, and it can be specifically set according to actual needs.

[0142] S322. Filter the first feature point set and the second feature point set to determine the first pixel point set and the second pixel point set.

[0143] The electronic device filters the first feature point set and the second feature point set, which can be to calculate the pixel offset of the corresponding pixel points in the two feature point sets in turn, or other ways to screen out abnormal points, and finally determine the first pixel point set and the second pixel point set.

[0144] In some alternative embodiments of this embodiment, the above S322 may include:

[0145] (1) Calculate the pixel offset of the matching pixel points in the first feature point set and the second feature point set.

[0146] (2) Filter the first feature point set and the second feature point set based on the pixel offset to obtain the first point set and the second point set.

[0147] (3) Calculate the pixel offset statistical value of the matching pixel points in the first point set and the second point set.

[0148] (4) Filter the first point set and the second point set based on the pixel offset statistical value to obtain a new first point set and a new second point set.

[0149] (5) Based on the first picture and the second picture, perform uniform sampling filtering on the new first point set and the new second point set to determine the first pixel point set and the second pixel point set.

[0150] The electronic device performs pixel offset filtering, pixel offset statistical value filtering, and uniform sampling filtering on the first feature point set and the second feature point set respectively, and finally determines the first pixel point set and the second pixel point set. Taking the pixel offset statistical value as an example, the electronic device can calculate the mean and variance of the offsets of each pixel point in the x and y directions in the first point set and the second point set, and use the outlier detection formula: [mean - 3 * variance, mean + 3 * variance]. Only the pixel points with the offset amount falling within the interval will be retained, and the rest will be filtered. Screening the feature point set from the perspectives of pixel offset and pixel offset statistical value respectively further ensures the accuracy of the screened pixel point set.

[0151] Further optionally, step (5) of S322 may include:

[0152] 5.1) Divide the first picture and the second picture into equal parts respectively.

[0153] 5.2) For each equal division area in the first picture and the second picture, determine the central coordinates of the equal division area, and screen the matching pixel points closest to the central coordinates from the corresponding point sets to determine the first pixel point set and the second pixel point set.

[0154] For example, if the resolution of the first picture and the second picture is 1920 * 1080, divide their width and height evenly into 16 * 9 equal parts, take the central coordinates of each block as the anchor points, traverse all the points in the first point set and the second point set, retain the matching pixel points closest to each anchor point, and filter the rest to obtain the first pixel point set and the second pixel point set.

[0155] Performing uniform sampling filtering in the new first point set and the new second point set can avoid the influence of a small area in the figure on the calculation accuracy in the new point set, and improve the accuracy of the calibration of the PTZ camera.

[0156] S33, obtain the actual position coordinates of each pixel point in the first pixel point set, and correspond the actual position coordinates to the second pixel coordinates of each pixel point in the second pixel point set one by one.

[0157] For details, please refer to Figure 1 S13 of the illustrated embodiment, which will not be elaborated here.

[0158] S34, based on the first virtual camera parameters and the one-to-one corresponding results, determine the second virtual camera parameters corresponding to the second picture.

[0159] For details, please refer to Figure 2 S24 of the illustrated embodiment, which will not be elaborated here.

[0160] The ball machine calibration method provided in this embodiment can filter the feature point set, remove some abnormal pixel points, reduce the data processing volume, and improve the calibration efficiency.

[0161] In this embodiment, a ball machine calibration device is also provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0162] This embodiment provides a ball machine calibration device, as Figure 4 shown, including:

[0163] An acquisition module 41, configured to acquire a first picture and a second picture collected by a target ball machine, and first virtual camera parameters corresponding to the first picture, where there are the same targets in the first picture and the second picture;

[0164] A matching module 42, configured to perform pixel point matching on the first picture and the second picture to determine a first pixel point set and a second pixel point set;

[0165] A corresponding module 43, configured to acquire the actual position coordinates of each pixel point in the first pixel point set, and perform one-to-one correspondence between the actual position coordinates and the second pixel coordinates of each pixel point in the second pixel point set;

[0166] An adjustment module 44, configured to determine second virtual camera parameters corresponding to the second picture based on the first virtual camera parameters and the one-to-one correspondence result.

[0167] For the ball machine calibration device provided in this embodiment, since there are the same targets in the first picture and the second picture, first perform pixel point matching on the two pictures to determine the actual position coordinates of the same targets. Since the actual position coordinates objectively exist and do not change, perform one-to-one correspondence between the second pixel coordinates in the second picture and the actual position coordinates, and use the one-to-one correspondence result and the first virtual camera parameters to determine the second virtual camera parameters corresponding to the second picture, that is, determine the virtual camera parameters corresponding to each picture collected by the target ball machine to represent the attitude of the target ball machine in each picture. The determination of this attitude is sequentially determined in combination with the actual scene, improving the accuracy of ball machine calibration.

[0168] The ball machine calibration device in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0169] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0170] The embodiment of the present invention also provides an electronic device having the above Figure 4 The ball camera calibration device shown.

[0171] See also Figure 5 , Figure 5 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the electronic device may include: at least one processor 51, such as a CPU (Central Processing Unit), at least one communication interface 53, a memory 54, and at least one communication bus 52. The communication bus 52 is used to realize the connection and communication between these components. The communication interface 53 may include a display screen (Display) and a keyboard (Keyboard), and the optional communication interface 53 may also include a standard wired interface and a wireless interface. The memory 54 may be a high-speed RAM memory (Random Access Memory) or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 54 may optionally be at least one storage device located away from the aforementioned processor 51. The processor 51 may be combined with Figure 4 In the described device, the memory 54 stores an application program, and the processor 51 calls the program code stored in the memory 54 to execute any of the above method steps.

[0172] The communication bus 52 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 52 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0173] Among them, the memory 54 may include a volatile memory, such as a random-access memory (RAM); the memory may also include a non-volatile memory, such as a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 54 may further include a combination of the above types of memories.

[0174] Among them, the processor 51 may be a central processing unit (CPU), a network processor (NP) or a combination of a CPU and an NP.

[0175] Among them, the processor 51 may further include a hardware chip. The above hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0176] Optionally, the memory 54 is further configured to store program instructions. The processor 51 may call the program instructions to implement the ball machine calibration method as shown in any embodiment of the present application.

[0177] An embodiment of the present invention further provides a non-transitory computer storage medium, which stores computer-executable instructions that can execute the ball machine calibration method in any of the above method embodiments. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memories.

[0178] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A calibration method for a PTZ camera, characterized in that, it includes: Obtaining a first picture and a second picture collected by a target PTZ camera, and first virtual camera parameters corresponding to the first picture, where there are the same targets in the first picture and the second picture; the second picture is a picture collected by the target PTZ camera after continuing to rotate a preset rotation angle in the same direction after collecting the first picture; Performing pixel point matching on the first picture and the second picture to determine a first pixel point set and a second pixel point set; Obtaining the actual position coordinates of each pixel point in the first pixel point set, and corresponding the actual position coordinates to the second pixel coordinates of each pixel point in the second pixel point set one by one; Determining the search range of second virtual camera parameters based on the first virtual camera parameters; Obtaining current second virtual camera parameters within the search range, and obtaining virtual pixel coordinates corresponding to each of the actual position coordinates under the current second virtual camera parameters; Calculating the differences between each of the virtual pixel coordinates and the corresponding second pixel coordinates according to the one-to-one correspondence result of the actual position coordinates and the second pixel coordinates; When each of the differences meets a preset condition, determining the current second virtual camera parameters as the second virtual camera parameters corresponding to the second picture.

2. The method according to claim 1, characterized in that, the determining the search range of the second virtual camera parameters based on the first virtual camera parameters includes: Determining the starting search range of the second virtual camera parameters based on the first virtual camera parameters; Obtaining the current search step size; Determining the search range based on the current search step size and the starting search range.

3. The method according to claim 2, characterized in that, the when each of the differences meets a preset condition, determining the current second virtual camera parameters as the second virtual camera parameters corresponding to the second picture includes: Obtaining multiple groups of virtual camera parameters from the search range according to the current search step size; Respectively taking the multiple groups of virtual camera parameters as the current second virtual camera parameters, and obtaining the corresponding differences under each group of virtual camera parameters; Comparing the differences corresponding to each group of virtual camera parameters to determine the current best virtual camera parameters from the multiple groups of virtual camera parameters, and taking the current best virtual camera parameters as the second virtual camera parameters corresponding to the second picture.

4. The method according to claim 3, characterized in that, the taking the current best virtual camera parameters as the second virtual camera parameters corresponding to the second picture includes: Judging whether the current search step size reaches a preset search accuracy; If not, updating the starting search range of the second virtual camera parameters based on the current best virtual camera parameters; Reducing the current search step size to obtain an updated search step size; Determining an updated search range based on the updated starting search range and the updated search step size; Obtain multiple sets of virtual camera parameters from the updated search range according to the updated search step size to re-determine the second virtual camera parameters of the second picture.

5. The method according to claim 1, wherein, the performing pixel point matching on the first picture and the second picture to determine a first pixel point set and a second pixel point set includes: performing feature extraction and feature detection on the first picture and the second picture respectively to obtain a first feature point set and a second feature point set; filtering the first feature point set and the second feature point set to determine the first pixel point set and the second pixel point set.

6. The method according to claim 5, wherein, the filtering the first feature point set and the second feature point set to determine the first pixel point set and the second pixel point set includes: calculating the pixel offset of the matching pixel points in the first feature point set and the second feature point set; filtering the first feature point set and the second feature point set based on the pixel offset to obtain a first point set and a second point set; calculating the pixel offset statistical value of the matching pixel points in the first point set and the second point set; filtering the first point set and the second point set based on the pixel offset statistical value to obtain a new first point set and a new second point set; performing uniform sampling filtering on the new first point set and the new second point set based on the first picture and the second picture to determine the first pixel point set and the second pixel point set.

7. The method according to claim 6, wherein, the performing uniform sampling filtering on the new first point set and the new second point set based on the first picture and the second picture to determine the first pixel point set and the second pixel point set includes: equally dividing the first picture and the second picture respectively; determining the central coordinates of each equal division area in the first picture and the second picture, and screening the matching pixel points closest to the central coordinates from the corresponding point sets to determine the first pixel point set and the second pixel point set.

8. An electronic device, wherein, it includes: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the dome camera calibration method according to any one of claims 1-7.

9. A computer-readable storage medium, wherein, the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the dome camera calibration method according to any one of claims 1-7.

Images