A camera calibration and zoom tracking method with scene adaptation

Abstract

The application provides a camera calibration and zoom tracking method with scene self-adaption, first, according to the actual scene, selecting a suitable target as the calibration point of zoom tracking, obtaining the geographic position information of the calibration point based on 3D-GIS and saving it. Then, for all the calibration points, the camera automatically focuses at different focal lengths, fitting the zoom tracking curve, realizing zoom tracking. Finally, when the application scene changes, repeat the previous two steps, quickly obtain the zoom tracking curve of the current scene. The application uses 3D-GIS data to help the camera to calibrate, quickly obtain the zoom tracking curve under different scenes, improve the precision of zoom tracking, and can be applied to zoom tracking in various scenes.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of camera equipment and image processing, and particularly relates to a camera calibration and zoom tracking method with scene self-adaptation. BACKGROUND

[0002] A zoom (variable magnification) optical imaging system refers to an optical system in which, when the object distance changes along the optical axis, the focal length of the optical system can be continuously adjusted within a certain range in order to obtain an image of appropriate size. The zoom optical imaging system is widely used in photography, videography, security monitoring, visual tracking, precision measurement and other fields. In the field of civilian photography, zoom lenses have replaced fixed focus lenses and become an important part of cameras. In panoramic imaging, the zoom lens is rotated around the perspective projection center to capture multiple images, and image stitching technology is used to obtain a larger panoramic image. In the field of visual tracking and navigation, a camera or a camera with adjustable azimuth angle, elevation angle and magnification can maintain the size of the image obtained by tracking unchanged while maintaining high image resolution, and can also obtain the attitude and position information of the target. Therefore, the study of related technologies and scientific problems in the imaging process of the zoom optical system has important academic research and application value. With the widespread use of zoom lenses, zoom tracking technology has become an important function of digital cameras and is widely used in industrial detection, military security and leisure and other aspects. Zoom tracking refers to changing the focal length of the system by moving the zoom lens during zoom operation. Since the focal length of the system changes, the image will not be focused on the original focusing plane, so the position of the focusing lens needs to be adjusted while the zoom lens moves to focus the scene on the image sensor during zooming.

[0003] However, for zoom tracking, an accurate zoom tracking curve must be obtained. There are mainly two methods for obtaining the traditional zoom tracking curve: one is to use an auto-focusing algorithm to calculate the tracking curve at different object distances point by point, which requires a large amount of calculation and occupies a large amount of storage space of the camera; the other is to use geometric optics knowledge to calculate the zoom tracking curve at different object distances according to the optical position relationship of the lens group, but this method will reduce the quality of zoom tracking when applied to the camera system due to mechanical errors and other factors. The problem to be solved by the zoom tracking algorithm is how to accurately estimate the current object distance. However, the existing method needs to make additional calibration on the lens tracking curve provided by the lens manufacturer to determine the object distance, which is complex to operate, has high cost, and when the picture changes dramatically during zooming, the image detail information may be unreliable, leading to misjudgment of the object distance. Therefore, how to implement the zoom tracking algorithm while reducing the calibration cost and avoiding misjudgment caused by unreliable image detail information when the picture changes dramatically during zooming is a technical problem to be solved in the field. SUMMARY

[0004] The present application aims to solve the problem of poor zoom tracking performance and the problem that the tracking result is greatly affected by environmental changes, and proposes a camera calibration and zoom tracking method with scene self-adaptation.

[0005] In a first aspect, the present application provides a camera calibration and zoom tracking method with scene self-adaptation, comprising the following steps:

[0006] Step 100, according to the actual scene, selecting a suitable target as a calibration point for zoom tracking, obtaining the geographical position information of the calibration point based on 3D-GIS and saving it, and calculating the distance between the camera and the calibration point;

[0007] Step 200, for all the calibration points, the camera is adjusted to different focal lengths for automatic focusing, the focusing position corresponding to the focal length is obtained, the zoom tracking curve is fitted, and zoom tracking is realized;

[0008] Step 300, when the actual scene changes, repeating the steps 100 and 200 to quickly obtain the zoom tracking curve of the current scene, and improving the zoom tracking performance and imaging quality.

[0009] Further, according to the actual scene, selecting a suitable target as a calibration point for zoom tracking further comprises: manually selecting 10-20 targets from close to the camera to far away from the camera as calibration points, and recording the focal length and image distance of each calibration point.

[0010] Further, the distance from each calibration point to the camera is different.

[0011] Further, obtaining the geographical position information of the calibration point based on the 3D-GIS map and saving it further comprises:

[0012] Reading the data of the calibration points and the camera itself from the 3D-GIS map to obtain the latitude and longitude information of the camera and the calibration points, wherein the 3D-GIS map of the calibration points includes MIF and MID files, and the MIF and MID files include the spatial information of all spatial objects in the map and the attribute information of the road.

[0013] Further, calculating the distance between the camera and the calibration point further comprises: based on the obtained latitude and longitude information of the camera and the calibration point, calculating the distance from all calibration points to the camera by using the Euclidean distance calculation formula, and using the different object distances as the calibration.

[0014] Further, the step 200 comprises the following sub-steps:

[0015] Step 201, for each calibration point selected in step 100, the focal length of the camera lens is divided into equal parts, and automatic focusing is performed for each focal length to obtain the accurate focusing position;

[0016] Step 202, using a curve fitting method, the discrete focal length and focusing position point pairs are fitted into a continuous zoom tracking curve;

[0017] Step 203, after obtaining the zoom tracking curve for multiple object distances in step 202, for any object distance, a traditional zoom tracking method is used to generate a zoom tracking curve corresponding to the object distance, and the focal length and image distance are controlled to follow the curve to ensure that the image is always in focus.

[0018] Further, the method of automatic focusing of the camera at different focal lengths in step 201 adopts contrast-based automatic focusing and / or sharpness-based automatic focusing and / or neural network-based automatic focusing.

[0019] Further, the zoom tracking curve fitting method in step 202 adopts a polynomial fitting method and / or a spline interpolation method.

[0020] Further, the traditional zoom tracking method in step 203 adopts a geometric zoom tracking method and / or an adaptive zoom tracking method and / or a feedback zoom tracking method.

[0021] In a second aspect, the present application also provides a camera calibration and zoom tracking device with scene adaptation, comprising:

[0022] An information acquisition module is configured to select a suitable target as a calibration point for zoom tracking according to the actual scene, acquire the geographic location information of the calibration point based on 3D-GIS and save it, and calculate the distance between the camera and the calibration point.

[0023] A zoom tracking module is configured to adjust the camera to different focal lengths for automatic focusing for all the calibration points, acquire the focusing position corresponding to the focal length, fit the zoom tracking curve, and realize zoom tracking.

[0024] A cycle module is configured to repeat the steps 100 and 200 when the application scene changes, quickly obtain the zoom tracking curve of the current scene, and improve the zoom tracking performance and imaging quality.

[0025] Compared with the prior art, the present application has the beneficial effects as follows: the camera calibration and zoom tracking method with scene self-adaptation provided by the present application is different from the existing zoom tracking method, which uses 3D-GIS map information to help the camera obtain a zoom tracking curve, and can be quickly applied to different actual application scenes. First, according to the actual scene, a suitable target is selected as a calibration point for zoom tracking, and the geographic position information of the calibration point is obtained based on 3D-GIS and saved. Then, for all the calibration points, the camera automatically focuses at different focal lengths to fit the zoom tracking curve and realize zoom tracking. Finally, when the application scene changes, the above two steps are repeated to quickly obtain the zoom tracking curve of the current scene. The zoom imaging system using the method has obvious improvement in adaptability to the environment. Moreover, the calibration method provided by the present application is efficient and effective, and can be applied to various different scenes. In summary, the method of the present application can effectively improve the zoom tracking performance and the clarity of the obtained image. BRIEF DESCRIPTION OF DRAWINGS

[0026] Other features, objects, and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments with reference to the attached drawings:

[0027] Figure 1 The figure is a flowchart of the scene self-adaptive camera calibration and zoom tracking method in an embodiment of the present application.

[0028] Figure 2 The figure is a schematic diagram of the relative position between the calibration target and the camera in an embodiment of the present application.

[0029] Figure 3 The figure is a schematic diagram of the zoom tracking curve fitting in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0031] In a first aspect, the present embodiment provides a camera calibration and zoom tracking method with scene self-adaptation, comprising the following steps:

[0032] Step 100: According to the actual scene, a suitable target is selected as a calibration point for zoom tracking, the geographic position information of the calibration point is obtained based on 3D-GIS and saved, and the distance between the camera and the calibration point is calculated;

[0033] Step 200, for all the calibration points, the camera is adjusted to different focal lengths for auto-focusing, the focal position corresponding to the focal length is obtained, the zoom tracking curve is fitted, and zoom tracking is realized.

[0034] Step 300, when the actual scene changes, repeat the steps 100 and 200 to quickly obtain the zoom tracking curve of the current scene, and improve the zoom tracking performance and imaging quality.

[0035] For step 100, please refer to Figure 2 The schematic diagram of the calibration target and the relative position of the camera in this embodiment. When selecting the calibration points, 10-20 targets are manually selected from close to the camera to far away from the camera as the calibration points, and the focal length and image distance of each calibration point are recorded. The distance of each calibration point to the camera is different.

[0036] Preferably, 10 targets are selected as calibration points in this embodiment.

[0037] Further, in step 100, the geographic position information of the calibration points is obtained based on the 3D-GIS map and saved, which further includes:

[0038] The data of the calibration points and the camera itself are read from the 3D-GIS map to obtain the latitude and longitude information of the camera and the calibration points, wherein the 3D-GIS map of the calibration points includes MIF and MID files, and the MIF and MID files include the spatial information of all spatial objects in the map and the attribute information of the road.

[0039] Further, the distance between the camera and the calibration points further includes: based on the obtained latitude and longitude information of the camera and the calibration points, the distance between all calibration points and the camera is calculated by the Euclidean distance calculation formula, which is used as the different object distances during calibration.

[0040] Specifically, in an embodiment of the present application, the 3D GIS map of the calibration points obtained in step 100 contains MIF and MID files, which contain the spatial information of all spatial objects in the map, the attribute information of the road, etc. The data of the calibration points and the camera itself in the map are read to obtain the latitude and longitude information of the camera and the calibration points. The latitude and longitude information obtained from the 3D GIS map and the Euclidean distance calculation formula are used to calculate the distance between all calibration points and the camera, which is used as the different object distances during calibration. The distance calculation method is as follows:

[0041] The latitude and longitude of the calibration point A and the camera position B are (latA, lonA) and (latB, lonB) respectively. The latitude difference and longitude difference between the two points are calculated:

[0042] dlat = latB - latA

[0043] dlon = lonB - lonA

[0044] The spherical distance between two points is calculated as:

[0045]

[0046]

[0047] distance = R x c

[0048] Where R is the radius of the earth, generally taken as 6371km. Through the above formula, the Euclidean distance between the calibration point and the camera is obtained. The distances of multiple calibration points and the camera are arranged roughly uniformly from near to far.

[0049] In step 100, the latitude and longitude information of the calibration point is obtained through the 3D-GIS map, and the position information between the calibration point and the camera is calculated according to the latitude and longitude information, which has small calculation amount and can be quickly applied to different actual application scenarios.

[0050] Further, step 200 includes the following sub-steps:

[0051] Step 201, for each calibration point selected in step 100, divide the focal length of the camera lens by 50, automatically focus on each focal length, and obtain the accurate focus position;

[0052] Step 202, using a curve fitting method, fit the discrete focal length and focus position point pairs into a continuous zoom tracking curve;

[0053] Step 203, after obtaining the zoom tracking curve under multiple object distances through step 202, for any object distance, using the traditional zoom tracking method, generate the zoom tracking curve under the corresponding object distance, control the focal length and image distance following the curve, and ensure that the image is clear at all times.

[0054] Specifically, for step 201, for each calibration point selected, divide the focal length of the camera lens by 50, automatically focus on each focal length, and obtain the accurate focus position. The method of automatic focusing of the camera at different focal lengths adopts contrast-based automatic focusing, sharpness-based automatic focusing, and neural network-based automatic focusing. The selected automatic focusing method is based on the content of the image, and a focus value is calculated. All focus positions are traversed, and the position with the maximum focus value is considered as the ideal focus point.

[0055] For step 202, as Figure 3After all the pairs of focal length and focus position are obtained at each calibration point, the pairs of focal length and focus position are fitted into a continuous zoom tracking curve using a curve fitting method.

[0056] In this embodiment, the zoom tracking curve fitting method adopts a polynomial fitting method and a spline interpolation method.

[0057] For step 203, after the zoom tracking curves at multiple object distances are obtained through step 202, it is obviously impossible to obtain the zoom tracking curve at any object distance through calibration. When actually applied, the object can be automatically focused at a certain focal length once, and the object distance can be determined through the following formula:

[0058]

[0059] In the formula, f represents the focal length, and d and v represent the object distance and the image distance. It can be seen that when the focal length and the image distance are determined, the object distance can also be uniquely determined. Then, a zoom tracking curve at the corresponding object distance is generated using a traditional zoom tracking method, as shown by the dashed line in FIG. 1, and the focal length and the image distance are controlled along the curve, so that the image is always in focus. Figure 3

[0060] In this embodiment, the zoom tracking method adopts a geometric zoom tracking method, an adaptive zoom tracking method, and a feedback zoom tracking method.

[0061] In this embodiment, the changes in the environment described in step 300 include temperature, humidity, altitude, and internal stress of the camera. Since the working environment of the camera is uncontrollable, such as high mountains, oceans, and cities, these changes will cause the in-focus position to change under the same object distance and focal length, and the zoom tracking curve needs to be recalibrated. Based on the steps of this embodiment, the zoom tracking curve can be quickly recalibrated in any scene, the influence of the environment on the camera is avoided, and therefore a higher-quality imaging effect can be obtained.

[0062] In a second aspect, the present application further provides a camera calibration and zoom tracking device with scene adaptation, comprising:

[0063] An information acquisition module is configured to select a suitable target as a calibration point for zoom tracking according to an actual scene, acquire geographical position information of the calibration point based on 3D-GIS and save the information, and calculate a distance between the camera and the calibration point.

[0064] A zoom tracking module is configured to automatically focus the camera at different focal lengths for all the calibration points, acquire focus positions corresponding to the focal lengths, fit a zoom tracking curve, and realize zoom tracking.

[0065] ​A circulation module is configured to repeat the steps 100 and 200 when the application scene changes, so as to quickly obtain a zoom tracking curve of the current scene, and improve zoom tracking performance and imaging quality.

[0066] The above examples of the present application are only used to illustrate the technical solutions of the present application, and are not intended to limit the embodiments of the present application. Based on the above description, other different forms of changes or variations can be made by those of ordinary skill in the art, and it is impossible to exhaust all embodiments here. Any obvious changes or variations derived from the technical solutions of the present application are still within the protection scope of the present application.

Claims

1. A method for camera calibration and zoom tracking with scene adaptation, the method comprising: The method comprises the following steps: Step 100, selecting appropriate targets as calibration points for zoom tracking according to an actual scene, obtaining geographical position information of the calibration points based on 3D-GIS and saving the information, and calculating distances between the camera and the calibration points; The step of selecting appropriate targets as calibration points for zoom tracking according to an actual scene further comprises: manually selecting 10-20 targets as calibration points from close to the camera to far away from the camera, and recording focal lengths and image distances of each calibration point; The step of obtaining geographical position information of the calibration points based on 3D-GIS and saving the information further comprises: The step of obtaining data of the calibration points and the camera itself from a 3D-GIS map to obtain longitude and latitude information of the camera and the calibration points, wherein the 3D-GIS map of the calibration points comprises MIF and MID files, and the MIF and MID files comprise spatial information of all spatial objects in the map and attribute information of roads; The step of calculating distances between the camera and the calibration points further comprises: calculating distances between all the calibration points and the camera based on the obtained longitude and latitude information of the camera and the calibration points by using a Euclidean distance calculation formula, and using the distances as different object distances during calibration; Step 200, for all the calibration points, the camera is adjusted to different focal lengths for automatic focusing, focal positions corresponding to the focal lengths are obtained, a zoom tracking curve is fitted, and zoom tracking is realized; Step 300, when the actual scene changes, the steps 100 and 200 are repeated to quickly obtain a zoom tracking curve of the current scene, and zoom tracking performance and imaging quality are improved.

2. The method of claim 1, wherein, The distances of the calibration points to the camera are different.

3. The method of claim 1, wherein, The step 200 comprises the following sub-steps: Step 201, for each calibration point selected in the step 100, the focal length of the camera lens is equally divided into 50 parts, automatic focusing is performed on each focal length, and accurate focal positions are obtained; Step 202, using a curve fitting method, discrete focal lengths and focal position points are fitted into a continuous zoom tracking curve; Step 203, after the zoom tracking curves under multiple object distances are obtained in the step 202, for any object distance, a zoom tracking curve under the corresponding object distance is generated by using a traditional zoom tracking method, the focal length and the image distance are controlled following the curve, and the image is ensured to be clear at all times.

4. The method of claim 3, wherein, In the step 201, the automatic focusing method of the camera at different focal lengths comprises contrast-based automatic focusing, sharpness-based automatic focusing and / or neural network-based automatic focusing.

5. The method of claim 3, wherein, In the step 202, the zoom tracking curve fitting method comprises a polynomial fitting method and / or a spline interpolation method.

6. The method of claim 3, wherein, In the step 203, the traditional zoom tracking method comprises a geometric zoom tracking method, an adaptive zoom tracking method and / or a feedback zoom tracking method.

7. A camera calibration and zoom tracking apparatus with scene adaptation, comprising: The method comprises: An information acquisition module is configured to select appropriate targets as calibration points for zoom tracking according to an actual scene, obtain geographical position information of the calibration points based on 3D-GIS and save the information, and calculate distances between the camera and the calibration points. According to the actual scene, selecting appropriate targets as calibration points of zoom tracking further comprises: manually selecting 10-20 targets from close to the camera to far away from the camera as calibration points, and recording the focal length and image distance of each calibration point; Based on the 3D-GIS map, the geographic location information of the calibration points is obtained and saved, which further comprises: The data of the calibration points and the camera itself are obtained from the 3D-GIS map to obtain the latitude and longitude information of the camera and the calibration points, wherein the 3D-GIS map of the calibration points includes MIF and MID files, and the MIF and MID files include the spatial information of all spatial objects in the map and the attribute information of the road; The distance between the camera and the calibration points is calculated, which further comprises: based on the obtained latitude and longitude information of the camera and the calibration points, the distance between all calibration points and the camera is calculated by the Euclidean distance calculation formula, which is used as different object distances during calibration; The zoom tracking module is used to automatically focus on different focal lengths for all calibration points, obtain the focusing position corresponding to the focal length, fit the zoom tracking curve, and realize zoom tracking. The cycle module is used to repeat steps 100 and 200 when the application scene changes, quickly obtain the zoom tracking curve of the current scene, and improve the zoom tracking performance and imaging quality.

Images