A method and device for controlling the association between an eagle-eye camera and a GIS map

Abstract

The present invention discloses a method and device for controlling the association between an eagle-eye camera and a GIS map. The invention includes: determining a target calibration point according to a target calibration threshold and a target point; obtaining camera information, calibration point information and target point geographic coordinates, wherein the camera information includes the camera initial state, the camera geographic coordinates, the camera imaging model and the sensor size, and the calibration point information includes the calibration point geographic coordinates and the calibration point pixel coordinates; obtaining the target horizontal offset and the target vertical offset according to the camera's initial horizontal offset, initial vertical offset and the target point geographic coordinates; adjusting the camera according to the target horizontal offset and the target vertical offset. The present invention solves the problem of inaccurate camera viewing angle adjustment position caused by image blur or image distortion in the related art.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a method and device for associating and controlling an eagle-eye camera with a GIS map. Background Art

[0002] Hawkeye technology has been widely used in many fields, including security monitoring, traffic management, sports competition, etc. In the construction of information systems such as real-life maps, rotating the direction of the ball camera and focusing on the designated target area and picture according to the site selection on the GIS map are the keys to achieve remote, accurate and rapid monitoring.

[0003] There are currently a variety of solutions to solve the problem of adjusting the camera's viewing angle. The two most commonly used methods are based on sensor models and machine vision technology. The conversion method based on the sensor model has high requirements for the accuracy of the camera's internal and external parameters. If the parameter calibration is inaccurate or changes, the conversion result may be biased. The conversion method based on machine vision uses image feature extraction and matching technology. If the image is blurred, the features are not obvious, or the feature points are unevenly distributed, the conversion accuracy may decrease. In addition, the initial state of the camera is unknown, and some cameras are not ideal in dealing with image distortion. These factors may lead to inaccurate calculation results of coordinate transformation.

[0004] In the related art, the technical problem of inaccurate camera viewing angle adjustment position caused by image blur or image distortion has not yet been effectively solved. Summary of the invention

[0005] The main purpose of the present application is to provide a method and device for controlling the association between an Eagle Eye camera and a GIS map, so as to at least solve the problem of inaccurate camera viewing angle adjustment position caused by image blur or image distortion in the related art.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for controlling the association between an Eagle Eye camera and a GIS map is provided. The method comprises: determining a target calibration point according to a target calibration threshold and a target point; obtaining camera information, calibration point information and target point geographic coordinates, wherein the camera information comprises the camera initial state, the camera geographic coordinates, the camera imaging model and the sensor size, and the calibration point information comprises the calibration point geographic coordinates and the calibration point pixel coordinates; obtaining a target horizontal offset and a target vertical offset according to the camera's initial horizontal offset, initial vertical offset and the target point geographic coordinates; and adjusting the camera according to the target horizontal offset and the target vertical offset.

[0007] Optionally, in step S201, the camera's field of view is divided into at least one calibration image, and initial calibration points are selected on the calibration images, wherein at least one initial calibration point is calibrated on one calibration image; in step S202, experimental points are marked on the GIS map and the real scene according to the geographic information of the initial calibration points, wherein one initial calibration point corresponds to one experimental point; in step S203, the accuracy of the initial calibration points and the experimental points corresponding to them is determined; in step S204, when the initial calibration points and the experimental points corresponding to them are inaccurate, steps S202-S203 are repeated; in step S205, when the initial calibration points and the experimental points corresponding to them are accurate, the calibration distance of the initial calibration point closest to the target point is obtained, and it is determined whether the calibration distance is less than the target calibration threshold; in step S206, when the calibration distance is greater than or equal to the target calibration threshold, steps S202-S205 are repeated; in step S207, when the calibration distance is less than the target calibration threshold, the initial calibration point corresponding to the calibration distance is determined to be the target calibration point.

[0008] Optionally, the target calibration threshold is calculated using the formula: ,in, Set the threshold for the target. is the horizontal viewing angle of the camera, is the horizontal distance between the camera and the target calibration point.

[0009] Optionally, a camera imaging model is established, which includes an image plane, a world coordinate system, and an optical center horizontal plane. The image plane includes a pixel coordinate system and an image coordinate system. The origin of the pixel coordinate system is the upper left corner of the image plane, and the origin of the image coordinate system is the center point of the image plane. The image plane is perpendicular to the optical axis of the camera, and the optical axis of the camera passes through the center point. The focal length is calculated using the following formula: , is the width of the camera sensor, is the horizontal viewing angle of the camera, and the focal length is the distance from the camera to the center point of the image plane; the vertical angle between the target calibration point and the horizontal plane of the optical center is calculated using the formula: , is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the height difference between the camera and the target calibration point, is the horizontal distance between the camera and the target calibration point; the distance from the camera to the target calibration point is calculated using the formula: , is the distance from the camera to the target calibration point, is the focal length, is the pixel horizontal coordinate of the target calibration point, The pixel ordinate of the target calibration point.

[0010] Optionally, calculate the angle between the target calibration point and the true north direction, and the calculation formula is: ,in, is the angle between the target calibration point and the true north direction, is the distance between the target calibration point and the camera, is the vertical distance of the target calibration point in the north-south direction of the camera; calculate the distance between the focus of the camera and the target calibration point perpendicular to the horizontal plane of the optical center, and the calculation formula is , is the distance between the camera and the focus of the target calibration point perpendicular to the horizontal plane of the optical center, is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the distance from the camera to the target calibration point; calculate the horizontal angle between the target calibration point and the current camera optical axis, the calculation formula is , is the horizontal angle between the target calibration point and the current camera optical axis, is the horizontal coordinate of the target calibration point on the image plane; calculate the initial horizontal offset, the calculation formula is ,in, is the initial horizontal offset, The horizontal rotation angle of the camera.

[0011] Optionally, calculate the angle between the target calibration point and the optical axis relative to the camera using the formula: , is the angle between the target calibration point and the optical axis relative to the camera, is the pixel ordinate of the target calibration point, is the focal length; calculate the initial vertical offset, the calculation formula is ,in, is the initial vertical offset, is the vertical angle between the target calibration point and the horizontal plane of the optical center, The vertical rotation angle of the camera.

[0012] Optionally, calculate the angle between the target point and the north direction. The calculation formula is: ,in, is the angle between the target point and the true north direction, is the distance between the target point and the camera, is the vertical distance of the target point in the north-south direction of the camera; calculate the vertical angle between the target point and the horizontal plane of the optical center, and the calculation formula is ,in, is the vertical angle between the target point and the horizontal plane of the optical center, is the height difference between the camera and the target point, is the horizontal distance between the camera and the target point; the target horizontal offset is calculated using the formula: ,in, is the target horizontal offset, is the initial horizontal offset, is the horizontal rotation angle of the camera; calculate the vertical offset of the target, the calculation formula is , is the target vertical offset, is the vertical rotation angle of the camera, is the initial vertical offset.

[0013] According to another aspect of the present application, a device for controlling the association between an Eagle Eye camera and a GIS map is provided. The device comprises:

[0014] A determination unit is used to determine a target calibration point according to a target calibration threshold and a target point; a first acquisition unit is used to acquire camera information, calibration point information and geographic coordinates of the target point, wherein the camera information includes an initial state of the camera, the geographic coordinates of the camera, a camera imaging model and a sensor size, and the calibration point information includes the geographic coordinates of the calibration point and the pixel coordinates of the calibration point; a second acquisition unit is used to acquire a target horizontal offset and a target vertical offset according to an initial horizontal offset and an initial vertical offset of the camera and the geographic coordinates of the target point; an adjustment unit is used to adjust the camera according to the target horizontal offset and the target vertical offset.

[0015] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium comprising a stored program, wherein the program executes any one of the above-mentioned methods for associating and controlling an Eagle Eye camera with a GIS map.

[0016] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include a method for controlling the association of an Eagle Eye camera with a GIS map for executing any one of the items.

[0017] Through the present application, the following steps are adopted: determining the target calibration point according to the target calibration threshold and the target point; obtaining camera information, calibration point information and target point geographic coordinates, wherein the camera information includes the camera initial state, the camera geographic coordinates, the camera imaging model and the sensor size, and the calibration point information includes the calibration point geographic coordinates and the calibration point pixel coordinates; obtaining the target horizontal offset and the target vertical offset according to the camera's initial horizontal offset, the initial vertical offset and the target point geographic coordinates; adjusting the camera according to the target horizontal offset and the target vertical offset, thereby solving the problem of inaccurate camera viewing angle adjustment position caused by image blur or image distortion in the related art, thereby achieving the effect that when adjusting the camera viewing angle, the coordinate calculation is not disturbed by image deformation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of a method for associating and controlling an Eagle Eye camera with a GIS map according to an embodiment of the present application;

[0020] Figure 2 It is a flow chart for confirming the target calibration point;

[0021] Figure 3 is a schematic diagram of the camera imaging model;

[0022] Figure 4 It is a schematic diagram of the pixel coordinate system and the image coordinate system;

[0023] Figure 5 It is a schematic diagram of the camera imaging model with auxiliary lines added;

[0024] Figure 6 A schematic diagram of the vertical angle between the target calibration point and the horizontal plane;

[0025] Figure 7 is a schematic diagram of the initial horizontal offset;

[0026] Figure 8 A schematic diagram of the angle between the target calibration point and the true north direction;

[0027] Fig. 9 A plane diagram showing the angle between the target calibration point and the optical axis relative to the camera;

[0028] Fig.10 It is a structural block diagram of a device for associating and controlling an Eagle Eye camera with a GIS map provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] As introduced in the background technology, in the prior art, when adjusting the camera viewing angle, the coordinate conversion calculation results are inaccurate due to problems such as unclear images and unclear features. In order to solve the problem of inaccurate camera viewing angle adjustment position caused by image blur or image distortion, the embodiments of the present application provide a method and device for associating and controlling an Eagle Eye camera with a GIS map.

[0033] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0034] In this embodiment, a method for controlling the association between an Eagle Eye camera and a GIS map running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0035] Figure 1 FIG. 1 is a flow chart of a method for controlling the association between an Eagle Eye camera and a GIS map according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0036] Step S101, determining a target calibration point according to a target calibration threshold and a target point;

[0037] Specifically, the camera calibration function is used to manually calibrate the camera, calculate the initial state parameters of the camera, take a video screenshot of the monitoring device, manually mark any point in the image, and obtain the pixel coordinates of the point. The overall pixel of the image can be defined as needed, as long as the entire calculation system follows one method. This method is defined as 680*480. At the same time, the GPS coordinates corresponding to the target calibration point are found. Based on the information of the calibration point closest to the target point, the rotation angle required to observe the target point is reversely calculated.

[0038] Step S102, obtaining camera information, calibration point information and target point geographic coordinates, wherein the camera information includes the camera initial state, camera geographic coordinates, camera imaging model and sensor size, and the calibration point information includes the calibration point geographic coordinates and calibration point pixel coordinates;

[0039] Specifically, the API interface is called to obtain various parameters of the camera, and the obtained parameters provide a data basis for subsequent calculations. Or the obtained parameters are used to perform simple calculations to obtain basic parameters for subsequent calculations.

[0040] Step S103, obtaining a target horizontal offset and a target vertical offset according to the initial horizontal offset and the initial vertical offset of the camera and the geographic coordinates of the target point;

[0041] Specifically, by selecting a target calibration point and calculating an initial horizontal offset and an initial vertical offset corresponding to the target calibration point, a vertical angle between the target point and the horizontal plane is obtained, and then a target horizontal offset and a target vertical offset are calculated.

[0042] Step S104, adjusting the camera according to the target horizontal offset and the target vertical offset;

[0043] Specifically, with the target horizontal offset and target vertical offset, the latest PT value can be passed to the camera, and the camera gimbal automatically adjusts the angle of the Hawkeye camera according to the new PT to achieve automatic linkage between the geographic location and the detail camera. Tracking the real scene near the target point, real-time monitoring around the target point can improve the efficiency of urban management and save the cost of manual inspection.

[0044] In the process of using this embodiment, the PTZ user uses the mouse to select a point on the GIS map (the visible range of the Hawkeye panoramic camera is mapped to the area on the map) and then operates the Hawkeye detail ball camera PTZ to rotate and focus. First, the display screen is a panoramic screen composed of multiple Hawkeye gun cameras. The pop-up window on the screen displays a two-dimensional map. The fan-shaped area is the field of view mapped by the gun camera on the GIS map. If you want to see a specific location, you can double-click the corresponding location on the map in the pop-up window. The ball camera rotates and calls up the monitoring screen centered on the clicked location.

[0045] The present invention utilizes algorithms such as optical imaging principles, the distance between two points on a sphere, and the principle of similar triangles to enable Hawkeye to quickly follow up and capture any target position on a map, and to display a high-definition picture near the target point by calling a video stream. The core algorithm of the present invention involves calculating the horizontal and vertical viewing angles of the camera adjusted from the current state to any target point, and responding to the challenges of different camera types and states with a more accurate and comprehensive algorithm. Specifically, it includes: 1. Calculating the camera initial state values ​​of multiple groups of calibration points using known parameters such as the initial state of the camera, the longitude and latitude of the camera installation location, the geographic location coordinates and pixel coordinates of the calibration point, the size and viewing angle of the camera sensor; 2. Determine the selection criteria and thresholds of the target calibration point using the principles of the distance between two points on a sphere and local similarity, and save the cost of manual calibration while improving the calibration accuracy; 3. Use the camera initial state values ​​of the above steps to calculate the horizontal rotation angle and vertical rotation angle that need to be adjusted based on the current state of the camera to capture the high-definition picture of the target point.

[0046] Through this embodiment, the optical imaging principle can be used to calculate the angle that the camera of the target point needs to be adjusted according to the geographic point coordinates, camera coordinates, and camera parameters through calibration methods and relationship formulas. Using the imaging principle and local similarity, the determination rules of the target calibration points are formulated to improve the calibration accuracy and efficiency. The angle of the ball camera and the zoom value Z of the camera can also be adjusted through page navigation. This calculation theory and related conversion formulas are converted into the application of GIS maps and camera orientations to achieve the effect of "pointing to see where to go, and real-time monitoring of the real scene", saving technical training and guidance costs.

[0047] In an optional embodiment, step S201, divide the field of view of the camera into at least one calibration image, select initial calibration points on the calibration image, wherein at least one initial calibration point is calibrated on one calibration image; step S202, mark the experimental points on the GIS map and the real scene according to the geographic information of the initial calibration points, wherein one initial calibration point corresponds to one experimental point; step S203, determine the accuracy of the initial calibration points and the experimental points corresponding to them; step S204, when the initial calibration points and the experimental points corresponding to them are inaccurate, repeat steps S202-S203; step S205, when the initial calibration points and the experimental points corresponding to them are accurate, obtain the calibration distance of the initial calibration point closest to the target point, and determine whether the calibration distance is less than the target calibration threshold; step S206, when the calibration distance is greater than or equal to the target calibration threshold, repeat steps S202-S205; step S207, when the calibration distance is less than the target calibration threshold, determine that the initial calibration point corresponding to the calibration distance is the target calibration point.

[0048] like Figure 2 As shown, Figure 2The flowchart for confirming the target calibration point is as follows: Specifically, the calibration point process: for the horizontal (360°) and vertical (0°-90°) field of view covered by the camera, select multiple images for calibration to cover all field of view. For one image, first calibrate a few points, use geographic information to scatter points on the GIS map and the real scene, judge the accuracy of the calibrated points, and repeat the above process until the information position in the above image is relatively accurate. The accuracy of the calibration will be affected by factors such as the calibration personnel's operating experience, proficiency, and technical knowledge background. The number of calibration points varies, and about 4 to 10 points are set for one image. Determination of the target calibration point (the calibration point closest to the target point and meeting the threshold): First, calculate the distance between all calibration points and the target point, get the nearest calibration point, and then determine whether the distance meets the specified threshold. Repeat the above calibration operation until a calibration point that meets the conditions is found.

[0049] In an optional embodiment, the target calibration threshold is calculated using the formula: ,in, Set the target threshold. is the horizontal viewing angle of the camera, is the horizontal distance between the camera and the target calibration point.

[0050] Specifically, the sensor size width of the camera is W. Here, the distance between the target calibration point that meets the conditions and the target point in the image coordinate system is less than W / 2. The smaller the value, the more accurate it is, but the calibration workload increases relatively. According to the relationship between the horizontal distance between the camera and the target calibration point (L, calculated according to the coordinates), the focal length (R), and the threshold set in the image coordinate system (W / 2, here 4mm), the formula is: Calculate the approximate threshold of the distance between the target point and the target calibration point that meets the conditions: , substitute R into Formula, we get: Through the above-mentioned target calibration point and target point determination rule, not only the accuracy of camera calibration is improved, but also the calibration efficiency is improved.

[0051] In an optional embodiment, a camera imaging model is established, which includes an image plane, a world coordinate system, and an optical center horizontal plane. The image plane includes a pixel coordinate system and an image coordinate system. The origin of the pixel coordinate system is the upper left corner of the image plane, and the origin of the image coordinate system is the center point of the image plane. The image plane is perpendicular to the optical axis of the camera, and the optical axis of the camera passes through the center point. The focal length is calculated using the following formula: , is the width of the camera sensor, is the horizontal viewing angle of the camera, and the focal length is the distance from the camera to the center point of the image plane; the vertical angle between the target calibration point and the horizontal plane of the optical center is calculated using the formula: , is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the height difference between the camera and the target calibration point, is the horizontal distance between the camera and the target calibration point; the distance from the camera to the target calibration point is calculated using the formula: , is the distance from the camera to the target calibration point, is the focal length, is the pixel horizontal coordinate of the target calibration point, The pixel ordinate of the target calibration point.

[0052] Above It is a known parameter that can be obtained. The height difference between the camera and the target calibration point can be obtained through the gimbal interface. If the target calibration point is on the ground, the target calibration point height is 0, and △H is equal to the camera height; if it is far away from the ground, the target calibration point height is based on the floor height of the building. Generally, the height of one floor is set to 3 meters. The target calibration point height is approximately obtained by the number of floors * 3 meters.

[0053] like Figure 3 As shown, Figure 3 It is a schematic diagram of the camera imaging model. For ease of understanding, the camera imaging model is introduced. Coordinate system description: C (camera first letter) point is the optical center of the camera (projection center), C-XcYcZc is the camera coordinate system. The intersection of the optical axis and the image plane is the principal point O of the image, O-xy is the image coordinate system, the optical axis is perpendicular to the image plane, and CO is the camera focal length R. In pixels, the upper left corner of the image is the origin of the pixel coordinate system o-uv. The point P0 geographic coordinate (lgt, lat) in the camera coordinate is Pc (Xc, Yc, Zc), and the light through the projection center is projected onto the image plane. The corresponding image point is p (x, y), and the pixel coordinate is P (u, v).

[0054] like Figure 4 As shown, Figure 4 This is a schematic diagram of the pixel coordinate system and the image coordinate system. If the size of the CCD sensor inside the camera is 8mmx6mm, the final image photo is 640x480 pixels. For the convenience of calculation, we simplify the model and use the image coordinate system O-xy for reference calculation.

[0055] like Figure 5 , Figure 5 This is a schematic diagram of the camera imaging model with auxiliary lines added. From the right angle △CP3P2 of the horizontal plane in the figure, the formula can be obtained: , we get: , is a known value, Unknown, continue to solve .

[0056] From the right angle △PP2C of the vertical plane in the figure, we can get the formula: ;

[0057] In the vertical plane right angle △COP1, the formula is obtained: ;

[0058] In the cross-section right angle △CP1P, the formula is obtained: ;

[0059] Combined .

[0060] Calculated based on sensor size and viewing angle:

[0061] The focal length R can be approximated using the following formula from the camera’s sensor size (e.g., width W) and horizontal viewing angle (e.g., θ):

[0062] Figure 6 The vertical angle between the target calibration point and the horizontal plane is shown in Figure 2. The height difference between the camera and the target calibration point is △H. The horizontal distance L between the two points can be obtained from the longitude and latitude coordinates of the camera and the target calibration point, and the angle can be obtained. :

[0063]

[0064] The final calculation is :

[0065]

[0066] In an optional embodiment, the angle between the target calibration point and the true north direction is calculated using the following formula: ,in, is the angle between the target calibration point and the true north direction, is the distance between the target calibration point and the camera, is the vertical distance of the target calibration point in the north-south direction of the camera; calculate the distance between the focus of the camera and the target calibration point perpendicular to the horizontal plane of the optical center, and the calculation formula is , is the distance between the camera and the focus of the target calibration point perpendicular to the horizontal plane of the optical center, is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the distance from the camera to the target calibration point; calculate the horizontal angle between the target calibration point and the current camera optical axis, the calculation formula is , is the horizontal angle between the target calibration point and the current camera optical axis, is the horizontal coordinate of the target calibration point on the image plane; calculate the initial horizontal offset, the calculation formula is ,in, is the initial horizontal offset, The horizontal rotation angle of the camera.

[0067] Specifically, Figure 7 As shown, Figure 7 The following is a schematic diagram of the initial horizontal offset. The steps for calculating the initial horizontal offset are briefly described as follows: First, the north direction is taken as the reference 0° and the east direction is taken as the horizontal 90°. The API interface is called to obtain the value of the horizontal rotation angle dp of the camera PTZ. The angle between the target calibration point and the north direction is calculated using the camera position coordinates and the geographic coordinates of the target calibration point. , through the principle of optical imaging, the correspondence relationship of similar triangles, and the known parameters such as the pixel coordinates of the target calibration point, the horizontal angle gx between the target calibration point and the current camera is calculated, and then the initial horizontal offset pan is calculated through the relationship between the angles.

[0068] like Figure 8 , Figure 8 is a schematic diagram of the angle between the target calibration point and the true north direction. is the distance between the target calibration point and the camera, It is the distance between the projection P1 of the target calibration point and the camera. The distance between the two points can be calculated by longitude and latitude.

[0069] To calculate the distance between the longitude and latitude of two points, we can use the Haversine formula. This formula gives the approximate shortest distance between two points based on the great circle distance on a sphere (i.e., the "straight line" distance along the surface of the Earth). Suppose we have two points A and B with longitude and latitude (lat1, lon1) and longitude (lat2, lon2) in degrees. The Haversine formula is as follows:

[0070] 1. Convert longitude and latitude from degrees to radians:

[0071] lat1,lon1,lat2,lon2= , , ,

[0072] 2. Calculate the difference in longitude and latitude between two points:

[0073]

[0074]

[0075] 3. Use the Haversine formula to calculate the distance:

[0076]

[0077]

[0078]

[0079] Where R is the average radius of the Earth (about 6371 km).

[0080] In an optional embodiment, the angle between the target calibration point and the optical axis relative to the camera is calculated using the formula: , is the angle between the target calibration point and the optical axis relative to the camera, is the pixel ordinate of the target calibration point, is the focal length; calculate the initial vertical offset, the calculation formula is ,in, is the initial vertical offset, is the vertical angle between the target calibration point and the horizontal plane of the optical center, The vertical rotation angle of the camera.

[0081] Reference Figure 5 as well as Fig. 9 , calculate the angle between the target calibration point and the optical axis relative to the camera:

[0082] From △P1CO, we get:

[0083]

[0084] but,

[0085] y is the image coordinate y value of the target calibration point, R is the focal length, and the above formula 6 has a solution process to obtain the final gy value:

[0086]

[0087] In an optional embodiment, the angle between the target point and the true north direction is calculated using the following formula: ,in, is the angle between the target point and the north direction, is the distance between the target point and the camera, is the vertical distance of the target point in the north-south direction of the camera; calculate the vertical angle between the target point and the horizontal plane of the optical center, and the calculation formula is ,in, is the vertical angle between the target point and the horizontal plane of the optical center, is the height difference between the camera and the target point, is the horizontal distance between the camera and the target point; the target horizontal offset is calculated using the formula: ,in, is the target horizontal offset, is the initial horizontal offset, is the horizontal rotation angle of the camera; calculate the vertical offset of the target, the calculation formula is , is the target vertical offset, is the vertical rotation angle of the camera, is the initial vertical offset.

[0088] Specifically, The target point coordinates and camera coordinates can be calculated; It is the horizontal offset angle of the current camera and can be obtained through the camera API interface. It is the target calibration point that is closest to the target point and meets the threshold. From this target calibration point, the horizontal offset angle of the camera's initial state can be obtained. . Thus, the horizontal offset angle of the target point is obtained , relative to the current optical axis, the horizontal camera adjustment Spend.

[0089] is the angle between the target point to be solved and the current optical axis, The vertical offset angle of the current camera, which can be obtained through the camera API interface. is the initial state value of the target calibration point. is the vertical angle between the target point and the horizontal direction, which is unknown. The following solution is given This process is similar to the process of calculating the initial state of the camera, replacing the calibration point with the target point. .

[0090] In addition, the image coordinates and pixel coordinates of the target point are obtained from the image imaging model and similarity, which is the reverse calculation of the calibration algorithm.

[0091] The embodiment of the present application also provides a device for associating and controlling an eagle-eye camera with a GIS map. It should be noted that the device for associating and controlling an eagle-eye camera with a GIS map in the embodiment of the present application can be used to execute the method for associating and controlling an eagle-eye camera with a GIS map provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been explained will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0092] The following introduces a device for associating and controlling an Eagle Eye camera with a GIS map provided in an embodiment of the present application.

[0093] Fig.10 1 is a structural block diagram of a device for controlling the association between an Eagle Eye camera and a GIS map according to an embodiment of the present application. Fig.10 As shown, the device includes: a determination unit 1001, used to determine the target calibration point according to the target calibration threshold and the target point; a first acquisition unit 1002, used to acquire camera information, calibration point information and target point geographic coordinates, wherein the camera information includes the camera initial state, the camera geographic coordinates, the camera imaging model and the sensor size, and the calibration point information includes the calibration point geographic coordinates and the calibration point pixel coordinates; a second acquisition unit 1003, used to acquire the target horizontal offset and the target vertical offset according to the initial horizontal offset and the initial vertical offset of the camera and the target point geographic coordinates; an adjustment unit 1004, used to adjust the camera according to the target horizontal offset and the target vertical offset.

[0094] In an optional embodiment, the determination unit 1001 includes: a determination subunit, which is used to execute step S201, divide the field of view of the camera into at least one calibration image, select initial calibration points on the calibration image, wherein at least one initial calibration point is calibrated on one calibration image; step S202, mark the experimental points on the GIS map and the real scene according to the geographic information of the initial calibration points, wherein one initial calibration point corresponds to one experimental point; step S203, determine the accuracy of the initial calibration point and the corresponding experimental point; step S2 04, when the initial calibration point and the corresponding experimental point are inaccurate, repeat steps S202-S203; step S205, when the initial calibration point and the corresponding experimental point are accurate, obtain the calibration distance of the initial calibration point closest to the target point, and determine whether the calibration distance is less than the target calibration threshold; step S206, when the calibration distance is greater than or equal to the target calibration threshold, repeat steps S202-S205; step S207, when the calibration distance is less than the target calibration threshold, determine that the initial calibration point corresponding to the calibration distance is the target calibration point.

[0095] In an optional embodiment, the determination unit 1001 includes: a first calculation subunit, configured to calculate a target calibration threshold, wherein the calculation formula is: ,in, Set the target threshold. is the horizontal viewing angle of the camera, is the horizontal distance between the camera and the target calibration point.

[0096] In an optional embodiment, the first acquisition unit 1002 includes: an establishment subunit, which is used to establish a camera imaging model, the camera imaging model includes an image plane, a world coordinate system and an optical center horizontal plane, the image plane includes a pixel coordinate system and an image coordinate system, the origin of the pixel coordinate system is the upper left corner of the image plane, the origin of the image coordinate system is the center point of the image plane, the image plane is perpendicular to the optical axis of the camera, and the optical axis of the camera passes through the center point; a second calculation subunit, which is used to calculate the focal length, and the calculation formula is: , is the width of the camera sensor, is the horizontal viewing angle of the camera, and the focal length is the distance from the camera to the center point of the image plane; the third calculation subunit is used to calculate the vertical angle between the target calibration point and the horizontal plane of the optical center, and the calculation formula is: , is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the height difference between the camera and the target calibration point, is the horizontal distance between the camera and the target calibration point; the fourth calculation subunit is used to calculate the distance from the camera to the target calibration point, and the calculation formula is: , is the distance from the camera to the target calibration point, is the focal length, is the pixel horizontal coordinate of the target calibration point, The pixel ordinate of the target calibration point.

[0097] In an optional embodiment, the second acquisition unit 1003 includes: a fifth calculation subunit, which is used to calculate the angle between the target calibration point and the true north direction, and the calculation formula is: ,in, is the angle between the target calibration point and the true north direction, is the distance between the target calibration point and the camera, is the vertical distance of the target calibration point in the north-south direction of the camera; the sixth calculation subunit is used to calculate the distance between the camera and the focus of the target calibration point perpendicular to the horizontal plane of the optical center. The calculation formula is , is the distance between the camera and the focus of the target calibration point perpendicular to the horizontal plane of the optical center, is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the distance from the camera to the target calibration point; the seventh calculation subunit is used to calculate the horizontal angle between the target calibration point and the current camera optical axis, and the calculation formula is: , is the horizontal angle between the target calibration point and the current camera optical axis, is the horizontal coordinate of the target calibration point on the image plane; the eighth calculation subunit is used to calculate the initial horizontal offset, and the calculation formula is: ,in, is the initial horizontal offset, The horizontal rotation angle of the camera.

[0098] In an optional embodiment, the second acquisition unit 1003 includes: a ninth calculation subunit, which is used to calculate the angle between the target calibration point and the optical axis relative to the camera, and the calculation formula is: , is the angle between the target calibration point and the optical axis relative to the camera, is the pixel ordinate of the target calibration point, is the focal length; the tenth calculation subunit is used to calculate the initial vertical offset, and the calculation formula is: ,in, is the initial vertical offset, is the vertical angle between the target calibration point and the horizontal plane of the optical center, The vertical rotation angle of the camera.

[0099] In an optional embodiment, the second acquisition unit 1003 includes: an eleventh calculation subunit, which is used to calculate the angle between the target point and the true north direction, and the calculation formula is: ,in, is the angle between the target point and the true north direction, is the distance between the target point and the camera, is the vertical distance of the target point in the north-south direction of the camera; the twelfth calculation subunit is used to calculate the vertical angle between the target point and the horizontal plane of the optical center. The calculation formula is: ,in, is the vertical angle between the target point and the horizontal plane of the optical center, is the height difference between the camera and the target point, is the horizontal distance between the camera and the target point; the thirteenth calculation subunit is used to calculate the horizontal offset of the target, and the calculation formula is: ,in, is the target horizontal offset, is the initial horizontal offset, is the horizontal rotation angle of the camera; the fourteenth calculation subunit is used to calculate the vertical offset of the target, and the calculation formula is: , is the target vertical offset, is the vertical rotation angle of the camera, is the initial vertical offset.

[0100] The device for controlling the association of an eagle-eye camera with a GIS map includes a processor and a memory, wherein the determination unit 1001 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0101] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the technical problem of inaccurate camera viewing angle adjustment position caused by image blur or image distortion can be solved by adjusting kernel parameters.

[0102] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0103] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute the method for controlling the association between an Eagle Eye camera and a GIS map.

[0104] Specifically, a method for associating and controlling an Eagle Eye camera with a GIS map includes:

[0105] Determine the target calibration point according to the target calibration threshold and the target point; obtain camera information, calibration point information and target point geographic coordinates, wherein the camera information includes the camera initial state, camera geographic coordinates, camera imaging model and sensor size, and the calibration point information includes the calibration point geographic coordinates and the calibration point pixel coordinates; obtain the target horizontal offset and the target vertical offset according to the camera's initial horizontal offset, initial vertical offset and the target point geographic coordinates; adjust the camera according to the target horizontal offset and the target vertical offset.

[0106] Optionally, in step S201, the camera's field of view is divided into at least one calibration image, and initial calibration points are selected on the calibration images, wherein at least one initial calibration point is calibrated on one calibration image; in step S202, experimental points are marked on the GIS map and the real scene according to the geographic information of the initial calibration points, wherein one initial calibration point corresponds to one experimental point; in step S203, the accuracy of the initial calibration points and the experimental points corresponding to them is determined; in step S204, when the initial calibration points and the experimental points corresponding to them are inaccurate, steps S202-S203 are repeated; in step S205, when the initial calibration points and the experimental points corresponding to them are accurate, the calibration distance of the initial calibration point closest to the target point is obtained, and it is determined whether the calibration distance is less than the target calibration threshold; in step S206, when the calibration distance is greater than or equal to the target calibration threshold, steps S202-S205 are repeated; in step S207, when the calibration distance is less than the target calibration threshold, the initial calibration point corresponding to the calibration distance is determined to be the target calibration point.

[0107] Optionally, the target calibration threshold is calculated using the formula: ,in, Set the threshold for the target. is the horizontal viewing angle of the camera, is the horizontal distance between the camera and the target calibration point.

[0108] Optionally, a camera imaging model is established, which includes an image plane, a world coordinate system, and an optical center horizontal plane. The image plane includes a pixel coordinate system and an image coordinate system. The origin of the pixel coordinate system is the upper left corner of the image plane, and the origin of the image coordinate system is the center point of the image plane. The image plane is perpendicular to the optical axis of the camera, and the optical axis of the camera passes through the center point. The focal length is calculated using the following formula: , is the width of the camera sensor, is the horizontal viewing angle of the camera, and the focal length is the distance from the camera to the center point of the image plane; the vertical angle between the target calibration point and the horizontal plane of the optical center is calculated using the formula: , is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the height difference between the camera and the target calibration point, is the horizontal distance between the camera and the target calibration point; the distance from the camera to the target calibration point is calculated using the formula: , is the distance from the camera to the target calibration point, is the focal length, is the pixel horizontal coordinate of the target calibration point, The pixel ordinate of the target calibration point.

[0109] Optionally, calculate the angle between the target calibration point and the true north direction, and the calculation formula is: ,in, is the angle between the target calibration point and the true north direction, is the distance between the target calibration point and the camera, is the vertical distance of the target calibration point in the north-south direction of the camera; calculate the distance between the focus of the camera and the target calibration point perpendicular to the horizontal plane of the optical center, and the calculation formula is , is the distance between the camera and the focus of the target calibration point perpendicular to the horizontal plane of the optical center, is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the distance from the camera to the target calibration point; calculate the horizontal angle between the target calibration point and the current camera optical axis, the calculation formula is , is the horizontal angle between the target calibration point and the current camera optical axis, is the horizontal coordinate of the target calibration point on the image plane; calculate the initial horizontal offset, the calculation formula is ,in, is the initial horizontal offset, The horizontal rotation angle of the camera.

[0110] Optionally, calculate the angle between the target calibration point and the optical axis relative to the camera using the formula: , is the angle between the target calibration point and the optical axis relative to the camera, is the pixel ordinate of the target calibration point, is the focal length; calculate the initial vertical offset, the calculation formula is ,in, is the initial vertical offset, is the vertical angle between the target calibration point and the horizontal plane of the optical center, The vertical rotation angle of the camera.

[0111] Optionally, calculate the angle between the target point and the north direction. The calculation formula is: ,in, is the angle between the target point and the north direction, is the distance between the target point and the camera, is the vertical distance of the target point in the north-south direction of the camera; calculate the vertical angle between the target point and the horizontal plane of the optical center, and the calculation formula is ,in, is the vertical angle between the target point and the horizontal plane of the optical center, is the height difference between the camera and the target point, is the horizontal distance between the camera and the target point; the target horizontal offset is calculated using the formula: ,in, is the target horizontal offset, is the initial horizontal offset, is the horizontal rotation angle of the camera; calculate the vertical offset of the target, the calculation formula is , is the target vertical offset, is the vertical rotation angle of the camera, is the initial vertical offset.

[0112] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the method for associating and controlling an Eagle Eye camera with a GIS map when running.

[0113] Specifically, a method for associating and controlling an Eagle Eye camera with a GIS map includes:

[0114] Determine the target calibration point according to the target calibration threshold and the target point; obtain camera information, calibration point information and target point geographic coordinates, wherein the camera information includes the camera initial state, camera geographic coordinates, camera imaging model and sensor size, and the calibration point information includes the calibration point geographic coordinates and the calibration point pixel coordinates; obtain the target horizontal offset and the target vertical offset according to the camera's initial horizontal offset, initial vertical offset and the target point geographic coordinates; adjust the camera according to the target horizontal offset and the target vertical offset.

[0115] Optionally, in step S201, the camera's field of view is divided into at least one calibration image, and initial calibration points are selected on the calibration images, wherein at least one initial calibration point is calibrated on one calibration image; in step S202, experimental points are marked on the GIS map and the real scene according to the geographic information of the initial calibration points, wherein one initial calibration point corresponds to one experimental point; in step S203, the accuracy of the initial calibration points and the experimental points corresponding to them is determined; in step S204, when the initial calibration points and the experimental points corresponding to them are inaccurate, steps S202-S203 are repeated; in step S205, when the initial calibration points and the experimental points corresponding to them are accurate, the calibration distance of the initial calibration point closest to the target point is obtained, and it is determined whether the calibration distance is less than the target calibration threshold; in step S206, when the calibration distance is greater than or equal to the target calibration threshold, steps S202-S205 are repeated; in step S207, when the calibration distance is less than the target calibration threshold, the initial calibration point corresponding to the calibration distance is determined to be the target calibration point.

[0116] Optionally, the target calibration threshold is calculated using the formula: ,in, Set the target threshold. is the horizontal viewing angle of the camera, is the horizontal distance between the camera and the target calibration point.

[0117] Optionally, a camera imaging model is established, which includes an image plane, a world coordinate system, and an optical center horizontal plane. The image plane includes a pixel coordinate system and an image coordinate system. The origin of the pixel coordinate system is the upper left corner of the image plane, and the origin of the image coordinate system is the center point of the image plane. The image plane is perpendicular to the optical axis of the camera, and the optical axis of the camera passes through the center point. The focal length is calculated using the following formula: , is the width of the camera sensor, is the horizontal viewing angle of the camera, and the focal length is the distance from the camera to the center point of the image plane; the vertical angle between the target calibration point and the horizontal plane of the optical center is calculated using the formula: , is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the height difference between the camera and the target calibration point, is the horizontal distance between the camera and the target calibration point; the distance from the camera to the target calibration point is calculated using the formula: , is the distance from the camera to the target calibration point, is the focal length, is the pixel horizontal coordinate of the target calibration point, The pixel ordinate of the target calibration point.

[0118] Optionally, calculate the angle between the target calibration point and the true north direction, and the calculation formula is: ,in, is the angle between the target calibration point and the true north direction, is the distance between the target calibration point and the camera, is the vertical distance of the target calibration point in the north-south direction of the camera; calculate the distance between the focus of the camera and the target calibration point perpendicular to the horizontal plane of the optical center, and the calculation formula is , is the distance between the camera and the focus of the target calibration point perpendicular to the horizontal plane of the optical center, is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the distance from the camera to the target calibration point; calculate the horizontal angle between the target calibration point and the current camera optical axis, the calculation formula is , is the horizontal angle between the target calibration point and the current camera optical axis, is the horizontal coordinate of the target calibration point on the image plane; calculate the initial horizontal offset, the calculation formula is ,in, is the initial horizontal offset, The horizontal rotation angle of the camera.

[0119] Optionally, calculate the angle between the target calibration point and the optical axis relative to the camera using the formula: , is the angle between the target calibration point and the optical axis relative to the camera, is the pixel ordinate of the target calibration point, is the focal length; calculate the initial vertical offset, the calculation formula is ,in, is the initial vertical offset, is the vertical angle between the target calibration point and the horizontal plane of the optical center, The vertical rotation angle of the camera.

[0120] Optionally, calculate the angle between the target point and the north direction. The calculation formula is: ,in, is the angle between the target point and the true north direction, is the distance between the target point and the camera, is the vertical distance of the target point in the north-south direction of the camera; calculate the vertical angle between the target point and the horizontal plane of the optical center, and the calculation formula is ,in, is the vertical angle between the target point and the horizontal plane of the optical center, is the height difference between the camera and the target point, is the horizontal distance between the camera and the target point; the target horizontal offset is calculated using the formula: ,in, is the target horizontal offset, is the initial horizontal offset, is the horizontal rotation angle of the camera; calculate the vertical offset of the target, the calculation formula is , is the target vertical offset, is the vertical rotation angle of the camera, is the initial vertical offset.

[0121] The embodiment of the present invention provides a device, the device includes a processor, a memory and a program stored in the memory and can be run on the processor, and the processor implements at least the following steps when executing the program: determining the target calibration point according to the target calibration threshold and the target point; obtaining camera information, calibration point information and target point geographic coordinates, wherein the camera information includes the camera initial state, camera geographic coordinates, camera imaging model and sensor size, and the calibration point information includes the calibration point geographic coordinates and calibration point pixel coordinates; obtaining the target horizontal offset and the target vertical offset according to the camera's initial horizontal offset, initial vertical offset and target point geographic coordinates; adjusting the camera according to the target horizontal offset and the target vertical offset. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0122] Optionally, in step S201, the camera's field of view is divided into at least one calibration image, and initial calibration points are selected on the calibration images, wherein at least one initial calibration point is calibrated on one calibration image; in step S202, experimental points are marked on the GIS map and the real scene according to the geographic information of the initial calibration points, wherein one initial calibration point corresponds to one experimental point; in step S203, the accuracy of the initial calibration points and the experimental points corresponding to them is determined; in step S204, when the initial calibration points and the experimental points corresponding to them are inaccurate, steps S202-S203 are repeated; in step S205, when the initial calibration points and the experimental points corresponding to them are accurate, the calibration distance of the initial calibration point closest to the target point is obtained, and it is determined whether the calibration distance is less than the target calibration threshold; in step S206, when the calibration distance is greater than or equal to the target calibration threshold, steps S202-S205 are repeated; in step S207, when the calibration distance is less than the target calibration threshold, the initial calibration point corresponding to the calibration distance is determined to be the target calibration point.

[0123] Optionally, the target calibration threshold is calculated using the formula: ,in, Set the target threshold. is the horizontal viewing angle of the camera, is the horizontal distance between the camera and the target calibration point.

[0124] Optionally, a camera imaging model is established, which includes an image plane, a world coordinate system, and an optical center horizontal plane. The image plane includes a pixel coordinate system and an image coordinate system. The origin of the pixel coordinate system is the upper left corner of the image plane, and the origin of the image coordinate system is the center point of the image plane. The image plane is perpendicular to the optical axis of the camera, and the optical axis of the camera passes through the center point. The focal length is calculated using the following formula: , is the width of the camera sensor, is the horizontal viewing angle of the camera, and the focal length is the distance from the camera to the center point of the image plane; the vertical angle between the target calibration point and the horizontal plane of the optical center is calculated using the formula: , is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the height difference between the camera and the target calibration point, is the horizontal distance between the camera and the target calibration point; the distance from the camera to the target calibration point is calculated using the formula: , is the distance from the camera to the target calibration point, is the focal length, is the pixel horizontal coordinate of the target calibration point, The pixel ordinate of the target calibration point.

[0125] Optionally, calculate the angle between the target calibration point and the true north direction, and the calculation formula is: ,in, is the angle between the target calibration point and the true north direction, is the distance between the target calibration point and the camera, is the vertical distance of the target calibration point in the north-south direction of the camera; calculate the distance between the focus of the camera and the target calibration point perpendicular to the horizontal plane of the optical center, and the calculation formula is , is the distance between the camera and the focus of the target calibration point perpendicular to the horizontal plane of the optical center, is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the distance from the camera to the target calibration point; calculate the horizontal angle between the target calibration point and the current camera optical axis, the calculation formula is , is the horizontal angle between the target calibration point and the current camera optical axis, is the horizontal coordinate of the target calibration point on the image plane; calculate the initial horizontal offset, the calculation formula is ,in, is the initial horizontal offset, The horizontal rotation angle of the camera.

[0126] Optionally, calculate the angle between the target calibration point and the optical axis relative to the camera using the formula: , is the angle between the target calibration point and the optical axis relative to the camera, is the pixel ordinate of the target calibration point, is the focal length; calculate the initial vertical offset, the calculation formula is ,in, is the initial vertical offset, is the vertical angle between the target calibration point and the horizontal plane of the optical center, The vertical rotation angle of the camera.

[0127] Optionally, calculate the angle between the target point and the north direction. The calculation formula is: ,in, is the angle between the target point and the true north direction, is the distance between the target point and the camera, is the vertical distance of the target point in the north-south direction of the camera; calculate the vertical angle between the target point and the horizontal plane of the optical center, and the calculation formula is ,in, is the vertical angle between the target point and the horizontal plane of the optical center, is the height difference between the camera and the target point, is the horizontal distance between the camera and the target point; the target horizontal offset is calculated using the formula: ,in, is the target horizontal offset, is the initial horizontal offset, is the horizontal rotation angle of the camera; calculate the vertical offset of the target, the calculation formula is , is the target vertical offset, is the vertical rotation angle of the camera, is the initial vertical offset.

[0128] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes at least the following method steps: determining a target calibration point based on a target calibration threshold and a target point; obtaining camera information, calibration point information, and target point geographic coordinates, wherein the camera information includes the camera initial state, the camera geographic coordinates, the camera imaging model, and the sensor size, and the calibration point information includes the calibration point geographic coordinates and the calibration point pixel coordinates; obtaining a target horizontal offset and a target vertical offset based on the camera's initial horizontal offset, initial vertical offset, and the target point geographic coordinates; and adjusting the camera based on the target horizontal offset and the target vertical offset.

[0129] Optionally, in step S201, the camera's field of view is divided into at least one calibration image, and initial calibration points are selected on the calibration images, wherein at least one initial calibration point is calibrated on one calibration image; in step S202, experimental points are marked on the GIS map and the real scene according to the geographic information of the initial calibration points, wherein one initial calibration point corresponds to one experimental point; in step S203, the accuracy of the initial calibration points and the experimental points corresponding to them is determined; in step S204, when the initial calibration points and the experimental points corresponding to them are inaccurate, steps S202-S203 are repeated; in step S205, when the initial calibration points and the experimental points corresponding to them are accurate, the calibration distance of the initial calibration point closest to the target point is obtained, and it is determined whether the calibration distance is less than the target calibration threshold; in step S206, when the calibration distance is greater than or equal to the target calibration threshold, steps S202-S205 are repeated; in step S207, when the calibration distance is less than the target calibration threshold, the initial calibration point corresponding to the calibration distance is determined to be the target calibration point.

[0130] Optionally, the target calibration threshold is calculated using the formula: ,in, Set the threshold for the target. is the horizontal viewing angle of the camera, is the horizontal distance between the camera and the target calibration point.

[0131] Optionally, a camera imaging model is established, which includes an image plane, a world coordinate system, and an optical center horizontal plane. The image plane includes a pixel coordinate system and an image coordinate system. The origin of the pixel coordinate system is the upper left corner of the image plane, and the origin of the image coordinate system is the center point of the image plane. The image plane is perpendicular to the optical axis of the camera, and the optical axis of the camera passes through the center point. The focal length is calculated using the following formula: , is the width of the camera sensor, is the horizontal viewing angle of the camera, and the focal length is the distance from the camera to the center point of the image plane; the vertical angle between the target calibration point and the horizontal plane of the optical center is calculated using the formula: , is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the height difference between the camera and the target calibration point, is the horizontal distance between the camera and the target calibration point; the distance from the camera to the target calibration point is calculated using the formula: , is the distance from the camera to the target calibration point, is the focal length, is the pixel horizontal coordinate of the target calibration point, The pixel ordinate of the target calibration point.

[0132] Optionally, calculate the angle between the target calibration point and the true north direction, and the calculation formula is: ,in, is the angle between the target calibration point and the true north direction, is the distance between the target calibration point and the camera, is the vertical distance of the target calibration point in the north-south direction of the camera; calculate the distance between the focus of the camera and the target calibration point perpendicular to the horizontal plane of the optical center, and the calculation formula is , is the distance between the camera and the focus of the target calibration point perpendicular to the horizontal plane of the optical center, is the vertical angle between the target calibration point and the horizontal plane of the optical center, is the distance from the camera to the target calibration point; calculate the horizontal angle between the target calibration point and the current camera optical axis, the calculation formula is , is the horizontal angle between the target calibration point and the current camera optical axis, is the horizontal coordinate of the target calibration point on the image plane; calculate the initial horizontal offset, the calculation formula is ,in, is the initial horizontal offset, The horizontal rotation angle of the camera.

[0133] Optionally, calculate the angle between the target calibration point and the optical axis relative to the camera using the formula: , is the angle between the target calibration point and the optical axis relative to the camera, is the pixel ordinate of the target calibration point, is the focal length; calculate the initial vertical offset, the calculation formula is ,in, is the initial vertical offset, is the vertical angle between the target calibration point and the horizontal plane of the optical center, The vertical rotation angle of the camera.

[0134] Optionally, calculate the angle between the target point and the north direction. The calculation formula is: ,in, is the angle between the target point and the true north direction, is the distance between the target point and the camera, is the vertical distance of the target point in the north-south direction of the camera; calculate the vertical angle between the target point and the horizontal plane of the optical center, and the calculation formula is ,in, is the vertical angle between the target point and the horizontal plane of the optical center, is the height difference between the camera and the target point, is the horizontal distance between the camera and the target point; the target horizontal offset is calculated using the formula: ,in, is the target horizontal offset, is the initial horizontal offset, is the horizontal rotation angle of the camera; calculate the vertical offset of the target, the calculation formula is , is the target vertical offset, is the vertical rotation angle of the camera, is the initial vertical offset.

[0135] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0136] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0137] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0138] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0140] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0141] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0142] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0143] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0144] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for associating and controlling an Eagle Eye camera with a GIS map, characterized in that: include: Determine the target calibration point according to the target calibration threshold and the target point; The target calibration point is determined according to the target calibration threshold and the target point, including: step S201, dividing the field of view of the camera into at least one calibration image, selecting initial calibration points on the calibration image, wherein at least one initial calibration point is calibrated on one calibration image; step S202, marking experimental points on the GIS map and the real scene according to the geographic information of the initial calibration point, wherein one initial calibration point corresponds to one experimental point; step S203, judging the accuracy of the initial calibration point and the corresponding experimental point; step S204, when the initial calibration point is consistent with the experimental point, the experimental point is marked with the initial calibration point. When the experimental point corresponding to the initial calibration point is inaccurate, repeat steps S202-S203; Step S205, when the initial calibration point and the experimental point corresponding to the initial calibration point are accurate, obtain the calibration distance of the initial calibration point closest to the target point, and determine whether the calibration distance is less than the target calibration threshold; Step S206, when the calibration distance is greater than or equal to the target calibration threshold, repeat steps S202-S205; Step S207, when the calibration distance is less than the target calibration threshold, determine that the initial calibration point corresponding to the calibration distance is the target calibration point; Determining the target calibration point according to the target calibration threshold and the target point includes: calculating the target calibration threshold, and the calculation formula is: Wherein, λ is the target calibration threshold, θ is the horizontal viewing angle of the camera, and L is the horizontal distance between the camera and the target calibration point; Obtaining camera information, calibration point information, and target point geographic coordinates, wherein the camera information includes the camera initial state, camera geographic coordinates, camera imaging model, and sensor size, and the calibration point information includes the calibration point geographic coordinates and calibration point pixel coordinates; Obtaining a target horizontal offset and a target vertical offset according to an initial horizontal offset and an initial vertical offset of the camera and the geographic coordinates of the target point; The camera is adjusted according to the target horizontal offset and the target vertical offset.

2. The method according to claim 1, characterized in that Get camera information, calibration point information, and target point geographic coordinates, including: Establishing the camera imaging model, the camera imaging model includes an image plane, a world coordinate system and an optical center horizontal plane, the image plane includes a pixel coordinate system and an image coordinate system, the origin of the pixel coordinate system is the upper left corner of the image plane, the origin of the image coordinate system is the center point of the image plane, the image plane is perpendicular to the optical axis of the camera, and the optical axis of the camera passes through the center point; Calculate the focal length using the formula: W is the width of the sensor of the camera, θ is the horizontal viewing angle of the camera, and the focal length is the distance from the camera to the center point of the image plane; Calculate the vertical angle between the target calibration point and the horizontal plane of the optical center, and the calculation formula is: globalY is the vertical angle between the target calibration point and the horizontal plane of the optical center, △H is the height difference between the camera and the target calibration point, and L is the horizontal distance between the camera and the target calibration point; Calculate the distance from the camera to the target calibration point, the calculation formula is q 2 =R 2 +y 2 +x 2 , q is the distance from the camera to the target calibration point, R is the focal length, x is the pixel horizontal coordinate of the target calibration point, and y is the pixel vertical coordinate of the target calibration point.

3. The method according to claim 2, characterized in that Obtaining a target horizontal offset and a target vertical offset according to the initial horizontal offset and the initial vertical offset of the camera and the geographic coordinates of the target point, including: Calculate the angle between the target calibration point and the true north direction using the calculation formula globalX=arccos(n / m), where globalX is the angle between the target calibration point and the true north direction, m is the distance between the target calibration point and the camera, and n is the vertical distance between the target calibration point and the camera in the north-south direction. Calculate the distance between the camera and the focus of the target calibration point perpendicular to the optical center horizontal plane, the calculation formula is k=cos(globalY)*q, k is the distance between the camera and the focus of the target calibration point perpendicular to the optical center horizontal plane, globalY is the vertical angle between the target calibration point and the optical center horizontal plane, q is the distance from the camera to the target calibration point; Calculate the horizontal angle between the target calibration point and the current camera optical axis. The calculation formula is: gx is the horizontal angle between the target calibration point and the current camera optical axis, and x is the horizontal coordinate of the target calibration point on the image plane; The initial horizontal offset is calculated using the formula pan=globalX-gx-dp, where pan is the initial horizontal offset and dp is the horizontal rotation angle of the camera.

4. The method according to claim 2, characterized in that: Obtaining a target horizontal offset and a target vertical offset according to the initial horizontal offset and the initial vertical offset of the camera and the geographic coordinates of the target point, including: Calculate the angle between the target calibration point and the optical axis relative to the camera. The calculation formula is: gy is the angle between the target calibration point and the optical axis relative to the camera, y is the pixel ordinate of the target calibration point, and R is the focal length; The initial vertical offset is calculated using the formula til=globalY-gy-dt, where til is the initial vertical offset, globalY is the vertical angle between the target calibration point and the horizontal plane of the optical center, and dt is the vertical rotation angle of the camera.

5. The method according to claim 2, characterized in that: Obtaining a target horizontal offset and a target vertical offset according to the initial horizontal offset and the initial vertical offset of the camera and the geographic coordinates of the target point, including: Calculate the angle between the target point and the north direction using the formula globalX'=arccos(n' / m'), where globalX' is the angle between the target point and the north direction, m' is the distance between the target point and the camera, and n' is the vertical distance between the target point and the north and south directions of the camera. Calculate the vertical angle between the target point and the horizontal plane of the optical center, and the calculation formula is: Wherein, globalY' is the vertical angle between the target point and the horizontal plane of the optical center, △H' is the height difference between the camera and the target point, and L' is the horizontal distance between the camera and the target point; Calculate the target horizontal offset using the formula gx'=globalX'-pan-dp, where gx' is the target horizontal offset, pan is the initial horizontal offset, and dp is the horizontal rotation angle of the camera; Calculate the target vertical offset, the calculation formula is gy'=globalY'-dt-til, gy' is the target vertical offset, dt is the vertical rotation angle of the camera, and til is the initial vertical offset.

6. A device for associating and controlling an Eagle Eye camera with a GIS map, characterized in that: include: A determination unit, used to determine a target calibration point according to a target calibration threshold and a target point; The determination unit includes: a determination subunit, which is used to execute step S201, divide the field of view of the camera into at least one calibration image, select initial calibration points on the calibration image, wherein at least one initial calibration point is calibrated on one calibration image; step S202, mark the experimental points on the GIS map and the real scene according to the geographic information of the initial calibration points, wherein one initial calibration point corresponds to one experimental point; step S203, determine the accuracy of the initial calibration point and the corresponding experimental point; step S204, when the initial calibration point When the initial calibration point and the experimental point corresponding to it are inaccurate, repeat steps S202-S203; Step S205, when the initial calibration point and the experimental point corresponding to it are accurate, obtain the calibration distance of the initial calibration point closest to the target point, and determine whether the calibration distance is less than the target calibration threshold; Step S206, when the calibration distance is greater than or equal to the target calibration threshold, repeat steps S202-S205; Step S207, when the calibration distance is less than the target calibration threshold, determine that the initial calibration point corresponding to the calibration distance is the target calibration point; The determination unit includes: a first calculation subunit, which is used to calculate the target calibration threshold, and the calculation formula is: Among them, λ is the target calibration threshold, θ is the horizontal viewing angle of the camera, and L is the horizontal distance between the camera and the target calibration point; A first acquisition unit is used to acquire camera information, calibration point information and target point geographic coordinates, wherein the camera information includes the camera initial state, camera geographic coordinates, camera imaging model and sensor size, and the calibration point information includes the calibration point geographic coordinates and calibration point pixel coordinates; A second acquisition unit is used to acquire a target horizontal offset and a target vertical offset according to an initial horizontal offset and an initial vertical offset of the camera and the geographic coordinates of the target point; An adjustment unit is used to adjust the camera according to the target horizontal offset and the target vertical offset.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute a method for controlling the association between an Eagle Eye camera and a GIS map as described in any one of claims 1 to 5.

8. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the association control of an Eagle Eye camera and a GIS map as described in any one of claims 1 to 5.

Images