Video and map data mapping method based on three-dimensional geographic information

Abstract

The invention relates to a video and map data mapping method based on three-dimensional geographic information, and the method comprises the steps: (1) obtaining the three-dimensional geographic coordinates of a starting point and a target point, respectively including the longitude, latitude and elevation data of each point, picking up the video pixel coordinates and the three-dimensional geographic coordinates of the target point, and constructing a serialized feature point pair; (2) according to the serialized feature point pairs, camera geographic calibration based on the feature point pairs is carried out; and step (3), mapping the three-dimensional geographic coordinates of all points in the geographic space in the video coverage area into video pixel coordinates based on a rotation matrix constructed after camera geographic calibration, or mapping the video pixel coordinates of all points in a camera picture into three-dimensional geographic coordinates.

Description

A mapping method for video and map data based on 3D geographic information Technical Field

[0001] This invention relates to the field of video surveillance and geographic information fusion technology, and in particular to a method for mapping video and map data based on three-dimensional geographic information. Background Technology

[0002] The technology based on the fusion of surveillance video and geographic information has been applied in smart cities and safe cities.

[0003] However, the inventors of this application discovered in their research that the prior art has limitations that severely restrict the universality and reliability of the technology's application.

[0004] First, existing technical solutions are based on the strong assumption of a "flat surface". Their core mapping models (such as homography transformation) do not take into account the depth changes caused by terrain undulations, causing them to fail completely in complex terrain areas with significant surface undulations, such as hills, mountains, and canyons.

[0005] Secondly, existing mapping methods rely on overly simplified mapping algorithms or mathematical models, resulting in coarse processing of camera geocalibration and coordinate system transformation. This leads to low accuracy in the generated mapping results, making them unsuitable for high-precision positioning, measurement, and analysis tasks, such as precise location positioning in emergency rescue and accurate target tracking in the military field.

[0006] In other words, existing technologies suffer from two major problems: poor terrain adaptability and insufficient algorithm accuracy. Summary of the Invention

[0007] To address the aforementioned problems, the purpose of this invention is to provide a mapping method for video and map data based on three-dimensional geographic information. In the bidirectional mapping model of video coordinates and map coordinates, elevation data from three-dimensional geographic information is introduced for analysis, and the algorithm is improved to enhance the terrain adaptability and accuracy of the mapping method.

[0008] To achieve the above objectives, the present invention adopts the following technical solutions: In the first aspect, this application provides a method for mapping video and map data based on three-dimensional geographic information. The method includes: step (1), obtaining the three-dimensional geographic coordinates of the starting point and the target point, including the longitude, latitude and elevation data of each point, picking the video pixel coordinates and three-dimensional geographic coordinates of the target point, and constructing a serialized feature point pair; step (2), performing camera geocalibration based on the serialized feature point pair; step (3), based on the rotation matrix constructed after camera geocalibration, mapping the three-dimensional geographic coordinates of all points in the geographic space within the video coverage area to video pixel coordinates, or mapping the video pixel coordinates of all points in the camera frame to three-dimensional geographic coordinates.

[0009] In one implementation, step (2) includes: observing the target point based on the starting point of the camera to obtain the observed horizontal azimuth and observed vertical azimuth between the two points; solving the actual horizontal azimuth and actual vertical azimuth between the two points based on the three-dimensional geographic information of the starting point and the target point; and constructing a rotation matrix based on the aforementioned observation and solution results.

[0010] In one implementation, the actual horizontal azimuth of the target point ( The following formula can be used to solve the problem:

[0011] in: These are the longitude and latitude of the starting point, respectively. These are the longitude and latitude of the target point, respectively. .

[0012] In one implementation, the actual vertical azimuth of the target point ( The following formula can be used to solve the problem:

[0013] in, , where is the elevation difference between the starting point and the target point;

[0014] in For the Earth's radius, The central angle is calculated from the latitude and longitude of the two points.

[0015] In one implementation, in step (3), the three-dimensional geographic coordinates of all points in the geographic space within the video coverage area are mapped to video pixel coordinates, including: calculating intermediate projection parameters; and calculating the horizontal and vertical pixel coordinates of the target point based on the intermediate projection parameters.

[0016] In one implementation, the intermediate projection parameter K is calculated using the following formula:

[0017]

[0018] in, The actual vertical azimuth angle calculated for the starting point and the target point. The calculated actual horizontal azimuth angle. The pitch angle after rotation matrix correction. Corrected horizontal azimuth.

[0019] In one implementation, the horizontal position of the target point on the image is determined by the following formula:

[0020]

[0021] in, For the horizontal resolution of the image, This is the equivalent focal length of the camera. The width of the target surface represents the image sensor; the vertical position of the target point on the image is determined by the following formula:

[0022]

[0023] in, For image vertical resolution, This represents the target surface height of the image sensor.

[0024] In one implementation, the mapping process of map coordinates includes: (1) Line of sight construction: Constructing the camera line of sight equation in a distance-elevation two-dimensional coordinate system:

[0025]

[0026] in, Elevation of the camera position. Horizontal distance (1) Vertical pitch angle; (2) Terrain profile construction along horizontal azimuth angle Construct terrain profiles by direction and set sampling step size. For each sampling distance :

[0027] in Calculations were performed based on the WGS84 ellipsoid model; according to coordinates Elevation values ​​obtained by interpolation from DEM Construct a set of terrain profile points:

[0028] (3) Analyze the line of sight and calculate the coordinates to find the intersection of the line of sight and the terrain profile:

[0029] Find the nearest intersection point between the cameras:

[0030] Calculate the geographic coordinates of the target point: .

[0032] In one implementation, the elevation data is acquired through a digital elevation model (DEM) or based on lidar or multi-view cameras.

[0033] Secondly, a computer-readable storage medium is provided, which stores a computer program that is executed by a processor to implement the method of the first aspect.

[0034] The present invention has the following advantages due to the adoption of the above technical solutions: Compared with the prior art, the beneficial effects of this application are: it can solve the problem of mutual mapping between video and map in complex terrain, and realize the accurate mapping between video pixels and geographic coordinates (including elevation) based on a high-precision mapping algorithm. It can be widely used in the accurate location of fire points in forest fire prevention in mountainous areas, the visual command and dispatch of drones in video, and the real-time overlay of geographic information and attribute data, which significantly improves the command efficiency and situational awareness of emergency rescue. Attached Figure Description

[0035] Figure 1 is a flowchart illustrating a method for mapping video and map data based on 3D geographic information according to an embodiment; Figure 2 is a schematic diagram illustrating camera geocalibration based on feature points according to an embodiment; Figure 3 is a schematic diagram illustrating the positional relationship between line of sight and DEM profile line according to an embodiment. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0037] Based on Figure 1, this application provides a method for mapping video and map data based on three-dimensional geographic information. The method includes: step (1) obtaining the three-dimensional geographic coordinates of the starting point and the target point, including the longitude, latitude and elevation data of each point, picking the video pixel coordinates and three-dimensional geographic coordinates of the target point, and constructing a serialized feature point pair; step (2) performing camera geocalibration based on the serialized feature point pair; step (3) mapping the three-dimensional geographic coordinates of all points in the geographic space within the video coverage area to video pixel coordinates, or mapping the video pixel coordinates of all points in the camera frame to three-dimensional geographic coordinates, based on the rotation matrix constructed after camera geocalibration.

[0038] Based on Figures 2 and 3, the method of this application is described in a more detailed embodiment.

[0039] In a detailed embodiment of this application, the steps of the method include: 1. Camera geocalibration based on feature point pairs. Figure 2 is a schematic diagram of camera geocalibration.

[0040] During actual camera installation, attitude errors are inevitable: firstly, there may be pitch angle deviations in the vertical direction; secondly, the zero azimuth reference in the horizontal direction may not be aligned with true north. Therefore, one of the core objectives of video image registration is to accurately measure and compensate for angular offsets in these two directions.

[0041] Traditional video image registration methods are typically designed for bullet cameras with a fixed field of view, and can only perform registration in a single direction. This technology, however, supports PTZ cameras, and by acquiring registration data from multiple directions, it constructs a complete high-dimensional rotation matrix, thereby achieving accurate correction across the entire field of view.

[0042] The actual horizontal azimuth of the target point ( The following formula can be used to solve the problem:

[0043] in: These are the longitude and latitude (in radians) of the starting point (usually the camera position).

[0044] These are the longitude and latitude (in radians) of the target point, respectively.

[0045] .

[0046] The ground distance between the starting point and the target point ( The solution can be obtained using Vincenty's formula or the great circle distance formula.

[0047] in For the Earth's radius, The central angle is calculated from the latitude and longitude of two points (using methods such as Vincent's formula or Havesing's formula).

[0048] When calculating the pitch angle, this technology integrates digital elevation model (DEM) data in real time, by incorporating elevation information. Transform the target point from two-dimensional plane coordinates Upgraded to three-dimensional spatial coordinates This allows for the accurate calculation of its actual vertical azimuth (elevation angle). ):

[0049] in,

[0050] Finally, the offset between the horizontal / vertical azimuth angle observed by the camera and the actual azimuth angle calculated by the above formula is determined. Construct rotation matrix This enables high-precision conversion from the camera coordinate system to the real-world coordinate system.

[0051]

[0052] in and These represent rotational transformations around the X-axis (pitch) and Y-axis (azimuth), respectively.

[0053] 2. Mapping from Geographic Coordinates to Video Pixel Coordinates: The mapping from geographic coordinates to video pixel coordinates is a complex process involving geometric correction and optical projection. Its core steps are as follows: First, the high-dimensional rotation matrix obtained during the camera geocalibration process is used... The actual horizontal azimuth and vertical pitch angles of the current target point are rotated to correct for camera installation errors. This step transforms the direction vector in the geodetic coordinate system to the camera coordinate system based on the camera's optical axis.

[0054] Subsequently, based on the pinhole camera model and the focal length method, the calibrated camera coordinate system direction vector is projected onto the two-dimensional imaging plane. The calculation process is as follows: Calculating projection parameters: A key intermediate projection parameter K is introduced, which combines the pitch angle and the azimuth angle after rotation correction. Its calculation formula is:

[0055]

[0056] in, The actual pitch angle calculated from the starting point and the target point. The calculated actual horizontal azimuth angle. The pitch angle after rotation matrix correction. Corrected horizontal azimuth.

[0057] Calculating horizontal pixel coordinates: The horizontal position (X coordinate) of the target point on the image is determined by the following formula:

[0058]

[0059] in, The horizontal resolution (pixel width) of the image. The equivalent focal length of the camera (unit: mm). This represents the target width of the image sensor.

[0060] Calculate the vertical pixel coordinates: The vertical position (Y coordinate) of the target point on the image is determined by the following formula:

[0061]

[0062] in, The vertical resolution (pixel height) of the image. This represents the target surface height of the image sensor. The subtraction in the formula is because the origin of the image coordinate system is usually located at the upper left corner, and the Y-axis is positive downwards, so it is necessary to transform the origin (optical center) of the optical coordinate system to the image center.

[0063] Ultimately, through the above process, a high-precision mapping from geodetic coordinates to video pixel coordinates was achieved.

[0064] 3. Video pixel coordinate to geographic coordinate mapping is based on the principle of 3D line-of-sight analysis. Through a digital elevation model (DEM) and profile analysis algorithms, a high-precision mapping from video pixel coordinates to geographic coordinates is achieved. This process constructs the geometric relationship between the camera's line of sight and the terrain profile, accurately calculating the first intersection point between the line of sight and the ground surface, thereby determining the geographic coordinates of the target point. Figure 3 illustrates the positional relationship between the line of sight and the DEM profile line.

[0065] Line of sight construction: Constructing the camera line of sight equation in a distance-elevation 2D coordinate system:

[0066]

[0067] in, Elevation of the camera position. Horizontal distance It is the vertical pitch angle.

[0068] Terrain profile construction: along horizontal azimuth Construct terrain profiles based on direction.

[0069] Set sampling step size For each sampling distance :

[0070] in Calculations were performed based on the WGS84 ellipsoid model.

[0071] According to coordinates Elevation values ​​obtained by interpolation from DEM Construct a set of terrain profile points:

[0072] Viewpoint analysis and coordinate calculation: Determining the intersection of the line of sight and the terrain profile:

[0073] Find the nearest intersection point between the cameras:

[0074] Calculate the geographic coordinates of the target point:

[0075] The elevation of the target point was obtained from the DEM data:

[0076] In one aspect of the embodiments of this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a computer device, implements the method described in the embodiments of this application.

[0077] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided by this invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of the units described above is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other can be through some interfaces, indirect coupling or communication connection between devices or units, and can be electrical, mechanical, or other forms.

[0078] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0079] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for mapping video and map data based on three-dimensional geographic information, characterized in that, The method includes: step (1), obtaining the three-dimensional geographic coordinates of the starting point and the target point, including the longitude, latitude and elevation data of each point, picking the video pixel coordinates and three-dimensional geographic coordinates of the target point, and constructing a serialized feature point pair; step (2), performing camera geocalibration based on the serialized feature point pair to obtain a rotation matrix; step (3), based on the rotation matrix constructed after camera geocalibration, mapping the three-dimensional geographic coordinates of all points in the geographic space within the video coverage area to video pixel coordinates, or mapping the video pixel coordinates of all points in the camera frame to three-dimensional geographic coordinates.

2. The method for mapping video and map data based on three-dimensional geographic information according to claim 1, characterized in that, in, Step (2) includes: observing the target point based on the starting point of the camera to obtain the observed horizontal azimuth and observed vertical azimuth between the two points; solving the actual horizontal azimuth and actual vertical azimuth between the two points based on the three-dimensional geographic coordinates of the starting point and the target point; and constructing a rotation matrix based on the aforementioned observation and solution results to realize the conversion between the camera coordinate system and the three-dimensional geographic coordinate system.

3. The method for mapping video and map data based on three-dimensional geographic information according to claim 2, characterized in that, The actual horizontal azimuth of the target point ( The following formula can be used to solve the problem: in: These are the longitude and latitude of the starting point, respectively. These are the longitude and latitude of the target point, respectively. 。 4. The method for mapping video and map data based on three-dimensional geographic information according to claim 3, characterized in that, The actual vertical azimuth of the target point ( The following formula can be used to solve the problem: in, , where is the elevation difference between the starting point and the target point; in For the Earth's radius, The central angle is calculated from the latitude and longitude of the two points.

5. The method for mapping video and map data based on three-dimensional geographic information according to claim 4, characterized in that, In step (3), the three-dimensional geographic coordinates of all points in the geographic space within the video coverage area are mapped to video pixel coordinates, including: calculating intermediate projection parameters; and calculating the horizontal and vertical pixel coordinates of the target point based on the intermediate projection parameters.

6. The method for mapping video and map data based on three-dimensional geographic information according to claim 5, characterized in that, The intermediate projection parameter K is calculated using the following formula: in, The actual vertical azimuth angle calculated for the starting point and the target point. The calculated actual horizontal azimuth angle. The pitch angle after rotation matrix correction. Corrected horizontal azimuth.

7. The method for mapping video and map data based on three-dimensional geographic information according to claim 6, characterized in that, The horizontal position of the target point on the image is determined by the following formula: in, For the horizontal resolution of the image, This is the equivalent focal length of the camera. The width of the target surface represents the image sensor; the vertical position of the target point on the image is determined by the following formula: in, For image vertical resolution, This represents the target surface height of the image sensor.

8. The method for mapping video and map data based on three-dimensional geographic information according to claim 7, characterized in that, The mapping process of three-dimensional geographic coordinates includes: (1) Line of sight construction: Constructing the camera line of sight equation in the distance-elevation two-dimensional coordinate system: in, Elevation of the camera position. Horizontal distance (1) Vertical pitch angle; (2) Terrain profile construction along horizontal azimuth angle Construct terrain profiles by direction and set sampling step size. For each sampling distance ： in Calculations were performed based on the WGS84 ellipsoid model; according to coordinates Elevation values ​​obtained by interpolation from DEM Construct a set of terrain profile points: (3) Analyze the line of sight and calculate the coordinates to find the intersection of the line of sight and the terrain profile: Find the nearest intersection point between the cameras: Calculate the geographic coordinates of the target point: 。 9. The method for mapping video and map data based on three-dimensional geographic information according to claim 1, characterized in that, The elevation data is obtained through a digital elevation model (DEM) or based on lidar or multi-view cameras.

10. A computer-readable storage medium, characterized in that, It contains a computer program, which is executed by a processor to implement the methods of claims 1 to 9.

Images