Method for matching real-time video of unmanned aerial vehicle with poi information

Abstract

The invention discloses an unmanned aerial vehicle real-time video poi information matching method, which comprises the steps of 1, acquiring unmanned aerial vehicle information in real time, including a current video picture and coordinate information; 2, comparing the current unmanned aerial vehicle information with the unmanned aerial vehicle information obtained last time, and judging whether the current unmanned aerial vehicle information changes or not; if the poi information is changed, calculating four angular coordinates of the current video picture, otherwise, writing the poi information recorded last time into the current video picture, and turning to the step 4; 3, searching whether corresponding new poi information exists in the four angular coordinate ranges or not; if yes, writing new poi information into the current video picture and recording the poi information to a local cache, otherwise, directly turning to the step 4; and step 4, pushing the current video picture to a streaming media server. According to the method, the problems that the poi information of the corresponding map cannot be visually seen in the inspection process and the area in the current video picture cannot be determined due to shielding or other reasons in some activities can be solved.

Description

Technical Field

[0001] The invention belongs to the technical field of IT and software development, and particularly relates to a method for matching POI information with real-time video of a drone. Background Art

[0002] Drone video live broadcast is different from traditional network live broadcast. As a mobile live broadcast method, drone live broadcast is not only highly flexible, but also has a wider viewing angle than traditional video acquisition methods. In recent years, my country's drone industry has seen explosive growth, and drones have been widely used in various fields such as aerial photography and aerial surveying, agricultural plant protection, power inspections, commercial performances, and disaster relief. With the rise of network live broadcast in my country and the continuous enhancement of enterprises' awareness and ability to go to the cloud, drone video live broadcast has extremely broad application prospects.

[0003] However, at present, due to the high requirements of drone live broadcasting on network and data collection technology, it has not yet been widely popularized in China. At the same time, drone video live broadcasting has the defect that it can only play the original aerial video images, and cannot superimpose POI data information on the video images in real time with high precision, such as: buildings, urban roads, rescue landmarks, interest collections and other geographic information in the video images, which greatly reduces the value of drone video live broadcasting; in addition, sending drone aerial video and flight attitude data to the ground for video secondary processing, although it can realize superimposing data information on the ground video images, the accuracy of the real data superimposed data information is greatly reduced due to the transmission delay.

[0004] In scenarios such as large-scale events and law enforcement inspections, real-time monitoring via drones cannot intuitively show information such as the buildings and streets in the picture, resulting in delayed command and low efficiency, and unable to fully leverage the advantages of drones in scenarios such as law enforcement and security. Summary of the invention

[0005] The purpose of the present invention is to provide a method for matching POI information of drone real-time video in view of the shortcomings of the prior art. The present invention can solve the problem that the corresponding map POI information cannot be directly seen during the inspection process and the area to which the current video screen belongs cannot be determined due to occlusion or other reasons in certain activities.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for matching POI information of real-time video of a drone comprises the following steps:

[0008] Step 1: Obtain drone information in real time, including current video images and coordinate information;

[0009] Step 2: compare the current drone information with the drone information obtained last time to determine whether it has changed; if it has changed, calculate the four corner coordinates of the current video screen, otherwise write the last recorded POI information into the current video screen and go to step 4;

[0010] Step 3: Check whether there is corresponding new POI information within the four corner coordinate ranges; if so, write the new POI information into the current video screen and record the current POI information into the local cache, otherwise go directly to step 4;

[0011] Step 4: Push the current video image to the streaming media server.

[0012] The present invention further illustrates that the step one is specifically: obtaining the drone live video stream through the live broadcast address configured by the background system, including the current video screen and the drone's latitude and longitude, altitude, pitch angle, yaw angle, roll angle, focal length, camera resolution and other information.

[0013] The present invention further illustrates that the step two is specifically: obtaining the drone information recorded last time in the local cache, and then determining whether the latitude, longitude, altitude, and angle information in this drone information and the last drone information have changed; if changed, calculating the four corner coordinates of the current video screen, otherwise writing the poi information recorded last time into the current video screen, and going to step four.

[0014] The present invention further illustrates that in step 2, the calculation of the four corner coordinates of the current video screen includes the following steps:

[0015] (1) Calculate the geographic coordinates of each pixel;

[0016] 11) Convert the pixel coordinates to normalized coordinates in the camera coordinate system: the camera's intrinsic parameter matrix is ​​K, the pixel coordinates are (u, v), then the normalized coordinates are:

[0017]

[0018] Among them, u represents the horizontal position of the pixel, v represents the vertical position of the pixel, l represents the constant term in the homogeneous coordinates; K: represents the intrinsic parameter matrix of the camera, including information such as focal length and principal point position; k 11 : focal length in the x-axis direction; k 12 : skew factor between pixels; k 13 : The coordinate of the center point of the image on the x-axis; k 21 : indicates that there is no offset perpendicular to the x-axis; k 22 : focal length in the y-axis direction; k 23 : The coordinate of the center point of the image on the y-axis; k 31: indicates that there is no offset perpendicular to the x-axis; k 32 : indicates that there is no offset perpendicular to the y-axis; k 33 : Normalization factor, ensuring that the value of the third row and third column is 1; the inverse matrix of K is used to convert pixel coordinates to normalized coordinates in the camera coordinate system; x, y, z represent the coordinate values ​​of the normalized coordinates in the camera coordinate system; through the above formula, the pixel coordinates (u, v) in the image are converted to the normalized coordinates (x, y, z) in the camera coordinate system;

[0019] 12) Convert the normalized coordinates to the coordinates in the camera coordinate system: The external parameters of the camera are the rotation matrix R and the translation vector T, the normalized coordinates are (x, y, z), and the coordinates in the camera coordinate system are:

[0020] [X c ,Y c ,Z c ] = R · [x, y, z] + T;

[0021] Among them, R represents the rotation matrix of the camera, which is used to describe the rotation relationship between the camera coordinate system and the world coordinate system. T represents the translation vector of the camera, which represents the translation of the origin of the camera coordinate system relative to the origin of the world coordinate system. Through the above formula, the normalized coordinates (x, y, z) in the camera coordinate system are converted to the coordinates (Xc, Yc, Zc) in the camera coordinate system, that is, the normalized coordinates are converted to the actual coordinates in the camera coordinate system.

[0022] 13) Convert the coordinates in the camera coordinate system to the coordinates in the world coordinate system: the camera's posture is pitch angle, yaw angle and roll angle, and the coordinates in the camera coordinate system are (X c ,Y c ,Z c ), the coordinates in the world coordinate system are:

[0023] [X w ,Y w ,Z w ]=R pitch ·R yaw ·R roll ·[X c ,Y c ,Z c ]+C;

[0024] Among them, R pitch , R yaw , R roll : The rotation matrix that describes the camera's posture (pitch angle, yaw angle, roll angle) is used to convert the coordinates in the camera coordinate system to the coordinates in the world coordinate system; C represents the position of the camera, that is, the position of the origin of the camera coordinate system in the world coordinate system. (Xc )、(Y c )、(Z c ) represent the position coordinates of the point in the camera coordinate system; (X c ) represents the x-axis position of the point in the camera coordinate system, which is composed of (x n ) multiplied by the first row of the rotation matrix plus the translation vector (T x ) component composition; (Y c ) represents the y-axis position of the point in the camera coordinate system, which is composed of (y n ) multiplied by the elements of the second row of the rotation matrix plus the translation vector (T y ) component composition; (Z c ) represents the z-axis position of the point in the camera coordinate system, which is composed of (z n ) multiplied by the third row of the rotation matrix plus the translation vector (T z ) components. By using the above formula, the coordinates (X c ,Y c ,Z c ) is converted to the coordinates in the world coordinate system (X w ,Y w ,Z w ), that is, converting the coordinates in the camera coordinate system into the actual coordinates in the world coordinate system for calculating the longitude and latitude coordinates of the four corners of the image;

[0025] (2) According to the coordinates in the world coordinate system (X w ,Y w ,Z w )Calculate geographic coordinates:

[0026] 21) Define the ellipsoid parameters:

[0027] a = 6378137.0 m (the semi-major axis of WGS84),

[0028] f = 1 / 298.257223563 (the flattening of WGS84);

[0029] 22) Calculate auxiliary variables:

[0030] The first eccentricity square Where b is the semi-minor axis of the ellipsoid, calculated from a and f;

[0031] Auxiliary variables

[0032] 23) Calculate latitude lat and altitude h:

[0033] Initial value lat1 = atan((Z w·a) / (p·b));

[0034] Iterate until convergence:

[0035] Radius of curvature N = a / sqrt(1-e 2 ·sin 2 (lat)),

[0036] Height h = p / cos(lat)-N,

[0037] Update lat = atan ((Z w +e' 2 ·N·sin(lat)) / p), where e' 2 =e 2 / (1-e 2 );

[0038] 24) Calculate longitude lon:

[0039] lon=[arctan(Y w / X w )·180 / π,lat·180 / π,h];

[0040] The above formula is used to calculate the geographic coordinates of the four pixel points (A lat ,A lon )、(B lat ,B lon )、(C lat ,C lon )、(D lat ,D lon ), where sqrt represents the square root function, atan represents the inverse tangent function, arctan is the extended inverse tangent function that can correctly handle quadrant problems, sin and cos are sine and cosine functions respectively;

[0041] (3) Determine the pixel coordinates corresponding to the upper left corner, upper right corner, lower left corner, and lower right corner of the image based on the size and resolution of the image;

[0042] 31) Get known geographic coordinates and pixel coordinates:

[0043] The geographic coordinates of the four pixel points are (A lat ,A lon )、(B lat ,B lon )、(C lat ,C lon )、(D lat ,D lon ), the corresponding pixel coordinates are (A u ,A v )、(Bu ,B v )、(C u ,C v )、(D u ,D v ); the pixel coordinates of the target pixel are (P u ,P v );

[0044] 32) Calculate the geographic coordinates of the target pixel:

[0045] For the pixel coordinates of the target pixel point (P u ,P v ), and use bilinear interpolation to calculate its geographic coordinates (P lat ,P lon ):

[0046] P lat =(1-w)·((1-h)·A lat +h·C lat )+w·((1-h)·B lat +h·D lat );

[0047] The above formula represents the latitude coordinate P of the target pixel point lat It is calculated by bilinear interpolation of the geographic latitude coordinates of four adjacent pixel points;

[0048] P lon =(1-w)·((1-h)·A lon +h·C lon )+w·((1-h)·B lon +h·D lon );

[0049] The above formula represents the longitude coordinate P of the target pixel point lon It is calculated by bilinear interpolation of the geographic longitude coordinates of four adjacent pixel points;

[0050] Among them, w and h represent the interpolation coefficients of the target pixel in the horizontal and vertical directions respectively; the geographic coordinates of the four corners of the image are calculated using the bilinear interpolation method, and finally the longitude and latitude coordinates of the four corners of the picture are obtained for POI information matching.

[0051] That is, there are four known pixel coordinates and the corresponding actual coordinates in the world coordinate system. The goal is to find a new pixel point (P u ,P v )'s geographic coordinates (P lat ,P lon ). The interpolation method here takes into account the relative position of the pixel in the image and uses it to weight the average of the geographic coordinates of the four surrounding points.

[0052] Calculation process:

[0053] There are four pixels (A u ,A v )、(B u ,B v )、(C u ,C v )、(D u ,D v ), their geographical coordinates are (A lat ,A lon )、(B lat ,B lon )、(C lat ,C lon )、(D lat ,D lon ). Target pixel (P u ,P v ) is located in the rectangular area enclosed by these four points.

[0054] 1. Determine weights (interpolation coefficients) (w) and (h)

[0055] It is necessary to determine the relative position of the target pixel relative to (A) and (B) in the horizontal direction, and relative position relative to (A) and (C) in the vertical direction. This can be calculated as follows:

[0056] Horizontal weight (w): (w = (uA u ) / (B u -A u ))

[0057] Vertical weight (h): (h = (vA v ) / (C v -A v ))

[0058] 2. Bilinear interpolation

[0059] With (w) and (h), the geographic coordinates of the target point can be calculated according to the bilinear interpolation formula:

[0060] Latitude P lat Calculation:

[0061] P lat =(1-w)·((1-h)·A lat +h·C lat )+w·((1-h)·B lat +h·D lat );

[0062] Longitude P lonCalculation:

[0063] P lon =(1-w)·((1-h)·A lon +h·Con)+w·((1-h)·B lon +h·D lon ).

[0064] The present invention further illustrates that in step three, the new POI information includes map POI information and custom POI information.

[0065] Advantages of the present invention:

[0066] The present invention obtains information such as drone video stream, longitude and latitude, altitude, pitch angle, yaw angle, roll angle, focal length, camera resolution, etc., calculates the actual position coordinates corresponding to the picture pixel points, matches the map POI information and the customized POI information through the known coordinates, writes the matched information into the video picture, and pushes the video to the command system through the streaming media server; the present invention realizes the automatic matching of POI information of the drone real-time picture, can intuitively display the street and regional information to which the current video picture belongs, provides decision-making basis for certain activity scenes, and gives play to the advantages of drones. The present invention solves the lack of information technology of real-time video matching POI of drones, and helps the rapid development of the drone industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic diagram of an implementation flow of an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The present invention is further described below in conjunction with the accompanying drawings.

[0069] Example: Figure 1 As shown, a method for matching POI information with real-time video of a drone comprises the following steps:

[0070] Step 1: Obtain the drone live video stream through the live broadcast address configured by the background system, including the current video image and the drone's latitude and longitude, altitude, pitch angle, yaw angle, roll angle, focal length, camera resolution and other information;

[0071] Step 2: Get the drone information recorded last time in the local cache, and then determine whether the latitude, longitude, altitude, and angle information in this drone information and the last drone information have changed; if they have changed, calculate the four corner coordinates of the current video screen, otherwise write the last recorded POI information into the current video screen and go to step 4;

[0072] Step 3: Check whether there is corresponding new POI information within the four corner coordinate ranges, including map POI information and custom POI information; if yes, write the new POI information into the current video screen and record the current POI information into the local cache, otherwise go directly to step 4;

[0073] Step 4: Push the current video image to the streaming media server.

[0074] This embodiment further illustrates that, in step 2, the calculation of the four corner coordinates of the current video screen includes the following steps:

[0075] (1) Calculate the geographic coordinates of each pixel;

[0076] 11) Convert the pixel coordinates to normalized coordinates in the camera coordinate system: the camera's intrinsic parameter matrix is ​​K, the pixel coordinates are (u, v), then the normalized coordinates are:

[0077]

[0078] Among them, u represents the horizontal position of the pixel, v represents the vertical position of the pixel, l represents the constant term in the homogeneous coordinates; x, y, z represent the coordinate values ​​of the normalized coordinates in the camera coordinate system. Through the above formula, the pixel coordinates (u, v) in the image are converted to the normalized coordinates (x, y, z) in the camera coordinate system;

[0079] 12) Convert the normalized coordinates to the coordinates in the camera coordinate system: The external parameters of the camera are the rotation matrix R and the translation vector T, the normalized coordinates are (x, y, z), and the coordinates in the camera coordinate system are:

[0080] [X c ,Y c ,Z c ] = R · [x, y, z] + T;

[0081] Among them, R represents the rotation matrix of the camera, which is used to describe the rotation relationship between the camera coordinate system and the world coordinate system. T represents the translation vector of the camera, which represents the translation of the origin of the camera coordinate system relative to the origin of the world coordinate system. Through the above formula, the normalized coordinates (x, y, z) in the camera coordinate system are converted to the coordinates (X c ,Y c ,Z c ), that is, converting the normalized coordinates into the actual coordinates in the camera coordinate system;

[0082] 13) Convert the coordinates in the camera coordinate system to the coordinates in the world coordinate system: the camera's posture is pitch angle, yaw angle and roll angle, and the coordinates in the camera coordinate system are (X c ,Yc ,Z c ), the coordinates in the world coordinate system are:

[0083] [X w ,Y w ,Z w ]=R pitch ·R yaw ·R roll ·[X c ,Y c ,Z c ]+C;

[0084] Among them, R pitch , R yaw , R roll : The rotation matrix describing the camera posture is used to convert the coordinates in the camera coordinate system into the coordinates in the world coordinate system; C represents the position of the camera, that is, the position of the origin of the camera coordinate system in the world coordinate system; through the above formula, the coordinates in the camera coordinate system (X c ,Y c ,Z c ) is converted to the coordinates in the world coordinate system (X w ,Y w ,Z w ), that is, converting the coordinates in the camera coordinate system into the actual coordinates in the world coordinate system for calculating the longitude and latitude coordinates of the four corners of the image;

[0085] (4) According to the coordinates in the world coordinate system (X w ,Y w ,Z w )Calculate geographic coordinates:

[0086] 21) Define the ellipsoid parameters:

[0087] a = 6378137.0 m (the semi-major axis of WGS84),

[0088] f = 1 / 298.257223563 (the flattening of WGS84);

[0089] 22) Calculate auxiliary variables:

[0090] The first eccentricity square Where b is the semi-minor axis of the ellipsoid, calculated from a and f;

[0091] Auxiliary variables

[0092] 23) Calculate latitude lat and altitude h:

[0093] Initial value lat1 = atan((Z w ·a) / (p·b));

[0094] Iterate until convergence:

[0095] Radius of curvature N = a / sqrt(1-e 2 ·sin 2 (lat)),

[0096] Height h = p / cos(lat)-N,

[0097] Update lat = atan ((Z w +e' 2 ·N·sin(lat)) / p), where e' 2 =e 2 / (1-e 2 );

[0098] 24) Calculate longitude lon:

[0099] lon=[arctan(Y w / X w )·180 / π,lat·180 / π,h];

[0100] The above formula is used to calculate the geographic coordinates of the four pixel points (A lat ,A lon )、(B lat ,B lon )、(C lat ,C lon )、(D lat ,D lon ), where sqrt represents the square root function, atan represents the inverse tangent function, arctan is the extended inverse tangent function that can correctly handle quadrant problems, sin and cos are sine and cosine functions respectively;

[0101] (5) Determine the pixel coordinates corresponding to the upper left corner, upper right corner, lower left corner, and lower right corner of the image based on the size and resolution of the image;

[0102] 31) Get known geographic coordinates and pixel coordinates:

[0103] The geographic coordinates of the four pixel points are (A lat ,A lon )、(B lat ,B lon )、(C lat ,C lon )、(D lat ,D lon ), the corresponding pixel coordinates are (A u ,A v )、(B u ,Bv )、(C u ,C v )、(D u ,D v ); the pixel coordinates of the target pixel are (P u ,P v );

[0104] 32) Calculate the geographic coordinates of the target pixel:

[0105] For the pixel coordinates of the target pixel point (P u ,P v ), and use bilinear interpolation to calculate its geographic coordinates (P lat ,P lon ):

[0106] P lat =(1-w)·((1-h)·A lat +h·C lat )+w·((1-h)·B lat +h·D lat );

[0107] The above formula represents the latitude coordinate P of the target pixel point lat It is calculated by bilinear interpolation of the geographic latitude coordinates of four adjacent pixel points;

[0108] P lon =(1-w)·((1-h)·A lon +h·Con)+w·((1-h)·B lon +h·D lon );

[0109] The above formula represents the longitude coordinate P of the target pixel point lon It is calculated by bilinear interpolation of the geographic longitude coordinates of four adjacent pixel points;

[0110] Among them, w and h represent the interpolation coefficients of the target pixel in the horizontal and vertical directions respectively; the geographic coordinates of the four corners of the image are calculated using the bilinear interpolation method, and finally the longitude and latitude coordinates of the four corners of the picture are obtained for POI information matching.

[0111] Obviously, the above embodiments are only examples for clearly explaining the present invention, and are not intended to limit the implementation of the present invention. For ordinary technicians in the relevant technical field, other different forms of changes or modifications can be made based on the above description; it is not necessary and impossible to list all the implementation methods here; and the obvious changes or modifications derived from them are still within the protection scope of the present invention.

Claims

1. A method for matching POI information of real-time video of unmanned aerial vehicle, characterized in that: The following steps are involved: Step 1: Obtain drone information in real time, including current video images and coordinate information; Step 2: compare the current drone information with the drone information obtained last time to determine whether there is a change; If there is a change, calculate the four corner coordinates of the current video screen, otherwise write the last recorded POI information into the current video screen and go to step 4; Step 3: Check whether there is corresponding new POI information within the four corner coordinate ranges; if so, write the new POI information into the current video screen and record the current POI information into the local cache, otherwise go directly to step 4; Step 4: Push the current video image to the streaming media server.

2. The method for matching POI information of drone real-time video according to claim 1 is characterized in that: The step one is specifically as follows: obtaining the drone live video stream through the live broadcast address configured by the background system, including the current video screen and the drone's latitude and longitude, altitude, pitch angle, yaw angle, roll angle, focal length, and camera resolution.

3. The method for matching POI information of drone real-time video according to claim 2 is characterized by: The step 2 is specifically as follows: obtaining the drone information recorded last time in the local cache, and then determining whether the latitude, longitude, altitude, and angle information in this drone information and the last drone information have changed; if changed, calculating the four corner coordinates of the current video screen, otherwise writing the POI information recorded last time into the current video screen, and going to step 4.

4. The method for matching POI information of drone real-time video according to claim 3 is characterized by: In step 2, the calculation of the four corner coordinates of the current video screen includes the following steps: (1) Calculate the geographic coordinates of each pixel; 11) Convert the pixel coordinates to normalized coordinates in the camera coordinate system: the camera's intrinsic parameter matrix is ​​K, the pixel coordinates are (u, v), then the normalized coordinates are: Among them, u represents the horizontal position of the pixel, v represents the vertical position of the pixel, l represents the constant term in the homogeneous coordinates; x, y, z represent the coordinate values ​​of the normalized coordinates in the camera coordinate system. Through the above formula, the pixel coordinates (u, v) in the image are converted to the normalized coordinates (x, y, z) in the camera coordinate system; 12) Convert the normalized coordinates to the coordinates in the camera coordinate system: The external parameters of the camera are the rotation matrix R and the translation vector T, the normalized coordinates are (x, y, z), and the coordinates in the camera coordinate system are: [X c ,Y c ,Z c ]=R·[x,y,z]+T; Among them, R represents the rotation matrix of the camera, which is used to describe the rotation relationship between the camera coordinate system and the world coordinate system. T represents the translation vector of the camera, which represents the translation of the origin of the camera coordinate system relative to the origin of the world coordinate system. Through the above formula, the normalized coordinates (x, y, z) in the camera coordinate system are converted to the coordinates (X c ,Y c ,Z c ), that is, converting the normalized coordinates into the actual coordinates in the camera coordinate system; 13) Convert the coordinates in the camera coordinate system to the coordinates in the world coordinate system: the camera's posture is pitch angle, yaw angle and roll angle, and the coordinates in the camera coordinate system are (X c ,Y c ,Z c ), the coordinates in the world coordinate system are: [X w ,Y w ,Z w ]=R pitch ·R yaw ·R roll ·[X c ,Y c ,Z c ]+C; Among them, R pitch , R yaw , R roll : The rotation matrix describing the camera posture is used to convert the coordinates in the camera coordinate system into the coordinates in the world coordinate system; C represents the position of the camera, that is, the position of the origin of the camera coordinate system in the world coordinate system; through the above formula, the coordinates in the camera coordinate system (X c ,Y c ,Z c ) is converted to the coordinates in the world coordinate system (X w ,Y w ,Z w ), that is, converting the coordinates in the camera coordinate system into the actual coordinates in the world coordinate system for calculating the longitude and latitude coordinates of the four corners of the image; (2) According to the coordinates in the world coordinate system (X w ,Y w ,Z w )Calculate geographic coordinates: 21) Define the ellipsoid parameters: a = 6378137.0 m (the semi-major axis of WGS84), f = 1 / 298.257223563 (the flattening of WGS84); 22) Calculate auxiliary variables: The first eccentricity square Where b is the semi-minor axis of the ellipsoid, calculated from a and f; Auxiliary variables 23) Calculate latitude lat and altitude h: Initial value lat1 = atan((Z w ·a) / (p·b)); Iterate until convergence: Radius of curvature N = a / sqrt(1-e 2 ·sin 2 (lat)), Height h = p / cos(lat)-N, Update lat = atan ((Z w +e' 2 ·N·sin(lat)) / p), where e' 2 =e 2 / (1-e 2 ); 24) Calculate longitude lon: lon=[arctan(Y w / X w )·180 / π,lat·180 / π,h]; The above formula is used to calculate the geographic coordinates of the four pixel points (A lat ,A lon )、(B lat ,B lon )、(C lat ,C lon )、(D lat ,D lon ), where sqrt represents the square root function, atan represents the inverse tangent function, arctan is the extended inverse tangent function that can correctly handle quadrant problems, sin and cos are sine and cosine functions respectively; (3) Determine the pixel coordinates corresponding to the upper left corner, upper right corner, lower left corner, and lower right corner of the image based on the size and resolution of the image; 31) Get known geographic coordinates and pixel coordinates: The geographic coordinates of the four pixel points are (A lat ,A lon )、(B lat ,B lon )、(C lat ,C lon )、(D lat ,D lon ), the corresponding pixel coordinates are (A u ,A v )、(B u ,B v )、(C u ,C v )、(D u ,D v ); the pixel coordinates of the target pixel are (P u ,P v ); 32) Calculate the geographic coordinates of the target pixel: For the pixel coordinates of the target pixel point (P u ,P v ), and use bilinear interpolation to calculate its geographic coordinates (P lat ,P lon ): P lat =(1-w)·((1-h)·A lat +h·C lat )+w·((1-h)·B lat +h·D lat ); The above formula represents the latitude coordinate P of the target pixel point lat It is calculated by bilinear interpolation of the geographic latitude coordinates of four adjacent pixel points; P lon =(1-w)·((1-h)·A lon +h·Con)+w·((1-h)·B lon +h·D lon ); The above formula represents the longitude coordinate P of the target pixel point lon It is calculated by bilinear interpolation of the geographic longitude coordinates of four adjacent pixel points; Among them, w and h represent the interpolation coefficients of the target pixel in the horizontal and vertical directions respectively; the geographic coordinates of the four corners of the image are calculated using the bilinear interpolation method, and finally the longitude and latitude coordinates of the four corners of the picture are obtained for POI information matching.

5. The method for matching POI information of drone real-time video according to claim 1 is characterized by: In step three, the new POI information includes map POI information and custom POI information.

Images