Panoramic video aircraft placarding method, system, electronic device, and storage medium

Abstract

The application provides a panoramic video aircraft hanging plate method, system and electronic equipment. The method comprises the following steps: an ADS-B device receives an ADS-B message by using a data link; a region is divided for a to-be-detected region; a scene view model is established according to the longitude and latitude coordinates and screen pixel coordinates of the divided to-be-detected region; a mapping matrix of matching points of a scene map image to matching points of a panoramic image is applied to obtain mapping of the scene map coordinates to the panoramic image coordinates, that is, a panoramic camera mapping model; and aircraft hanging plate in the panoramic image, that is, marking of aircraft identification and tracking display, is realized according to aircraft data of the real-time ADS-B message, the scene view model and the mapping of the scene map coordinates to the panoramic image coordinates. The scheme provided by the application can accurately mark the aircraft position in the panoramic video image.

Description

Technical Field

[0001] The present invention belongs to the field of civil aviation information, and in particular relates to a panoramic video aircraft listing method, system, electronic equipment and storage medium. Background Art

[0002] Apron panoramic monitoring technology is mainly used in airport operations at medium and large airports. Due to the large scale of such airports, there are more terminals and parking spaces. In order to accommodate more parking spaces, the apron is generally divided into different finger piers. This design makes it impossible for tower controllers or apron controllers to visually see all apron positions, and thus unable to observe the operation process of aircraft. Therefore, panoramic cameras are deployed on the apron and transmit images to the tower or apron control room to assist controllers in observing the above areas, so as to better command the operation of aircraft and ensure the orderly and safe operation of the airport.

[0003] Based on the panoramic apron monitoring technology, in order to better assist the controller in command, the real-time monitoring image can be superimposed with some marker layers through image processing methods, such as parking space numbers, marking lines on the apron, aircraft positions, vehicle positions, etc.

[0004] Current technologies for locating aircraft positions in panoramic images include technologies based on image processing methods. This method identifies aircraft by performing image recognition based on features or machine learning algorithms. This method is more expensive than the method of the present invention. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention proposes a technical solution for panoramic video aircraft listing, and the specific method is as follows.

[0006] A first aspect of the present invention discloses a method for listing a panoramic video aircraft; the method comprises:

[0007] Step S1, the ADS-B device receives the ADS-B message using the ground-to-air data link;

[0008] Step S2: Divide the area to be detected into regions, and establish a scene visual model based on the latitude and longitude coordinates and screen pixel coordinates of the divided area to be detected;

[0009] Step S3: applying a mapping matrix from the matching points of the scene map image to the matching points of the panoramic image to obtain a mapping from the scene map coordinates to the panoramic image coordinates, i.e., a panoramic camera mapping model;

[0010] Step S4: Mapping the aircraft data, scene visual model and scene map coordinates in the real-time ADS-B message to the panoramic image coordinates, thereby achieving aircraft placarding in the panoramic image, wherein the placarding includes marking the aircraft identification and tracking display.

[0011] According to the method of the first aspect of the present invention, in step S1, the ADS-B message broadcasts position information of the aircraft, including:

[0012] Aircraft number, longitude, latitude, altitude, speed, heading and time.

[0013] According to the method of the first aspect of the present invention, in step S2, the specific method of dividing the area to be detected includes:

[0014] According to the expected monitoring range and the airport's scene plan, the area to be detected is divided into regions using rectangular frames.

[0015] According to the method of the first aspect of the present invention, in step S2, the specific method of establishing the scene visual model based on the latitude and longitude coordinates of the divided area to be detected and the screen pixel coordinates is:

[0016] The inverse calculation of Gaussian projection coordinates is used to obtain the mapping relationship between the longitude and latitude coordinates and the screen pixel coordinates, that is, the established scene visual model is obtained.

[0017] According to the method of the first aspect of the present invention, in step S3, a mapping matrix of matching points of the scene map image to matching points of the panoramic image is applied to obtain a mapping of scene map coordinates to panoramic image coordinates, i.e., a specific method of the panoramic camera mapping model includes:

[0018] Selecting a preset number of pairs of matching points in the panoramic image and the scene map image respectively to form a panoramic image matching point set and a map image matching point set;

[0019] Using the panoramic image matching point set and the map image matching point set as learning data, the iterative learning of the deep learning network is applied to establish a mapping from scene map coordinates to panoramic image coordinates.

[0020] According to the method of the first aspect of the present invention, in step S4, the specific method for implementing the aircraft placarding, i.e., marking the aircraft identification and tracking display in the panoramic image based on the mapping of the aircraft data, scene visual model, and scene map coordinates in the real-time ADS-B message to the panoramic image coordinates includes:

[0021] Step S41: Obtain the airspace map coordinates and scene map coordinates according to the map mapping model

[0022] Step S42: determining whether the aircraft is in an airspace map based on the aircraft data in the real-time ADS-B message; and if so, listing the aircraft in the airspace map.

[0023] Step S43: determining whether the aircraft is on a scene map based on the aircraft data in the real-time ADS-B message; and if so, listing the aircraft on the scene map;

[0024] Step S44: Obtain the coordinates of the aircraft in the panoramic image based on the panoramic camera mapping model and the scene view model, determine whether the aircraft is in the panoramic image of the scene, display the aircraft number information in the image, and mark the aircraft identification and tracking display.

[0025] According to the method of the first aspect of the present invention, in step S4, the specific method of obtaining the coordinates of the aircraft in the panoramic image based on the panoramic camera mapping model and the scene visual model and determining whether the aircraft is in the panoramic image of the scene includes:

[0026] The model is mapped by the panoramic camera to obtain the coordinates of the aircraft in the panoramic image;

[0027] According to the obtained coordinates of the aircraft in the panoramic image, the scene visual model determines whether the aircraft is in the panoramic image.

[0028] A second aspect of the present invention discloses a panoramic video aircraft listing system; the system comprises:

[0029] The first processing module is configured to receive the ADS-B message by the ADS-B device using the ground-to-air data link;

[0030] The second processing module is configured to divide the area to be detected into regions and establish a scene visual model according to the latitude and longitude coordinates and screen pixel coordinates of the divided area to be detected;

[0031] a third processing module configured to apply a mapping matrix from matching points of the scene map image to matching points of the panoramic image to obtain a mapping of scene map coordinates to panoramic image coordinates, i.e., a panoramic camera mapping model;

[0032] The fourth processing module is configured to map the aircraft data, scene visual model and scene map coordinates of the real-time ADS-B message to the panoramic image coordinates, thereby realizing the aircraft placarding, i.e. marking the aircraft identification and tracking display in the panoramic image.

[0033] A third aspect of the present invention discloses an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the panoramic video aircraft listing methods of the first aspect of the present disclosure are implemented.

[0034] A fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the panoramic video aircraft listing methods of the first aspect of the present disclosure.

[0035] It can be seen that the solution proposed by the present invention is to conduct video surveillance of the apron and runway through cameras during the daily operation of the airport. In order to monitor the operation of the aircraft more intuitively, the aircraft can be accurately positioned in the panoramic video.

[0036] In summary, the solution proposed in the present invention can accurately mark the position of the aircraft in the panoramic video image. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 Flowchart of a method for listing a panoramic video aircraft according to an embodiment of the present invention;

[0039] Figure 2 is a plan view of an airport according to an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of runway modeling according to an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of calibration of ADS-B coordinates and Google Map coordinates according to an embodiment of the present invention;

[0042] Figure 5 An example diagram of a panoramic camera image according to an embodiment of the present invention;

[0043] Figure 6 A matching view of a panoramic image and a map image according to an embodiment of the present invention;

[0044] Figure 7 is the horizontal coordinate mapping matrix mapx_map2View according to an embodiment of the present invention;

[0045] Figure 8 is the ordinate mapping matrix mapy_map2View according to an embodiment of the present invention;

[0046] 9(a)-(b) illustrate matching point mapping verification in a panoramic view according to an embodiment of the present invention;

[0047] Figure 10 A flowchart of an aircraft listing and tracking process based on position information according to an embodiment of the present invention;

[0048] Figure 11 This is an example diagram of aircraft listing in a scene map according to an embodiment of the present invention;

[0049] Figure 12 Schematic diagram of aircraft placarding based on location information according to an embodiment of the present invention;

[0050] Figure 13 This is a structural diagram of a panoramic video aircraft listing system according to an embodiment of the present invention;

[0051] Figure 14 FIG. 4 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0053] Example 1:

[0054] A first aspect of the present invention discloses a method for listing a panoramic video aircraft. Figure 1 FIG. 1 is a flow chart of a method for listing a panoramic video aircraft according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0055] Step S1, the ADS-B device receives the ADS-B message using the ground-to-air data link;

[0056] Step S2: Divide the area to be detected into regions, and establish a scene visual model based on the latitude and longitude coordinates and screen pixel coordinates of the divided area to be detected;

[0057] Step S3: applying a mapping matrix from the matching points of the scene map image to the matching points of the panoramic image to obtain a mapping from the scene map coordinates to the panoramic image coordinates, i.e., a panoramic camera mapping model;

[0058] Step S4: Mapping the aircraft data, scene visual model and scene map coordinates in the real-time ADS-B message to the panoramic image coordinates, thereby achieving aircraft placarding in the panoramic image, wherein the placarding includes marking the aircraft identification and tracking display.

[0059] In step S1, the ADS-B device receives the ADS-B message using the ground-to-air data link.

[0060] In some embodiments, in step S1, the ADS-B message broadcasts the position information of the aircraft and includes:

[0061] Aircraft number, longitude, latitude, altitude, speed, heading and time.

[0062] Specifically, the ADS-B message broadcasts the location information of the aircraft, including: aircraft number, longitude, latitude, altitude, speed, heading and time; it can provide a detailed and comprehensive understanding of the surrounding air traffic conditions; ADS-B messages update data quickly, with a refresh rate of 1 time per second, and most medium and large airports in my country have ADS-B ground stations. The data interface of the ADS-B device adopts a unified standard, which can be connected to obtain real-time aircraft data on the airport apron and surrounding areas. The present invention is only sensitive to the call sign and position and other related information of the moving target described in the ADS-B message, so it only processes the position, speed, heading, target call sign and other data items in the WGS-84 coordinates in the message, where the aircraft can be an airplane, a passenger plane, a fighter jet, etc.

[0063] In step S2, the area to be detected is divided into regions, and a scene visual model is established based on the latitude and longitude coordinates and screen pixel coordinates of the divided area to be detected.

[0064] In some embodiments, the specific method of dividing the area to be detected includes:

[0065] According to the expected monitoring range and the airport's scene plan, the area to be detected is divided into regions using rectangular frames.

[0066] Specifically, according to the expected monitoring range and the airport's surface plan, the area to be detected is divided into runway area, taxiway area, apron area and other areas. The area division adopts the rectangular frame method, such as Figure 2 As shown in the rectangular frames 201, 202, and 203 in FIG, the division can generally be performed using rectangular frames according to the actual shape of the airport runway apron area.

[0067] In some embodiments, the specific method of establishing the scene visual model based on the latitude and longitude coordinates and screen pixel coordinates of the divided area to be detected is:

[0068] The inverse calculation of Gaussian projection coordinates is used to obtain the mapping relationship between the longitude and latitude coordinates and the screen pixel coordinates, that is, the established scene visual model is obtained.

[0069] Specifically, when a moving target appears in a panoramic video, if the center point of the moving target detection frame falls within the drawn detection area, the longitude and latitude of the moving target are directly solved by mapping the scene map coordinates of the detection area to the panoramic image coordinates. If the center point of the moving target detection frame deviates from the drawn detection area, the position information of the moving target in the previous second is retrieved, and the historical trajectory of the moving target is obtained through a sliding window. The target's historical trajectory is used to infer the target's location and then solve the longitude and latitude coordinates.

[0070] Model the divided areas to be detected separately. Take the airport runway as an example. Figure 3 As shown. Given the latitude and longitude coordinates of the two endpoints A and B of the airport runway centerline in the geographic coordinate system, A(log1,lat1), B(log2,lat2), and the screen pixel coordinates of the two endpoints a and b of the runway centerline in the screen coordinate system, a(u1,v1), b(u2,v2);

[0071] The conversion of screen coordinates u and v into geographic coordinates log and lat is done using the inverse calculation of Gaussian projection coordinates. This uses the Gaussian plane as the origin and the ellipsoid as the projection surface. A 3° projection zone is used. The origin of the Gaussian projection is translated to the screen origin, which is the upper left corner of the screen. Based on these conditions, the inverse calculation expression for Gaussian projection coordinates is as follows.

[0072] [log,lat,1]=[u,v,1]T

[0073] The projection function must meet the following conditions:

[0074] The first condition: the screen u-coordinate axis is projected into the central meridian, which is the symmetry axis of the projection;

[0075] The second condition: the length of the u-axis remains unchanged after projection;

[0076] The third condition: meet the conditions of conformal projection;

[0077] Fourth condition: The coordinate origin of the Gaussian projection coincides with the screen coordinate origin after translation transformation.

[0078] In order to facilitate calculation, the data calibration method is adopted: Figure 4 As shown, nine feature points 401 are selected on the Google Map. Generally, based on the geographical features around the airport, landmarks with obvious geographical features that can be easily distinguished are selected as feature points, such as bridge heads, intersections, runway ends, roofs, etc., and their image coordinates and latitude and longitude coordinates in the map are obtained respectively. The 'Projective' algorithm of the Matlab fitgeotrans function is used to obtain the mapping relationship T between the latitude and longitude and the map image.

[0079] In some embodiments, the conversion T between longitude and latitude and map coordinates is:

[0080]

[0081] In step S3, a mapping matrix from the matching points of the scene map image to the matching points of the panoramic image is applied to obtain a mapping from the scene map coordinates to the panoramic image coordinates, ie, a panoramic camera mapping model.

[0082] In some embodiments, a mapping matrix from matching points of a scene map image to matching points of a panoramic image is applied to obtain a mapping from scene map coordinates to panoramic image coordinates, i.e., a specific method of a panoramic camera mapping model includes:

[0083] Selecting a preset number of pairs of matching points in the panoramic image and the scene map image respectively to form a panoramic image matching point set and a map image matching point set;

[0084] Using the panoramic image matching point set and the map image matching point set as learning data, the iterative learning of the deep learning network is applied to establish a mapping from scene map coordinates to panoramic image coordinates.

[0085] Specifically, such as Figure 5 The image of the panoramic camera shown is synthesized by multiple independent camera apertures. A variety of difference and transformation algorithms may be used in the synthesis process, which makes it difficult to synthesize the panoramic image with the scene terrain.

[0086] In order to solve this problem, a general real scene image calibration algorithm is designed. Figure 6 As shown, 106 pairs of matching points 601 are selected in the panoramic image and the scene map image, respectively. Figure 6 Only one matching point 601 is marked in FIG. Figure 6 The white rectangular boxes that are the same as the marked matching points 601 are all matching points. The panoramic image matching point set is view_points, and the map image matching point set is map_points. Figure 6 The matching view is shown. The following steps are used to obtain the coordinate mapping from the map coordinates to the panoramic image coordinates:

[0087] Step S31, generating a horizontal coordinate mapping matrix mapx_map2-View and a vertical coordinate mapping matrix mapy_map2View with the same map image size;

[0088] Step S32: using each matching point i (i=1-106) in the matching point set, modify the two mapping matrices;

[0089] mapx_map2View(map_points(i,2),map_points(i,1))=view_points(i,1);

[0090] mapy_map2View(map_points(i,2),map_points(i,1))=view_points(i,2);

[0091] Step S33, using a disk convolution kernel of size 5, perform five iterative convolutions on mapx_map2View and mapy_map2View, respectively, to obtain two mapping matrices as follows Figure 7 and Figure 8 As shown, Figure 7 and Figure 8 The bright part (non-black part);

[0092] Step S34: Use the obtained mapping matrix to perform mapping verification on all matching points. The mapping verification results are shown in Figures 9(a) and 9(b). Figure 9(b) is a partial enlarged view of Figure 9(a). The cross point and the solid point in the figure are the origin and the mapping point, respectively. The sum of the distances between the origin and the mapping point is used to verify whether the mapping is accurate. It can be seen from Figures 9(a) and 9(b) that the mapping of the matching points is basically accurate.

[0093] In step S4, the aircraft data, scene visual model and scene map coordinates in the real-time ADS-B message are mapped to the panoramic image coordinates to achieve the aircraft placarding, i.e. marking the aircraft identification and tracking display in the panoramic image.

[0094] In some embodiments, the specific method for implementing the aircraft placarding, i.e., marking the aircraft identification and tracking display in the panoramic image based on the mapping of the real-time aircraft data, the scene visual model, and the scene map coordinates to the panoramic image coordinates includes:

[0095] Step S41: Obtaining airspace map coordinates and scene map coordinates according to the map mapping model;

[0096] Step S42: determining whether the aircraft is in an airspace map based on the aircraft data in the real-time ADS-B message; and if so, listing the aircraft in the airspace map.

[0097] Step S43: determining whether the aircraft is on a scene map based on the aircraft data in the real-time ADS-B message; and if so, listing the aircraft on the scene map;

[0098] Step S44: Map the model using the panoramic camera to obtain the coordinates of the aircraft in the panoramic image;

[0099] According to the obtained coordinates of the aircraft in the panoramic image, the scene visual model determines whether the aircraft is in the panoramic image;

[0100] Display the aircraft number information in the image and mark the aircraft identification and tracking display.

[0101] Specifically, according to the mapping of the aircraft data, scene visual model and scene map coordinates of the real-time ADS-B message to the panoramic image coordinates, the aircraft is marked in the panoramic image, that is, the aircraft identification and tracking display are realized. The specific processing flow is as follows: Figure 10 As shown,

[0102] Step S41: Obtain the airspace map coordinates and scene map coordinates according to the map mapping model

[0103] Step S42: Determine whether the aircraft is in the airspace map based on the aircraft data in the real-time ADS-B message. If it is in the airspace map, list the aircraft in the airspace map to facilitate the location monitoring of departing and approaching aircraft.

[0104] Step S43: Determine whether the aircraft is on the scene map based on the aircraft data in the real-time ADS-B message. If the aircraft is on the scene map, list the aircraft on the scene map. Figure 11 As shown, Figure 11 The middle black frame is the location for the sign;

[0105] Step S44: Map the model using the panoramic camera to obtain the coordinates of the aircraft in the panoramic image;

[0106] According to the obtained coordinates of the aircraft in the panoramic image, the scene visual model determines whether the aircraft is in the panoramic image;

[0107] Display the aircraft number information in the image, such as Figure 12 As shown, Figure 12 The position 1301 is the aircraft identification.

[0108] In summary, the solution proposed in this application can accurately mark the position of the aircraft in the panoramic video image.

[0109] Example 2:

[0110] A second aspect of the present invention discloses a panoramic video aircraft listing system. Figure 13 FIG. 1 is a structural diagram of a panoramic video aircraft listing system according to an embodiment of the present invention; FIG. Figure 13 As shown, the system 400 includes:

[0111] The first processing module 401 is configured to receive the ADS-B message by the ADS-B device using the ground-to-air data link;

[0112] The second processing module 402 is configured to divide the area to be detected into regions and establish a scene visual model based on the latitude and longitude coordinates and screen pixel coordinates of the divided area to be detected;

[0113] The third processing module 403 is configured to apply a mapping matrix from matching points of the scene map image to matching points of the panoramic image to obtain a mapping from scene map coordinates to panoramic image coordinates, i.e., a panoramic camera mapping model;

[0114] The fourth processing module 404 is configured to map the aircraft data, scene visual model and scene map coordinates in the real-time ADS-B message to the panoramic image coordinates, thereby achieving aircraft placarding, i.e., marking aircraft identification and tracking display in the panoramic image.

[0115] Example 3:

[0116] A third aspect of the present invention discloses an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the panoramic video aircraft listing methods of the first aspect of the present disclosure are implemented.

[0117] Figure 14 FIG. 1 is a structural diagram of an electronic device according to an embodiment of the present invention. Figure 14 As shown, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the electronic device housing, or an external keyboard, touchpad or mouse.

[0118] Those skilled in the art will understand that Figure 14The structure shown in the figure is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0119] Example 4:

[0120] A fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any one of the steps in the method for listing a panoramic video aircraft according to the first aspect of the present disclosure.

[0121] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application. The description is relatively specific and detailed, but it cannot be understood as a limitation on the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of this application, several variations and improvements can be made, which all fall within the scope of protection of this application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

Claims

1. A panoramic video aircraft listing method, characterized in that: The method comprises: Step S1, the ADS-B device receives the ADS-B message using the ground-to-air data link; Step S2: Divide the area to be detected into regions, and establish a scene visual model based on the latitude and longitude coordinates and screen pixel coordinates of the divided area to be detected; Step S3: applying a mapping matrix from the matching points of the scene map image to the matching points of the panoramic image to obtain a mapping from the scene map coordinates to the panoramic image coordinates, i.e., a panoramic camera mapping model; Step S4: mapping the aircraft data, scene visual model, and scene map coordinates from the real-time ADS-B message to the panoramic image coordinates, thereby achieving aircraft placarding in the panoramic image, wherein the placarding includes marking the aircraft identification and tracking display; In step S1, the ADS-B message broadcasts the position information of the aircraft and includes: Aircraft number, longitude, latitude, altitude, speed, heading and time; In step S2, the specific method of dividing the area to be detected includes: According to the expected monitoring range and the airport's scene plan, the area to be detected is divided into regions using rectangular frames; In step S2, the specific method of establishing the scene visual model according to the latitude and longitude coordinates and screen pixel coordinates of the divided area to be detected is: The mapping relationship between the longitude and latitude coordinates and the screen pixel coordinates is obtained by inverse calculation of Gauss projection coordinates, that is, the established scene visual model is obtained; The specific method is as follows: When a moving target appears on a panoramic video, if the center point of the moving target detection frame falls within the drawn detection area, the longitude and latitude of the moving target are directly solved by mapping the scene map coordinates of the detection area to the panoramic image coordinates; If the center point of the moving target detection frame deviates from the drawn detection area, the position information of the moving target in the previous second is retrieved, and the historical trajectory of the moving target is obtained through the sliding window method. The target's location is inferred based on the historical trajectory of the moving target, and the longitude and latitude coordinates are then solved. Model the divided areas to be detected separately. The longitude and latitude coordinates of the two endpoints A and B of the airport runway centerline in the geographic coordinate system are known as A(log1,lat1) and B(log2,lat2), and the screen pixel coordinates of the two endpoints a and b of the runway centerline in the screen coordinate system are known as a(u1,v1) and b(u2,v2). The conversion of screen coordinates u, v into geographic coordinates log, lat is done by inverse calculation of Gaussian projection coordinates, that is, the Gaussian plane is used as the original surface and the ellipsoid is used as the projection surface; Using a 3° projection band, the coordinate origin of the Gaussian projection is translated to the position of the screen coordinate origin, which is the position of the upper left corner of the screen. According to the above conditions, the inverse calculation expression of the Gaussian projection coordinate is as follows: [log,lat,1]=[u,v,1]T The projection function must meet the following conditions: The first condition: the screen u-coordinate axis is projected into the central meridian, which is the symmetry axis of the projection; The second condition: the length of the u-axis remains unchanged after projection; The third condition: meet the conditions of conformal projection; The fourth condition: the coordinate origin of the Gaussian projection coincides with the screen coordinate origin after translation transformation; The data calibration method is as follows: Select 9 feature points on the map, and obtain their image coordinates and longitude and latitude coordinates based on the geographical features around the airport. Use the 'Projective' algorithm of the Matlab fitgeotrans function to obtain the mapping relationship T between longitude and latitude and the map image. In step S3, a mapping matrix from the matching points of the scene map image to the matching points of the panoramic image is applied to obtain a mapping from the scene map coordinates to the panoramic image coordinates, i.e., a specific method of the panoramic camera mapping model includes: Selecting a preset number of pairs of matching points in the panoramic image and the scene map image respectively to form a panoramic image matching point set and a map image matching point set; Using the panoramic image matching point set and the map image matching point set as learning data, applying iterative learning of a deep learning network to establish a mapping from scene map coordinates to panoramic image coordinates; In step S4, the specific method for mapping the aircraft data, scene visual model, and scene map coordinates in the real-time ADS-B message to the panoramic image coordinates to achieve the aircraft placarding, i.e., marking the aircraft identification and tracking display in the panoramic image, includes: Step S41: Obtaining airspace map coordinates and scene map coordinates according to the map mapping model; Step S42: determining whether the aircraft is in the airspace map based on the aircraft data in the real-time ADS-B message; and if so, listing the aircraft in the airspace map; Step S43: determining whether the aircraft is on a scene map based on the aircraft data in the real-time ADS-B message; and if so, listing the aircraft on the scene map; Step S44: Obtain the coordinates of the aircraft in the panoramic image based on the panoramic camera mapping model and the scene visual model, determine whether the aircraft is in the panoramic image of the scene, display the aircraft number information in the image, and mark the aircraft identification and tracking display.

2. A panoramic video aircraft listing method according to claim 1, characterized in that: In step S4, the specific method of obtaining the coordinates of the aircraft in the panoramic image based on the panoramic camera mapping model and the scene visual model and determining whether the aircraft is in the panoramic image of the scene includes: The panoramic camera maps the model to obtain the coordinates of the aircraft in the panoramic image; According to the obtained coordinates of the aircraft in the panoramic image, the scene visual model determines whether the aircraft is in the panoramic image.

3. A panoramic video aircraft listing system, characterized in that: The system adopts the method according to claim 1, and the system includes: The first processing module is configured to receive the ADS-B message by the ADS-B device using the ground-to-air data link; The second processing module is configured to divide the area to be detected into regions and establish a scene visual model according to the latitude and longitude coordinates and screen pixel coordinates of the divided area to be detected; a third processing module configured to apply a mapping matrix from matching points of the scene map image to matching points of the panoramic image to obtain a mapping of scene map coordinates to panoramic image coordinates, i.e., a panoramic camera mapping model; The fourth processing module is configured to map the aircraft data, scene visual model and scene map coordinates of the real-time ADS-B message to the panoramic image coordinates, thereby realizing the aircraft placarding, i.e. marking the aircraft identification and tracking display in the panoramic image.

4. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the panoramic video aircraft listing method according to any one of claims 1 to 2 are implemented.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the panoramic video aircraft listing method according to any one of claims 1 to 2 are implemented.

Images