Airport target positioning method and airport target positioning system

Abstract

The invention discloses an airport target positioning method and an airport target positioning system. The airport target positioning method comprises the steps that the airport target positioning method is applied to an airport scene, and the airport target positioning method comprises the steps that an original image is acquired; identifying a target category based on the original image, and positioning a two-dimensional detection frame of the target; obtaining at least one first key point coordinate of the target appearance on the basis of the two-dimensional detection frame; and converting the first key point coordinate into a first three-dimensional coordinate based on a preset conversion parameter. According to the technical scheme of the invention, the positioning precision can be effectively improved, and the fine control of the target can be realized.

Description

Technical Field

[0001] The invention relates to the technical field of airport monitoring, and in particular to an airport target positioning method and an airport target positioning system. Background Art

[0002] The existing airport surface surveillance technologies mainly include surface surveillance radar (surface surveillance radar), automatic dependent surveillance-broadcast (ADS-B), multi-lateration positioning system (MLAT), etc. Among them, the surface surveillance radar is based on the basic working principle of radar, uses electromagnetic waves to detect targets, emits electromagnetic waves to illuminate the target and receives its echo, so as to obtain information such as the target's position, speed, direction and altitude; ADS-B automatically broadcasts its position, altitude, speed and other information through the aircraft's onboard equipment for other aircraft and ground stations to receive, realizing real-time monitoring and traffic management; MLAT determines the position of the aircraft through the time difference between the secondary radar signal emitted by the aircraft and reaching multiple stations. According to the principle of geometry, four or more stations are usually used to determine the three-dimensional position of the aircraft.

[0003] However, among the above-mentioned scene surveillance technologies, the positioning accuracy of scene surveillance radar, ADS-B, and MLAT are 10m, 7.5m, and 5m respectively. The positioning accuracy is low, making it difficult to achieve precise control of the target. Summary of the invention

[0004] In view of the defects in the prior art, the present invention provides an airport target positioning method which can effectively improve the positioning accuracy and realize the fine control of the target.

[0005] The present application provides an airport target positioning method, which is applied to an airport scene and includes:

[0006] Get the original image;

[0007] Identify the target category based on the original image and locate the two-dimensional detection frame of the target;

[0008] Obtain at least one first key point coordinate of the target shape based on the two-dimensional detection frame;

[0009] The first key point coordinates are converted into first three-dimensional coordinates based on preset conversion parameters.

[0010] In one aspect, the step of obtaining the original image comprises:

[0011] Video content covering the airport scene is obtained, and original images are acquired from the video content.

[0012] In one aspect, a video camera is used to shoot a video covering the airport scene, at least two video cameras are set for each target direction, the two video cameras are located at different viewing angles, and the two video cameras are at least one of a thermal imaging sensor and a visible light sensor.

[0013] In one aspect, the step of identifying a target category based on the original image and determining a two-dimensional detection frame of the target based on the target category includes:

[0014] The original image is provided to a preset target detection model, the target category is identified by the target detection model, and a two-dimensional detection frame of the target is marked on the original image.

[0015] In one aspect, before the step of providing the original image to a preset target detection model, the step includes:

[0016] Collect historical videos of airport scenes under different conditions;

[0017] Using time series sampling to filter out redundant images in the historical video to obtain training images;

[0018] Marking the category and position of the target on the training image to form marked data, and the training image and the corresponding marked data form a training sample;

[0019] The training samples are collected to form a training set, and based on the initial training model, the initial training model is trained iteratively based on the training set to obtain a target detection model.

[0020] In one aspect, the step of obtaining at least one first key point coordinate of the target shape based on the two-dimensional detection frame includes:

[0021] The two-dimensional detection frame is provided to a preset marker positioning model, and at least one first key point coordinate of the target shape is obtained through the marker positioning model.

[0022] In one aspect, before the step of providing the two-dimensional detection frame to a preset marker positioning model, the step includes:

[0023] Acquire the training image;

[0024] Marking key point positions of the target in the training image in combination with the category of the target, collecting the key point positions, and forming a label set corresponding to the training image;

[0025] Generate a heat map based on the training image and the label set, wherein the training image and the corresponding heat map form a training data pair, and the training data pairs are combined to form a training data set;

[0026] Based on the pre-trained model, the pre-trained model is trained iteratively based on the training data set to obtain a marker positioning model.

[0027] In one aspect, the step of converting the first key point coordinates into first three-dimensional coordinates based on preset conversion parameters includes:

[0028] Based on preset conversion parameters, the first key point coordinates are converted into first pixel coordinates, the first pixel coordinates are converted into first camera coordinates, and the first camera coordinates are converted into first three-dimensional coordinates.

[0029] In one aspect, after the step of converting the first key point coordinates into first three-dimensional coordinates based on preset conversion parameters, the method further comprises:

[0030] Collecting first three-dimensional coordinates of the same target at different viewing angles, converting the first three-dimensional coordinates at different viewing angles into second camera coordinates, and converting the second camera coordinates into second pixel coordinates;

[0031] Selecting a point with the largest confidence as the second key point coordinate of the target based on the second pixel coordinate;

[0032] The second key point coordinates are converted into second three-dimensional coordinates based on preset conversion parameters.

[0033] In addition, in order to solve the above problems, the present application also provides an airport target positioning system, which is applied to the airport scene, and the airport target positioning system includes:

[0034] An acquisition module, wherein the acquisition module is used to acquire an original image;

[0035] An object detection module, wherein the object detection module identifies the object category based on the original image and locates a two-dimensional detection frame of the object;

[0036] A marker point positioning module, the marker point positioning module is used to obtain at least one first key point coordinate of the target shape based on the two-dimensional detection frame;

[0037] A conversion module, wherein the conversion module converts the first key point coordinates into first three-dimensional coordinates based on preset conversion parameters.

[0038] The beneficial effects of the present invention are embodied in: identifying the target category from the acquired original image, and determining the two-dimensional detection frame of the target shape through the target category. Then, based on the two-dimensional detection frame, the coordinates of the key point positions of the target shape, i.e., the first key point coordinates, are obtained. It can be seen that the technical solution of the present application can not only determine the position of the target, but also determine the position of the key position points in the target shape, improve the positioning accuracy, and realize the fine control of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0040] Figure 1 This is a schematic diagram of the process steps of the airport target positioning method in this application;

[0041] Figure 2 A schematic diagram of a two-dimensional detection frame in the airport target positioning method of this application;

[0042] Figure 3 A schematic diagram of the key point locations in the airport target positioning method of this application;

[0043] Figure 4 A schematic diagram of the process steps for obtaining the original image in the airport target positioning method of this application;

[0044] Figure 5 A schematic diagram of the process steps for identifying a two-dimensional detection frame in the airport target positioning method of this application;

[0045] Figure 6 A schematic diagram of the process steps for obtaining a target detection model in the airport target positioning method of this application;

[0046] Figure 7 A schematic diagram of the process steps for marking the coordinates of the first key point in the airport target positioning method of this application;

[0047] Figure 8 A schematic diagram of the process steps for obtaining a marker positioning model in the airport target positioning method of this application;

[0048] Fig. 9 A schematic diagram of the process steps for obtaining the first three-dimensional coordinates by using the preset conversion parameters in the airport target positioning method of the present application;

[0049] Fig.10A schematic diagram of the process steps for optimizing and converting first three-dimensional coordinates under different viewing angles to obtain second three-dimensional coordinates in the airport target positioning method of the present application;

[0050] Fig.11 This is a schematic diagram of the functional modules of the airport target positioning system for this application.

[0051] In the accompanying drawings, 100 is a two-dimensional detection frame; 110 is a key point; 101 is an acquisition module; 102 is a target detection module; 103 is a marker positioning module; 104 is a conversion module; and 105 is an offline camera calibration module. DETAILED DESCRIPTION

[0052] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0053] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the invention belongs.

[0054] like Figure 1 As shown, the present application provides an airport target positioning method, which is applied to the airport scene, and the airport scene mainly includes the ground taxiing area during the take-off and landing process of the aircraft, and the parking area. The airport target positioning method includes:

[0055] Step S10, obtaining the original image; this can be done by directly shooting with a camera or by extracting the image from a video.

[0056] Step S20, based on the original image, the target category is identified and the two-dimensional detection frame of the target is located; in addition to aircraft, vehicles moving on the airport scene also include various vehicles, such as shuttle buses, tractors, and refueling trucks. Figure 2 As shown, after the target category is identified, the basic size and shape of the target can be determined and marked using a two-dimensional detection frame 100.

[0057] Step S30, based on the two-dimensional detection frame, obtain at least one first key point coordinate of the target shape; different targets have different shape structures. The position of the key point 110 is also different. The position of the key point 110 can be understood as a structure that is obviously protruding in the target shape, or an important position. One key point 110 can be obtained, or more key points 110 can be obtained as needed. Figure 3 As shown, for example, for an airplane, the position of the key point 110 may be the nose, tail, landing gear, wing, etc. The coordinates of the key point 110 on the image are the first key point coordinates.

[0058] Step S40: convert the first key point coordinates into first three-dimensional coordinates based on preset conversion parameters. The first key point coordinates are two-dimensional coordinates, and converting them into first three-dimensional coordinates can reflect the real position of the key point in the world coordinate system, so as to locate the target more accurately.

[0059] In this embodiment, the target category is identified from the acquired original image, and the two-dimensional detection frame of the target shape is determined by the target category. Then, the coordinates of the key point positions of the target shape are obtained based on the two-dimensional detection frame, that is, the first key point coordinates. It can be seen that the technical solution of the present application can not only determine the position of the target, but also determine the position of the key position points in the target shape, improve the positioning accuracy, and achieve fine control of the target.

[0060] In addition, through the technical solution of this embodiment, accurate and reliable scene target positioning data can be provided for apron and tower control, effectively improving the accuracy of scene situation information, reducing the risk of airport scene target collision and runway intrusion, and greatly improving the safety prevention capabilities in control work.

[0061] like Figure 4 As shown, in one embodiment of the present application, the step of obtaining the original image includes:

[0062] Step S11, obtain video content covering the airport scene, and obtain the original image from the video content. By shooting the video content, the airport scene can be continuously monitored, and the monitoring capability of the airport scene can be improved. When obtaining the original image from the video content, the images can be obtained in chronological order. The video content includes a large number of pictures, for example, there are 24 frames in one second, and one frame is a picture, and a frame-by-frame acquisition method is adopted.

[0063] More specifically, in this embodiment, video processing libraries such as FFmpeg (Fast Forward Mpeg) and GStreamer() can be used to read video stream data from a video camera through the RTSP (Real Time Streaming Protocol) protocol, and perform hard decoding on a GPU (Graphics Processing Unit). The calculation delay is reduced by hard decoding, so that the three-dimensional coordinates of the target can be calculated in real time.

[0064] In one embodiment of the present application, a video camera is used to shoot a video covering the airport scene. At least two video cameras are set for each target position. The two video cameras are located at different viewing angles, and the two video cameras are at least one of a thermal imaging sensor and a visible light sensor. The viewing angles of the two video cameras can be as far apart as possible, so that the target can be photographed from different angles, ensuring that the target is in any position and any posture. There are one or more video cameras that can clearly capture the key position of the target, which can more comprehensively reflect the details of the target. Of course, the number of video cameras set for the same target position can be more than two, three or more than three. Among them, the two video cameras can be thermal imaging sensors at the same time, and can also be visible light sensors at the same time, or one of them can be a thermal imaging sensor and the other can be a visible light sensor.

[0065] In addition, thermal imaging sensors are not affected by electromagnetic signal interference and do not rely on satellite communications. They can solve the problem of signal obstruction by obstacles by increasing the number of camera points. They are not affected by external conditions such as light and weather, and can achieve uninterrupted monitoring around the clock.

[0066] In order to reduce the hardware installation of video cameras, the existing cameras in the airport can be used. If there are blind spots, additional points can be added and cameras can be installed in the blind spots. All video cameras are synchronized with NTP (Network Time Protocol) to keep the time alignment of multiple video contents pulled at the same time; a high-stability bracket is used to reduce positioning errors caused by bracket shaking and physical displacement of the video camera.

[0067] like Figure 5 As shown, in one embodiment of the present application, the step of identifying the target category based on the original image and determining the two-dimensional detection frame of the target based on the target category includes:

[0068] Step S21, providing the original image to a preset target detection model, identifying the target category through the target detection model, and marking the two-dimensional detection box of the target on the original image. The target detection model can be a large language model or a large world model. The target detection model can realize automatic judgment of the original image, reduce the error of human judgment, and improve the detection efficiency. For example, the target detection model can be a YOLO (You Only Look Once) real-time target detection model.

[0069] like Figure 6 As shown, in one embodiment of the present application, before the step of providing the original image to the preset target detection model, the following steps are included:

[0070] Step S210, collect historical videos of airport scenes under different conditions; the different conditions here include different weather conditions and different lighting conditions. Weather and lighting have a direct impact on the video picture, affecting the brightness and even clarity of the picture. Historical videos of airport scenes are usually stored in a storage device, and videos with mainly moving targets are collected from the storage device. Coarse screening can be performed to actively select video data under different weather and lighting conditions.

[0071] Step S220, redundant images in the historical video are filtered out by time-series sampling to obtain training images; the historical video is long and contains a large amount of picture information. In order to improve efficiency, some redundant images can be filtered out. In this way, representative training images can be obtained more quickly. The process of filtering out redundant images can be completed automatically by instruments and equipment, or by human-machine cooperation, and the filtering results can be manually calibrated.

[0072] Step S230, mark the category and position of the target on the training image to form the annotation data, and the training image and the corresponding annotation data form the training sample; the annotation type and target position can be automatically annotated, for example, the collected training images are automatically annotated using a target detection model. In addition, the automatically annotated data may be incorrectly annotated due to missed detection or false detection, and the position of the two-dimensional detection frame may be inaccurate, such as not framing the entire target or framing excess background. Therefore, after marking the category and position of the target, the annotation can be corrected. The correction method can be based on model correction or manual correction to ensure the accuracy of the annotation.

[0073] Step S240, collect training samples to form a training set, based on the initial training model, and perform training iterations on the initial training model based on the training set to obtain a target detection model. In this case, each labeled data corresponds to a training image, and the two form a pair of training samples, and the training set includes the training samples. By training the initial training model, fine-tuning of the initial training model is achieved, thereby obtaining a target detection model.

[0074] The specific training process for the target detection model can be: during training, the initial learning rate is set to 1E-3, and the learning rate warm-up method is used. In the first iteration, the learning rate is 0.1 times the initial learning rate, and in the second iteration, the learning rate is 0.2 times the initial learning rate, and so on, until the initial learning rate is reached. When the preset number of iterations is reached, the adaptive moment estimation method (ADAM) can also be used to optimize the initial training model.

[0075] like Figure 7 As shown, in one embodiment of the present application, the step of obtaining at least one first key point coordinate of the target shape based on the two-dimensional detection frame includes:

[0076] Step S31, providing the two-dimensional detection frame to a preset marker positioning model, and obtaining at least one first key point coordinate of the target shape through the marker positioning model. The marker positioning model is used to mark the key point positions of the target and mark the detailed coordinates of the target. In this way, the position of the target in the scene can be known more accurately, collisions can be reduced, and early warnings can be provided. The marker positioning model can be a ResNet50 model, etc.

[0077] like Figure 8 As shown, in one embodiment of the present application, before the step of providing the two-dimensional detection frame to the preset marker positioning model, the following steps are included:

[0078] Step S310, obtaining a training image; the obtained training image may be from step S220. At the same time, image capture is achieved through the position of the two-dimensional frame image, and combined with the target category information, training images that need to be marked point positioning are collected, such as aircraft targets.

[0079] Step S320, mark the key point positions of the target in the training image in combination with the target category, collect the key point positions, and form a label set corresponding to the training image; the key point positions are estimated positions of the nose, tail, landing gear, etc. of the aircraft in the training image. It is possible to first use a manual interaction method to annotate the training image, collect the coordinates of the marked points in each training image, and use them as the label set of the training image.

[0080] Step S330, generate a heat map based on the training image and the label set, the training image and the corresponding heat map form a training data pair, and the training data pairs are grouped together to form a training data set; the heat map resolution is consistent with the original training image, and the initial grayscale value of the heat map is 0. Assuming that the coordinates of the marked point corresponding to the label are (x, y), a Gaussian kernel with a standard deviation of sigma and a size of kxk is generated with (x, y) as the center. Finally, the Gaussian kernel is superimposed on the corresponding position of the training image to generate a heat map, that is, a Gaussian heat map.

[0081] Step S340, based on the pre-trained model, the pre-trained model is iterated and trained based on the training data set to obtain a marker positioning model.

[0082] The specific training process for the marker localization model can be: during training, the initial learning rate is set to 1E-3, and the learning rate warm-up method is used. In the first iteration, the learning rate is 0.1 times the initial learning rate, and in the second iteration, the learning rate is 0.2 times the initial learning rate, and so on, until the initial learning rate is reached. When the preset number of iterations is reached, the adaptive moment estimation method (ADAM) can also be used to optimize the marker localization model. The training process of the marker localization model and the target detection model is basically the same.

[0083] like Fig. 9 As shown, in one embodiment of the present application, the step of converting the first key point coordinates into the first three-dimensional coordinates based on the preset conversion parameters includes:

[0084] Step S41, based on the preset conversion parameters, the first key point coordinates are converted into first pixel coordinates, the first pixel coordinates are converted into first camera coordinates, and the first camera coordinates are converted into first three-dimensional coordinates. Among them, the goal of offline camera calibration is to obtain a set of calibration parameters. Through this set of parameters, any pixel point in the image can be converted into three-dimensional coordinates, and any point in space can also be converted into pixel coordinates. This set of parameters is the conversion parameters. Before locating the airport target, the conversion parameters can be obtained in advance and saved. When needed, the conversion parameters can be directly called.

[0085] The following example illustrates the generation process of conversion parameters:

[0086] (1) Definition of calibration parameters:

[0087] (1.1) Pixel coordinate system: Its origin is at the upper left corner of the image, the x-axis is parallel to the horizontal direction of the image, and the y-axis is parallel to the vertical direction of the image;

[0088] (1.2) Camera coordinate system: Its origin is at the optical center of the camera, in meters, the X-axis and Y-axis are parallel to the image x and y axes, the Z-axis is the camera optical axis and is perpendicular to the imaging plane;

[0089] (1.3) World coordinate system: The world coordinate system is defined as an Earth-centered, Earth-fixed coordinate system, with its origin O being the Earth's center of mass, the Z axis being parallel to the Earth's axis and pointing to the North Pole, the X axis pointing to the intersection of the prime meridian and the equator, and the Y axis being perpendicular to the XOZ plane forming a right-handed coordinate system;

[0090] (2) Conversion relationship between coordinate axes:

[0091] (2.1) Pixel coordinate system and camera coordinate system: Assume that the point (x, y) in the pixel coordinate system is the same as the point (x, y) in the camera coordinate system. C , Y C , Z C ), according to the pinhole imaging principle:

[0092]

[0093] Among them, K represents the internal parameters of the camera, which is mainly composed of the focal length f in the x direction x , focal length in y direction f y , the intersection of the optical axis and the imaging plane (c x , c y ), α is the homogeneous coefficient, and the camera intrinsic parameter K is obtained by Zhang Zhengyou’s chessboard calibration method.

[0094] (2.2) Camera coordinate system and world coordinate system:

[0095] (2.2.1) First, calculate the ground equation in the camera coordinate system, which is divided into the following three steps.

[0096] The first step is to calibrate the chessboard coordinate system and the camera coordinate system. Place the chessboard on the ground and make it completely visible in the image. Define the chessboard coordinate system, assuming that the plane where the chessboard is located is Z = 0, and the coordinates of each point on the chessboard are determined according to the actual physical length. Assume (X t , Y t , Z t ) is a three-dimensional point in the chessboard coordinate system, and its corresponding pixel point in the pixel coordinate system is (x t ,y t ), then we have:

[0097]

[0098] Among them, [r x |r y |r z ] and t represent the rotation and translation matrices from the chessboard coordinate system to the camera coordinate system. Since Zt = 0 in the chessboard coordinate system, the Z-direction rotation vector rz cannot be constrained. Therefore, according to formula eq (2), the two-dimensional coordinate point (X t , Y t ) and the two-dimensional coordinate point (x t ,y t ) can be simplified into a 3×3 matrix, and combined with the vector orthogonality constraint and SVD decomposition method, it is solved to obtain [r x |r y |r z |t].

[0099] The second step is to transform the checkerboard coordinates in the checkerboard coordinate system into the camera coordinate system based on the solution parameters of 2.2.1 to obtain discrete sampling points on the ground in the camera coordinate system.

[0100] The third step is to use the plane fitting method to fit the above sampling points to the plane formula Eqn c =(a, b, c, d).

[0101] (2.2.2) Use the measuring tool to measure the coordinates of the above sampling points in the Earth-centered Earth-fixed coordinate system. Establish the world coordinate system (X w , Y w , Z w ) and the camera coordinate system (X C , Y C , Z C) are as follows:

[0102]

[0103] Among them, R and T represent the rotation and translation matrices from the world coordinate system to the camera coordinate system, respectively, which can be solved by the least squares method. The transformation parameters include R and T.

[0104] When using the preset conversion parameters R and T, taking the perspective of one of the video cameras as an example, the process of converting the three-dimensional coordinates under a single perspective is as follows:

[0105] (1) First, the key point coordinates are relative to the original image. It is necessary to combine the two-dimensional frame information to restore the key points to the image coordinate system.

[0106] (2) Estimate the projection point of the key point to the ground: First, estimate the projection of the target direction on the ground based on the position where the landing gear tire touches the ground. Assume that the front wheel touches the ground at (x1, y1), and the two rear wheels touch the ground at (x2, y2) and (x3, y3), respectively. Then the ground projection is (x1, y1) and the line connecting [(x2+x3) / 2, (y2+y3) / 2] is represented by l. Assume that the target key point is (x key ,y key ), the projection of the key point to the ground is (x key ,y key ) to l, denoted as (x proj ,y proj ).

[0107] (3) Combined with the camera calibration parameters, the projection point (x proj ,y proj ), which is the 3D position of the target. According to formula eq(1) and Eqn c have

[0108]

[0109] Solving formula eq(4) can obtain the 3D coordinates of the target in the camera coordinate system (X C , Y C , Z C ).

[0110] (4) Using the rotation and translation parameters R and T obtained in 2.2.2, the camera coordinate system coordinates (X C , Y C , Z C ) to the world coordinate system (X w , Y w , Z w ).

[0111] like Fig.10 As shown, in a single viewing angle, the positioning result is prone to deviation, and the positioning result can be optimized through multiple viewing angles. Specifically, in an embodiment of the present application, after the step of converting the first key point coordinates into the first three-dimensional coordinates based on the preset conversion parameters, the following steps are included:

[0112] Step S510, collect the first three-dimensional coordinates of the same target under different viewing angles, convert the first three-dimensional coordinates under different viewing angles into second camera coordinates, and convert the second camera coordinates into second pixel coordinates; first collect the first three-dimensional coordinates under each viewing angle. However, it is impossible to determine which of these first three-dimensional coordinates belong to the same target. To this end, all first three-dimensional coordinates are projected to viewing angle i. Assuming that k first three-dimensional coordinate points are projected into the same target frame on viewing angle i, it is considered that these k points belong to the same target.

[0113] Step S520: Select the point with the largest confidence as the second key point coordinate of the target based on the second pixel coordinate; assuming that X i The confidence of the coordinates of the second key point corresponding to (i=1, 2, ..., n) is β i , the confidence value is obtained from the heat map estimated in the marker positioning module, that is, the grayscale value corresponding to the coordinates of the two-dimensional marker in the heat map, usually normalized to [0, 1]. Select the point with the largest confidence max(β i ), i=1, 2, ..., n as the second key point coordinates of the target.

[0114] Step S530, based on the preset conversion parameters, the second key point coordinates are converted into second three-dimensional coordinates. It can be seen that the second key point coordinates are optimized through multiple perspectives, and the second key point coordinates are converted again to obtain the optimized second three-dimensional coordinates. The second three-dimensional coordinates can more accurately reflect the key point position. It should be noted that the first key point coordinates and the first three-dimensional coordinates are both before optimization, and the second key point coordinates and the second three-dimensional coordinates are after multi-perspective optimization.

[0115] like Fig.11 As shown, the present application also provides an airport target positioning system, which is applied to the airport scene. The airport target positioning system includes: an acquisition module 101, a target detection module 102, a marker point positioning module 103 and a conversion module 104.

[0116] The acquisition module 101 is used to acquire the original image; the acquisition module 101 obtains the video content covering the airport scene and captures part of the picture from the video content as the original image. Shooting the video content can continuously monitor the airport scene and improve the monitoring capability of the airport scene. When capturing the picture from the video content, the image can be captured in a time sequence or a time interval sequence.

[0117] The target detection module 102 identifies the target category based on the original image and locates the two-dimensional detection frame 100 of the target; the target detection module 102 can realize automatic judgment of the original image based on the target detection model, reduce errors in human judgment, and improve detection efficiency.

[0118] The marking point positioning module 103 is used to obtain at least one first key point coordinate of the target shape based on the two-dimensional detection frame 100; the marking point positioning module 103 can obtain at least one first key point coordinate of the target shape through the marking positioning model. The marking positioning model is used to mark the key point 110 position of the target and mark the detailed coordinates of the target. In this way, the position of the target in the scene can be known more accurately, collisions can be reduced, and early warnings can be given.

[0119] The conversion module 104 converts the first key point coordinates into first three-dimensional coordinates based on preset conversion parameters, thereby completing the positioning of the key point 110 .

[0120] The airport target positioning system further includes an offline camera calibration module 105, which is used to generate conversion parameters R and T in advance.

[0121] The specific embodiments and beneficial effects of the airport target positioning system in this application refer to the above-mentioned airport target positioning method, which will not be repeated here.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A method for locating an airport target, characterized in that: The airport target positioning method is applied to the airport scene, and the airport target positioning method comprises: Get the original image; Identify the target category based on the original image and locate the two-dimensional detection frame of the target; Obtain at least one first key point coordinate of the target shape based on the two-dimensional detection frame; The first key point coordinates are converted into first three-dimensional coordinates based on preset conversion parameters.

2. The airport target positioning method according to claim 1, characterized in that: The step of obtaining the original image comprises: Video content covering the airport scene is obtained, and original images are acquired from the video content.

3. The airport target positioning method according to claim 2, characterized in that: A video camera is used to shoot a video covering the airport scene. At least two video cameras are set for each target position. The two video cameras are located at different viewing angles. The two video cameras are at least one of a thermal imaging sensor and a visible light sensor.

4. The airport target positioning method according to claim 1, characterized in that: The step of identifying the target category based on the original image and locating the two-dimensional detection frame of the target includes: The original image is provided to a preset target detection model, the target category is identified by the target detection model, and a two-dimensional detection frame of the target is marked on the original image.

5. The airport target positioning method according to claim 4, characterized in that: Before the step of providing the original image to a preset target detection model, the method includes: Collect historical videos of airport scenes under different conditions; Using time series sampling to filter out redundant images in the historical video to obtain training images; Marking the category and position of the target on the training image to form marked data, and the training image and the corresponding marked data form a training sample; The training samples are collected to form a training set, and based on the initial training model, the initial training model is trained iteratively based on the training set to obtain a target detection model.

6. The airport target positioning method according to claim 5, characterized in that: The step of obtaining at least one first key point coordinate of the target shape based on the two-dimensional detection frame includes: The two-dimensional detection frame is provided to a preset marker positioning model, and at least one first key point coordinate of the target shape is obtained through the marker positioning model.

7. The airport target positioning method according to claim 6, characterized in that: Before the step of providing the two-dimensional detection frame to a preset marker positioning model, the method includes: Acquire the training image; Marking key point positions of the target in the training image in combination with the category of the target, collecting the key point positions, and forming a label set corresponding to the training image; Generate a heat map based on the training image and the label set, wherein the training image and the corresponding heat map form a training data pair, and the training data pairs are combined to form a training data set; Based on the pre-trained model, the pre-trained model is trained iteratively based on the training data set to obtain a marker positioning model.

8. The airport target positioning method according to any one of claims 1 to 7, characterized in that: The step of converting the first key point coordinates into first three-dimensional coordinates based on preset conversion parameters includes: Based on preset conversion parameters, the first key point coordinates are converted into first pixel coordinates, the first pixel coordinates are converted into first camera coordinates, and the first camera coordinates are converted into first three-dimensional coordinates.

9. The airport target positioning method according to claim 8, characterized in that: After the step of converting the first key point coordinates into first three-dimensional coordinates based on preset conversion parameters, the method further comprises: Collecting first three-dimensional coordinates of the same target at different viewing angles, converting the first three-dimensional coordinates at different viewing angles into second camera coordinates, and converting the second camera coordinates into second pixel coordinates; Selecting a point with the largest confidence as the second key point coordinate of the target based on the second pixel coordinate; The second key point coordinates are converted into second three-dimensional coordinates based on preset conversion parameters.

10. An airport target positioning system, characterized in that: The airport target positioning system is applied to the airport scene, and the airport target positioning system comprises: An acquisition module, wherein the acquisition module is used to acquire an original image; An object detection module, wherein the object detection module identifies the object category based on the original image and locates a two-dimensional detection frame of the object; A marker point positioning module, the marker point positioning module is used to obtain at least one first key point coordinate of the target shape based on the two-dimensional detection frame; A conversion module, wherein the conversion module converts the first key point coordinates into first three-dimensional coordinates based on preset conversion parameters.

Images