An aircraft video ranging positioning method for an airport target monitoring area

Abstract

This invention discloses a video ranging and positioning method for aircraft in an airport target monitoring area. The method includes: acquiring a video image sequence containing aircraft targets within the target monitoring area using a camera; constructing an aircraft model database containing aircraft type, identification information, and geometric dimension parameters; identifying the aircraft targets and retrieving their corresponding geometric dimension parameters; using a thermal disturbance deformation field model to proportionally compensate for deformation and reconstruct the video images containing the aircraft targets in the video image sequence using the geometric dimension parameters; obtaining the aircraft fuselage pixel length of the compensated and reconstructed video images; and calculating the distance between the camera and the aircraft target fuselage using the following formula. This invention is applied to areas where airport aircraft are not towed or around the apron as target monitoring areas, enabling distance measurement and position coordinate positioning of airport aircraft when ADS-B equipment is not activated.

Description

Technical Field

[0001] This invention relates to the field of aircraft operation monitoring and video image processing, and in particular to a method for aircraft video ranging and positioning in an airport target monitoring area. Background Technology

[0002] When aircraft are in unpowered towing areas or around the apron at an airport (such as during pushback operations on the apron), they are in a powerless state and onboard equipment such as Automatic Dependent Surveillance-Broadcast (ADS-B) is turned off. Therefore, they cannot actively broadcast data such as latitude, longitude, altitude, and speed, making distance measurement and positioning impossible. Cameras can be deployed in these areas. With the development of computer vision technology, target detection and tracking using widely deployed video surveillance has become a research hotspot. Traditional methods can only detect and track aircraft targets in acquired video images, mainly used for manual monitoring of operational situations and post-event safety evidence collection. These methods cannot achieve distance measurement and positioning of aircraft on the airport ground. Furthermore, aircraft targets in video images exhibit perspective distortion due to near objects appearing larger than distant objects. Additionally, the uneven spatial distribution of air refractive index caused by airport surface heating leads to nonlinear deformation of video images, particularly local distortion caused by thermal disturbances, which also affects aircraft target recognition and distance measurement / positioning. Summary of the Invention

[0003] The purpose of this invention is to provide an aircraft video ranging and positioning method for airport target monitoring areas. This method is applied to areas where airport aircraft are not towed by power or around the apron as target monitoring areas. Video image sequences are acquired by cameras within the target monitoring area. Deformation compensation and image reconstruction are performed on the video images in the video image sequence using a thermal disturbance deformation field model. This enables distance measurement and position coordinate positioning of airport aircraft when ADS-B equipment is not activated. It is suitable for aircraft taxiing and operation scenarios where onboard equipment is unavailable, thus improving the functionality of ground operation monitoring and ranging / positioning.

[0004] The objective of this invention is achieved through the following technical solution: A method for aircraft video ranging and positioning in an airport target monitoring area, the method comprising: S1. Deploy cameras at high points in the airport location to target the monitoring area. The cameras acquire video image sequences within the monitoring area that include aircraft targets. S2. Construct an aircraft model database that includes aircraft type, identification information, and geometric dimension parameters. The geometric dimension parameters include key structural points and the dimension parameters between key structural points. Key structural points include the nose center point, the nose key endpoint, the wing key endpoint, the fuselage key endpoint, and the tail key endpoint. S3. Identify the aircraft target in the video image sequence, identify the aircraft type, and retrieve the corresponding geometric dimension parameters of the aircraft type from the aircraft type database; construct a thermal disturbance deformation field model based on thin plate spline function, and use the geometric dimension parameters to perform deformation compensation and image reconstruction on the video image containing the aircraft target in the video image sequence according to the proportion; S4. Obtain the aircraft fuselage pixel length of the video image after compensation and image reconstruction. The distance between the camera and the aircraft target fuselage is calculated using the following formula. : , The focal length of the camera. Retrieve the fuselage length corresponding to the aircraft model from the aircraft model database.

[0005] To better achieve the present invention, the present invention also includes the following methods: S5. The camera constructs a camera coordinate system, obtains the pixel coordinates of the aircraft target in the camera coordinate system, and converts the pixel coordinates of the aircraft target in the camera coordinate system to the coordinates in the airport ground coordinate system through coordinate mapping transformation.

[0006] Preferably, in method S5, a direction vector in the camera coordinate system is constructed, the horizontal and vertical angles of the camera in the airport ground coordinate system are obtained, and the composite rotation matrix for converting the camera coordinate system to the airport ground coordinate system is obtained using the horizontal and vertical angles of the camera. The line-of-sight direction in the airport ground coordinate system is obtained using the following formula. : , The line of sight in the camera coordinate system; the coordinates in the airport ground coordinate system. Calculated using the following formula: , The coordinates of the camera in the airport's ground coordinate system; The pixel coordinates of the aircraft target in the camera coordinate system according to the line of sight direction Mapping data.

[0007] Preferably, the high point of the airport location includes the high point of the airport tower, the high point of the light pole, or the high point of the airport building. The high point of the airport building includes the high point of the apron. The target monitoring area is the area where the airport aircraft are towed without power or the area around the apron.

[0008] Preferably, method S3 includes the following steps: First, based on image recognition technology, aircraft targets in the video image sequence are identified and detected to obtain the target region of the aircraft target; then, the identification information of the aircraft target in the target region is identified and extracted, and the aircraft type is identified based on the identification information; then, the geometric dimension parameters corresponding to the aircraft type of the aircraft target are retrieved from the aircraft type database; and the key structural points of the aircraft target are extracted and labeled in the video image.

[0009] Preferably, in method S3, the thin-plate spline function in the thermal perturbation deformation field model is constructed with deformation mapping functions in the horizontal x and vertical y directions of the camera coordinate system, as expressed below: ; ;in Current pixel coordinates in the camera coordinate system Deformation in the lateral x-direction, Current pixel coordinates in the camera coordinate system Deformation in the longitudinal y-direction , , These are the affine transformation coefficients in the horizontal x-direction. , , These are the affine transformation coefficients in the longitudinal y-direction. The total number of control points, with the critical structural points of the aircraft target as the control points. , From current pixel coordinates to control point Deformation corresponding to Euclidean distance Control points The weighting coefficient of deformation in the lateral x-direction. Control points The weighting coefficient of deformation in the longitudinal y-direction.

[0010] Preferably, the thermal perturbation deformation field model employs an energy function, and the objective function of the thermal perturbation deformation field is constructed by minimizing the energy function: the energy function expression is as follows: ,in For the thermal perturbation deformation field model at the control point The squared fitting error, Control points The observed pixel position, Control points The fitted pixel position, For the bending energy of the thin plate spline, For bending energy weighting coefficients, For rigidly constrained energy terms, The rigid constraint weight coefficients are used; the thermal perturbation deformation field model outputs a continuous thermal perturbation deformation field.

[0011] Preferably, thermal perturbation deformation field is used. The inverse deformation compensation mapping of the video image is expressed as follows: The video image after inverse deformation compensation mapping is reconstructed by resampling and interpolation using an image resampling interpolation method.

[0012] Preferably, the deformation The expression is as follows: , ,in Control points pixel coordinates, Previous pixel coordinates To control point coordinate European distance, It is a small positive number.

[0013] Preferably, the rotation matrix is ​​synthesized. The expression is as follows: , To utilize the camera's horizontal angle Rotation matrix for rotation in the horizontal plane, rotation matrix The expression is: ; To utilize the camera's horizontal angle Rotation matrix in the pitch direction, rotation matrix The expression is: .

[0014] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) This invention is applied to the area where airport aircraft are towed without power or the area around the apron as the target monitoring area. The video image sequence is obtained by a camera in the target monitoring area. The video image in the video image sequence is deformed and reconstructed by a thermal disturbance deformation field model. This enables the distance measurement and position coordinate positioning measurement of airport aircraft when the ADS-B equipment is not turned on. It is suitable for the aircraft taxiing stage and operation scenarios where the onboard equipment is not available, and improves the functionality of ground operation monitoring and ranging positioning.

[0015] (2) The thermal disturbance deformation field model of the present invention models the thermal disturbance refraction mechanism unique to the airport environment, uses thin plate spline function to estimate and compensate for non-rigid geometric deformation in the image, combines automatic identification of identity information and query and retrieve the aircraft type database, and performs corresponding deformation compensation for each control point of the video image through the key structural points of the aircraft and the size parameters between the key structural points, effectively reducing the jitter and distortion of the aircraft outline in the image, providing high-quality video images for deformation compensation and image reconstruction for subsequent ranging and positioning, and improving the accuracy of ranging and positioning. Attached Figure Description

[0016] Figure 1 This is a flowchart of the aircraft video ranging and positioning method of the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to embodiments: Example like Figure 1 As shown, a video ranging and positioning method for aircraft in an airport target monitoring area includes the following steps: S1. Cameras targeting the monitoring area are deployed at high points within the airport. Preferably, these high points include the high points of the airport control tower, light poles, or airport buildings, with the latter including the high points of the apron. The target monitoring area is the area where aircraft are towed without power or the area surrounding the apron. The cameras are deployed at these high points within the target monitoring area. The cameras acquire a sequence of video images containing aircraft targets within the monitoring area. The video image sequence is a continuous time frame image, and the video images in the sequence exhibit non-rigid deformation due to surface thermal disturbance. This embodiment uses aircraft ranging and positioning in an airport's unpowered towing area (a typical and long-standing operational scenario at airports) as an example to introduce the technical implementation. Cameras are deployed at high points within the target monitoring area of ​​the airport (approximately 18-25 meters in height). The cameras are fixed high-definition surveillance cameras with resolutions of [missing information - likely related to resolution]. The frame rate is The equivalent focal length is The pixel size is .

[0018] S2. Construct an aircraft model database that includes aircraft type, identification information, and geometric parameters. Geometric parameters include the dimensions of key structural points and the dimensions between key structural points. Key structural points include the center point of the nose, the key endpoint of the nose, the key endpoint of the wing, the key endpoint of the fuselage, and the key endpoint of the tail.

[0019] S3. Identify and authenticate aircraft targets in the video image sequence, and retrieve the corresponding geometric parameters from the aircraft type database. This includes the following methods: First, use image recognition technology to identify and detect aircraft targets in the video image sequence, obtaining the target area of ​​the aircraft target; then, identify and extract the identification information of the aircraft target in the target area, and identify the aircraft type based on the identification information; then, retrieve the corresponding geometric parameters of the aircraft type from the aircraft type database. For the video images in the target area, identify the aircraft fuselage number or unique identifier, and query the aircraft type database based on the identification results to obtain the corresponding aircraft's true geometric parameters. These geometric parameters include key structural points and the dimensional parameters between key structural points. Key structural points include more than twenty aircraft structural points such as the nose center point, nose key endpoint, wing key endpoint, fuselage key endpoint, and tail key endpoint; extract and label the key structural points of the aircraft target in the video images, using these key structural points as control points for the thermal disturbance deformation field model, and obtain a set of control points. In the case study of an airport aircraft unpowered towing area as the target monitoring area, the aircraft target detected in the video image sequence was a narrow-body passenger aircraft taxiing. The aircraft type was identified as an Airbus A320 by its fuselage number. After querying the aircraft type database, the geometric parameters of the Airbus A320 (e.g., fuselage length) were obtained. Wingspan is wait).

[0020] A thermal perturbation deformation field model based on thin-plate spline functions is constructed. This model uses geometrical parameters to proportionally compensate for deformation and reconstruct video images containing aircraft targets within a video image sequence. In some embodiments, the thin-plate spline functions in the thermal perturbation deformation field model construct deformation mapping functions in the horizontal x-direction (i.e., the horizontal direction of the image) and vertical y-direction (i.e., the vertical direction of the image) of the camera coordinate system. The origin of the camera coordinate system (C) is the camera optical center. The optical axis direction For the horizontal direction of the image, Let x be the vertical direction of the image, x be the horizontal direction of the image in the camera coordinate system (C), and y be the vertical direction of the image in the camera coordinate system (C). The expression is as follows: .

[0021] .in Current pixel coordinates in the camera coordinate system Deformation in the horizontal x-direction (i.e., the horizontal x-direction of the camera coordinate system). Current pixel coordinates in the camera coordinate system Deformation in the longitudinal y-direction (i.e., the longitudinal y-direction of the camera coordinate system). , , These are the affine transformation coefficients in the horizontal x-direction, specifically... This represents the overall translation in the horizontal x-direction. This refers to the linear terms of stretching and scaling that change with the horizontal x-axis (i.e., the horizontal direction of the image). This is the linear term of the shearing transformation for the vertical y-axis (i.e., the vertical direction of the image) coordinate change (i.e., the shearing or tilting transformation corresponding to the vertical y-axis). , , These are the affine transformation coefficients in the longitudinal y-direction, specifically... This represents the overall translation in the longitudinal y-direction. This is the shearing transformation term that varies with the horizontal x-axis coordinate (i.e., the horizontal direction of the image). This is a linear term representing the stretching and scaling of the longitudinal y-axis coordinate. The total number of control points, with the critical structural points of the aircraft target as the control points. , From current pixel coordinates to control point Deformation corresponding to Euclidean distance Control points The weighting coefficient of deformation in the lateral x-direction. Control points The weighting coefficient of deformation in the longitudinal y-direction. Preferably, the deformation... The expression is as follows: , ,in Control points pixel coordinates, Previous pixel coordinates To control point coordinate European distance, It is a small positive number.

[0022] In some embodiments, the thermal perturbation deformation field model employs an energy function and constructs the objective function of the thermal perturbation deformation field by minimizing the energy function: the energy function expression is as follows: ,in For the thermal perturbation deformation field model at the control point The squared fitting error, Control points The observed pixel position, Control points The fitted pixel position, For the bending energy of the thin plate spline, For bending energy weighting coefficients, For rigidly constrained energy terms, These are the rigid constraint weighting coefficients. The thermal perturbation deformation field model outputs a continuous thermal perturbation deformation field. Bending energy. The expression is as follows: , These are the second-order partial derivatives of the deformation mapping function with respect to spatial coordinates.

[0023] Utilizing thermal perturbation deformation field The inverse deformation compensation mapping of the video image is expressed as follows: The video image after inverse deformation compensation mapping is reconstructed using image resampling interpolation; the reconstructed video image is... The expression is as follows: , To compensate for the reconstructed video images.

[0024] S4. Obtain the aircraft fuselage pixel length of the video image after compensation and image reconstruction. The distance between the camera and the aircraft target fuselage is calculated using the following formula. : , The focal length of the camera. The aircraft model database is used to retrieve the fuselage length corresponding to the aircraft model. In a case study where the target monitoring area is an airport aircraft unpowered towing area, the target aircraft detected in the video image sequence is a narrow-body passenger aircraft taxiing. The aircraft model is identified as an Airbus A320 by its fuselage number. After querying the aircraft model database, the geometric parameters of the Airbus A320 (e.g., fuselage length) are obtained. Wingspan is (etc.), to obtain the equivalent focal length of the camera as The pixel size is If the compensation is based on the aircraft fuselage pixel length in the reconstructed video image, ,but The distance between the camera and the aircraft target was found to be approximately 624m, thus realizing the ranging processing of the camera's position and the aircraft target.

[0025] S5. The camera establishes a camera coordinate system, obtains the pixel coordinates of the aircraft target in the camera coordinate system, and converts the pixel coordinates of the aircraft target in the camera coordinate system to coordinates in the airport ground coordinate system through coordinate mapping transformation. In some embodiments, a direction vector in the camera coordinate system is constructed, the horizontal and vertical angles of the camera in the airport ground coordinate system are obtained, and the composite rotation matrix for converting the camera coordinate system to the airport ground coordinate system is obtained using the horizontal and vertical angles of the camera. The line-of-sight direction in the airport ground coordinate system is obtained using the following formula. : , This refers to the line of sight in the camera's coordinate system. The coordinates in the airport's ground coordinate system... Calculated using the following formula: , The coordinates of the camera in the airport's ground coordinate system. The pixel coordinates of the aircraft target in the camera coordinate system according to the line of sight direction Mapping data. Synthesizing rotation matrices. The expression is as follows: , To utilize the camera's horizontal angle Rotation matrix for rotation in the horizontal plane, rotation matrix The expression is: To utilize the camera's horizontal angle Rotation matrix in the pitch direction, rotation matrix The expression is: In addition to measuring the distance between the aircraft target and the camera position by using the camera, this embodiment can also convert the pixel coordinates of the aircraft target in the camera coordinate system to the coordinates in the airport ground coordinate system, thus obtaining the coordinates of the aircraft target in the airport ground coordinate system.

[0026] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for aircraft video ranging and positioning in an airport target monitoring area, characterized in that: The methods include: S1. Deploy cameras at high points in the airport location to target the monitoring area. The cameras acquire video image sequences within the monitoring area that include aircraft targets. S2. Construct an aircraft model database that includes aircraft type, identification information, and geometric dimension parameters. The geometric dimension parameters include key structural points and the dimension parameters between key structural points. Key structural points include the nose center point, the nose key endpoint, the wing key endpoint, the fuselage key endpoint, and the tail key endpoint. S3. Identify and retrieve the corresponding geometric parameters of aircraft targets in the video image sequence, including the following methods: First, identify and detect aircraft targets in the video image sequence using image recognition technology to obtain the target area of ​​the aircraft target; then, identify and extract the identification information of the aircraft target in the target area, and identify the aircraft type based on the identification information; then, retrieve the corresponding geometric parameters of the aircraft target's aircraft type from the aircraft type database; extract and label the key structural points of the aircraft target in the video image; construct a thermal perturbation deformation field model based on thin-plate spline functions, wherein the thermal perturbation deformation field model uses geometric parameters to proportionally perform deformation compensation and image reconstruction on the video images containing aircraft targets in the video image sequence; the thin-plate spline function in the thermal perturbation deformation field model constructs deformation mapping functions in the horizontal x and vertical y directions of the camera coordinate system, with the following expressions: ； ;in Current pixel coordinates in the camera coordinate system Deformation in the lateral x-direction, Current pixel coordinates in the camera coordinate system Deformation in the longitudinal y-direction , , These are the affine transformation coefficients in the horizontal x-direction. , , These are the affine transformation coefficients in the longitudinal y-direction. The total number of control points, with the critical structural points of the aircraft target as the control points. , From current pixel coordinates to control point Deformation corresponding to Euclidean distance Control points The weighting coefficient of deformation in the lateral x-direction. Control points The weighting coefficient of deformation in the longitudinal y-direction; the thermal perturbation deformation field model adopts an energy function and constructs the objective function of the thermal perturbation deformation field by minimizing the energy function: the expression of the energy function is as follows: ,in For the thermal perturbation deformation field model at the control point The squared fitting error, Control points The observed pixel position, Control points The fitted pixel position, For the bending energy of the thin plate spline, For bending energy weighting coefficients, For rigidly constrained energy terms, The rigid constraint weighting coefficients; the thermal perturbation deformation field model outputs a continuous thermal perturbation deformation field; the deformation... The expression is as follows: , ,in Control points pixel coordinates, Previous pixel coordinates To control point coordinate European distance, It is a tiny positive number; S4. Obtain the aircraft fuselage pixel length of the video image after compensation and image reconstruction. The distance between the camera and the aircraft target fuselage is calculated using the following formula. : , The focal length of the camera. Retrieve the fuselage length corresponding to the aircraft model from the aircraft model database.

2. The aircraft video ranging and positioning method for an airport target monitoring area according to claim 1, characterized in that: It also includes the following methods: S5. The camera constructs a camera coordinate system, obtains the pixel coordinates of the aircraft target in the camera coordinate system, and converts the pixel coordinates of the aircraft target in the camera coordinate system to the coordinates in the airport ground coordinate system through coordinate mapping transformation.

3. The aircraft video ranging and positioning method for an airport target monitoring area according to claim 1, characterized in that: In method S5, the orientation vector in the camera coordinate system is constructed, and the horizontal and vertical angles of the camera in the airport ground coordinate system are obtained. The combined rotation matrix for transforming the camera coordinate system to the airport ground coordinate system is then obtained using the horizontal and vertical angles of the camera. The line-of-sight direction in the airport ground coordinate system is obtained using the following formula. : , The line of sight in the camera coordinate system; the coordinates in the airport ground coordinate system. Calculated using the following formula: , The coordinates of the camera in the airport's ground coordinate system; The pixel coordinates of the aircraft target in the camera coordinate system according to the line of sight direction Mapping data.

4. The aircraft video ranging and positioning method for an airport target monitoring area according to claim 1, characterized in that: The high points of the airport location include the high points of the airport tower, light poles, or airport buildings. The high points of the airport buildings include the high points of the apron. The target monitoring area is the area where airport aircraft are towed without power or the area around the apron.

5. The aircraft video ranging and positioning method for an airport target monitoring area according to claim 1, characterized in that: Utilizing thermal perturbation deformation field The inverse deformation compensation mapping of the video image is expressed as follows: The video image after inverse deformation compensation mapping is reconstructed by resampling and interpolation using an image resampling interpolation method.

6. The aircraft video ranging and positioning method for an airport target monitoring area according to claim 1, characterized in that: Synthetic rotation matrix The expression is as follows: , To utilize the camera's horizontal angle Rotation matrix for rotation in the horizontal plane, rotation matrix The expression is: ; To utilize the camera's horizontal angle Rotation matrix in the pitch direction, rotation matrix The expression is: .

Images