Monocular camera apron field depth identification method and system

Abstract

The invention discloses a monocular camera airport apron depth-of-field identification method and system, and belongs to the technical field of image processing, and the method comprises the steps: S1, obtaining an airport parking apron image through a monocular camera, determining the specification of an airport parking apron physical point line on the airport parking apron image, and obtaining the related size of the physical point line of a corresponding airport; s2, generating a ground calibration grid through an edge detection and physical point line marking method; s3, marking an aircraft reference point, and constructing an aircraft coordinate system according to the aircraft reference point; s4, generating a corresponding physical world grid according to the ground calibration grid; s5, homography matrix calculation: calculating a homography matrix of an image pixel distance and a real physical world distance according to the ground calibration grid and the physical world grid; s6, object distance calculation; s7, object angle calculation; and S8, outputting the distance and the angle between the two objects according to the input central point pixel coordinates of the two objects in the airport parking apron image.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a method and system for identifying the depth of field of an apron using a monocular camera. Background Art

[0002] With the rapid development of the air transportation industry, higher requirements are put forward for the management and monitoring of airport aprons. To meet these requirements, more advanced and efficient monitoring systems must be adopted. However, traditional monitoring systems usually rely on a single fixed monocular camera, which has obvious limitations in terms of viewing angle and depth of field. Since a monocular camera cannot directly obtain depth information, it is difficult to accurately determine the position and distance of objects on the airport surface, and thus it is impossible to accurately analyze the spatial relationship between targets. This limitation seriously affects the comprehensive monitoring of the apron scene, especially in key applications such as preventing object collisions, optimizing aircraft taxiing paths, and enhancing safety.

[0003] In the complex dynamic scene of the airport apron, traditional perspective transformation and homography matrix calculation methods face many challenges. For example, due to the low contrast between the floor tile joints on the apron and the background, it is difficult to clearly extract them by means of computer vision, which increases the difficulty of generating a grid based on physical points and lines. In addition, occlusions in the scene may cause the loss of marked points, thereby affecting the construction and accurate calibration of the physical grid. These problems make it more difficult for a monocular camera to accurately locate targets. To overcome these challenges, new technologies need to be developed or existing technologies need to be improved to enhance the performance and reliability of the airport apron monitoring system. Summary of the Invention

[0004] In view of the deficiencies of the prior art, the present invention provides a method and system for identifying the depth of field of an apron using a monocular camera, aiming to solve many problems in the prior art in airport apron monitoring and spatial relationship calculation. By combining computer vision technology and the accuracy of landmark information, the present invention realizes the accurate measurement of the distance and angle between objects in the apron scene, greatly improving the information utilization rate of airport apron monitoring and providing strong technical support for airport ground operation safety and optimization.

[0005] To achieve the above-mentioned invention purpose, the first object of the present invention is to provide a method for identifying the depth of field of an apron using a monocular camera, including the following steps:

[0006] S1. Use a monocular camera to obtain an image of the airport apron, determine the physical point-line specifications of the airport apron on the airport apron image, and obtain the relevant dimensions of the physical points and lines corresponding to the airport;

[0007] S2. Generate a ground calibration grid through edge detection and landmark physical point-line landmark method;

[0008] S3. Mark the reference points of the aircraft, and construct the aircraft coordinate system based on the reference points of the aircraft;

[0009] S4. Generate the corresponding physical world grid according to the ground calibration grid;

[0010] S5. Homography matrix calculation: Calculate the homography matrix of the image pixel distance and the real physical world distance according to the ground calibration grid and the physical world grid;

[0011] S6. Object distance calculation: According to the pixel coordinates of the object center, perform perspective transformation through the homography matrix to obtain the Euclidean distance between objects relative to the real physical world;

[0012] S7. Object angle calculation: Based on the angle distortion of the aircraft coordinate system itself, calculate the angle between objects according to the pixel coordinates of the objects;

[0013] S8. According to the input pixel coordinates of the centers of two objects in the airport apron image, output the distance and angle between the two objects.

[0014] Preferably, in S2: Based on the contrast between the brick joints of the airport apron and the ground background, perform edge detection to filter the ground background.

[0015] Preferably, in S3: Select the line connecting the pixel coordinate points of the wingtips on both sides of the wing as the x-axis, and the line connecting the pixel coordinate points of the nose and the tail as the y-axis to generate the mathematical expression of the aircraft coordinate system.

[0016] Preferably, in S4: First, complete the missing marked points, then use the lower left corner point as the origin to convert to physical world coordinate points, and set the distance between points to the apron physical point line scale specification in S1.

[0017] The second object of the present invention is to provide a monocular camera apron depth of field recognition system, including:

[0018] Specification preprocessing module, which uses a monocular camera to obtain an airport apron image, determines the physical point line specification of the airport apron on the airport apron image, and obtains the relevant dimensions of the physical point line of the corresponding airport;

[0019] Grid calibration module, which generates a ground calibration grid through edge detection and the method of marking physical point lines;

[0020] Coordinate system construction module, which marks the reference points of the aircraft and constructs the aircraft coordinate system based on the reference points of the aircraft;

[0021] Physical world grid generation module, which generates the corresponding physical world grid according to the ground calibration grid;

[0022] Matrix construction module, homography matrix calculation: Calculate the homography matrix of the image pixel distance and the real physical world distance according to the ground calibration grid and the physical world grid;

[0023] Distance analysis module, object distance calculation: According to the pixel coordinates of the object center, perform perspective transformation through the homography matrix to obtain the Euclidean distance between objects relative to the real physical world;

[0024] Angle analysis module, object angle calculation: Based on the angle distortion of the aircraft coordinate system itself, calculate the angle between objects according to the pixel coordinates of the objects;

[0025] Interaction module, according to the pixel coordinates of the center points of two input objects in the airport apron image, output the distance and angle between the two objects.

[0026] Preferably, in the grid calibration module: Based on the contrast between the brick joints of the airport apron and the ground background, perform edge detection to filter the ground background.

[0027] Preferably, in the coordinate system construction module: Select the line connecting the pixel coordinate points of the wingtips on both sides of the wing as the x-axis, and the line connecting the pixel coordinate points of the nose and the tail as the y-axis to generate the mathematical expression of the aircraft coordinate system.

[0028] Preferably, in the physical world grid generation module: First, complete the missing marked points, then use the lower left corner point as the origin to convert to physical world coordinate points, and set the distance between points to the apron physical point line scale specification in the specification preprocessing module.

[0029] The third object of the present invention is to provide a computer-readable storage medium that stores a computer program, and when the program is executed by a processor, it implements the above-mentioned monocular camera apron depth of field recognition method.

[0030] The fourth object of the present invention is to provide a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the above-mentioned monocular camera apron depth of field recognition method.

[0031] The advantages and positive effects of this application are:

[0032] The present invention proposes an innovative technical solution, which marks the physical points and lines in the airport apron scene through automated means and further calculates the homography matrix. This process combines the construction technology of physical grid points and can indirectly achieve the depth perception of the airport apron scene. Through this method, accurate positioning support can be effectively provided for the target object. This method significantly breaks through the limitations of traditional monocular cameras in obtaining depth of field information and opens up a new technical path for improving the efficiency and accuracy of the airport apron monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0034] Figure 1 Shows the flowchart provided by the preferred embodiment of the present invention;

[0035] Figure 2 Shows the processing process diagrams of the homography matrix and the body coordinate system in the preferred embodiment of the present invention;

[0036] Figure 3 Shows the calculation diagram of the object position relationship in the preferred embodiment of the present invention;

[0037] Figure 4 Shows the operation process diagram of the preferred embodiment of the present invention;

[0038] Figure 5 Is a schematic diagram of the ground calibration grid in the preferred embodiment of the present invention;

[0039] Figure 6 Is a schematic diagram of the body coordinate system in the preferred embodiment of the present invention;

[0040] Figure 7 Is a schematic diagram of the output position relationship in the preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0042] Please refer to Figures 1 to 4 ;

[0043] The first embodiment, a method for identifying the depth of field of a monocular camera apron, includes the following steps:

[0044] S1: Determine the specifications and dimensions of the physical points and lines on the airport apron; determine the specifications and dimensions of the physical points and lines on the airport apron. The airport apron scene is a relatively fixed visual scene. The construction specifications of different airports are usually inconsistent, but the construction standards of the same airport are the same. The physical points and lines on the ground of a single airport apron have consistent physical dimensions, and the dimension information will be used to calculate the homography matrix, so as to generate the subsequent required grid according to the specifications of the airport physical points and lines as parameters. Specifically include:

[0045] First, use a monocular camera to obtain an image of the airport apron, and then determine the specifications of the physical points and lines on the airport apron image on the said airport apron image, and obtain the relevant dimensions of the physical points and lines of the corresponding airport;

[0046] S2. Generate a ground calibration grid through edge detection and the method of marking physical points and lines, as Figure 5 shown; specifically include:

[0047] Since the contrast between the brick joints on the airport apron and the ground background is relatively low in the visual scene, edge detection can be performed on this color range to filter the influence of the ground background. At the same time, the apron scene is fixed, and physical points and lines such as ground brick joints can be used to automatically generate a ground calibration grid by combining the edge detection results and the marked lines;

[0048] The present invention extracts the pixel coordinates of key intersection points through edge detection in a specific color gamut and the method of marking physical points and lines, automatically generates a ground calibration grid, and reduces the problem of poor extraction accuracy caused by low contrast.

[0049] The present invention automatically annotates the ground physical points and lines of the input independent fixed scene according to edge detection, and improves the accuracy of the ground calibration grid with marked points and lines; for the points that cannot be annotated due to occlusion in the scene, use non-existent point markers (such as -1, etc.) for supplementation to generate a ground annotation grid.

[0050] S3. Mark the reference points of the aircraft, and construct an aircraft coordinate system according to the reference points of the aircraft; as Figure 6 shown; the reference points of the aircraft include the wing tips, the nose, and the tail. Establish an aircraft coordinate system through pixel coordinate points to provide a reference system for subsequent angle calculation; specifically include:

[0051] After the aircraft is on the wheel chocks and the fuselage is stable, mark the pixel coordinates of the wing tips and the pixel coordinates of the nose and tail and construct an aircraft body coordinate system, and use the coordinate system as the parameters required for measuring the angle;

[0052] For example: Select the line connecting the pixel coordinate points of the two wing tips on both sides of the wing as the x-axis according to the aircraft body coordinate system of the aircraft, and the line connecting the pixel coordinate points of the nose and tail as the y-axis to automatically generate the mathematical expression of the coordinate system.

[0053] S4. Generate the corresponding physical world grid according to the ground calibration grid. Since there are usually occluders in the scene, there will be missing markers between the rows (x-axis) and columns (y-axis) of markers in S3. After completing the points, they are converted into physical world coordinate points with the lower left corner point as the origin. The distance between points is set to the physical point ruler specification of the apron obtained in S1.

[0054] After completing all the processes that need to be marked, the marked points obtained in S2 are constructed into a physical world grid with the same shape. In the present invention, the lower left corner marked point is used as the physical grid origin to construct a grid that can correspond to the marked points in S2, and the physical world grid is used as the parameter required for ranging;

[0055] In S4, the physical points and lines annotated and completed in S2 are compared to construct the corresponding physical world grid, and the dimensions of the physical points and lines of the airport apron determined in S1 are substituted.

[0056] S5, homography matrix calculation: calculate the homography matrix of image pixel distance and real physical world distance according to the ground calibration grid and the physical world grid;

[0057] The present invention calculates the mapping relationship between image pixel distance and real physical world distance according to the ground calibration grid and the physical world grid, that is, its homography matrix, for perspective transformation.

[0058] S6. Object distance calculation: Based on the pixel coordinates of the object center, the perspective transformation is performed through the homography matrix to obtain the Euclidean distance between objects relative to the real physical world, and then the physical distance between the two center points is obtained; Figure 7 As shown;

[0059] The present invention utilizes a homography matrix to convert the pixel coordinates of an object into physical world coordinates through the homography matrix, and then calculates the Euclidean distance between objects, thereby achieving accurate measurement of the real physical world distance.

[0060] As attached Figure 3 As shown, the pixel coordinates of the center of the object input by the user are transformed into physical coordinates by using the homography matrix calculated by S5, and the Euclidean distance between points is calculated to obtain the distance between the objects. In this example, the distance between the object and the center point of the aircraft is calculated.

[0061] S7, Object Angle Calculation: Based on the angular distortion of the aircraft coordinate system itself, the angles between objects are calculated according to the pixel coordinates of the objects; providing support for direction and positioning during airport operations;

[0062] In the apron scene, the aircraft body coordinate system itself has the perspective transformation angle of the physical world. The pixel coordinates in the image are vectorized based on the body coordinate system and then the angle between the two is calculated.

[0063] As shown in the Figure 3 appendix, by using the center pixel coordinates of the object input by the user and the aircraft body coordinate system constructed by S3, the included angle between the object coordinates is converted into a vector, so as to calculate the angular relationship between the objects. In this example, the included angle between the object and the center point of the aircraft is calculated.

[0064] S8. According to the center pixel coordinates of two objects in the airport apron image input, output the distance and angle between the two objects.

[0065] For example: According to the center pixel coordinates of object A and object B in the image input, output the distance and angle between object A and object B, and display the distance and angle between the objects on the image.

[0066] The second embodiment is a monocular camera apron depth-of-field recognition system for implementing the method of the first embodiment. The system includes:

[0067] A specification preprocessing module captures images of the airport apron by using a monocular camera. In these captured airport apron images, the physical point-line specifications of the airport apron can be further determined. These specifications include various important marking lines and positioning points on the apron, which are crucial for the safe docking of aircraft and the operations of ground service personnel. After identifying and positioning these physical point-lines, specific dimension data related to these point-lines need to be obtained. These data usually come from the detailed planning drawings or technical specification descriptions of the corresponding airport. By analyzing these dimension data, it can be ensured that the layout of the airport apron meets safety and operation standards, thus providing an accurate reference for aircraft takeoff and landing and ground services.

[0068] A grid calibration module can effectively generate a ground calibration grid by applying edge detection technology and the method of marking physical point-line marks; specifically, by using the contrast difference between the brick joints on the airport apron and the surrounding ground background, edge detection is performed, so as to effectively filter out unnecessary ground background information.

[0069] A coordinate system construction module marks the reference points of the aircraft, which is the basic step for constructing the aircraft coordinate system. By accurately positioning these reference points, the accuracy and consistency of the coordinate system can be ensured. Next, a coordinate system suitable for the aircraft will be constructed based on these reference points. Specifically, the wing tip pixel coordinate points on both sides of the wing are selected, and the line connecting these two points is defined as the x-axis. Similarly, the pixel coordinate points of the nose and the tail are selected, and the line connecting these two points is defined as the y-axis. By this method, a coordinate system that can accurately describe the position and direction of the aircraft can be generated, and this coordinate system can be accurately described by a mathematical expression.

[0070] The physical world grid generation module generates corresponding physical world grids according to the ground calibration grids;

[0071] Given that there are often occlusions in the scene, there may be missing marker points between the rows (x-axis) and columns (y-axis) of the marker points in the coordinate system construction module. After filling in the missing points, with the bottom left corner point as the origin, the coordinates are converted into the coordinate points of the physical world, and the distance between points is set to the apron physical point line scale specification obtained in the specification preprocessing module;

[0072] After completing all necessary annotation processes, the marker points obtained in the grid calibration module will be constructed into physical world grids with consistent shapes. The present invention uses the bottom left corner marker point as the origin of the physical grid, constructs a grid corresponding to the marker points in the grid calibration module, and uses the physical world grid as the parameters required for distance measurement;

[0073] The matrix construction module calculates the homography matrix: Based on the ground calibration grid and the physical world grid, calculate the homography matrix between the image pixel distance and the real physical world distance;

[0074] The distance analysis module focuses on the accurate calculation of the object distance. By obtaining the pixel coordinates of the object center and performing perspective transformation using the homography matrix, the two-dimensional coordinates in the image can be converted into the coordinates in the three-dimensional space. This process enables the present invention to calculate the Euclidean distance between objects in the real physical world, that is, the actual distance between objects.

[0075] The angle analysis module is responsible for the accurate analysis of the object angles. Considering the possible angle distortion of the aircraft coordinate system, based on the pixel coordinates of the objects, through a series of complex geometric calculations, the relative angles between objects can be deduced. This module is crucial for understanding the positioning and orientation of objects in space.

[0076] The interaction module, as the bridge for user-system interaction, receives the pixel coordinates of the center points of two objects input by the user in the airport apron image. Based on this data, the interaction module calls the functions of the distance analysis module and the angle analysis module, comprehensively calculates the distance and angle between the two objects, and presents this information to the user in an intuitive way. In this way, the user can easily obtain the information they need about the relative position and orientation between objects.

[0077] The third embodiment is a computer-readable storage medium that stores a computer program, and when the program is executed by a processor, it implements the above-mentioned monocular camera apron depth of field recognition method.

[0078] The fourth embodiment is a computer program product that includes a computer program, and when the computer program is executed by a processor, it implements the above-mentioned monocular camera apron depth of field recognition method.

[0079] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0080] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for recognizing the depth of field of a monocular camera apron, characterized in that: include: S1. Using a monocular camera to obtain an image of an airport apron, determining the physical point and line specifications of the airport apron on the image of the airport apron, and obtaining the relevant dimensions of the physical point and line of the corresponding airport; S2, generating a ground calibration grid by edge detection and physical point and line marking method; S3, marking the aircraft reference point, and constructing the aircraft coordinate system according to the aircraft reference point; S4, generating a corresponding physical world grid according to the ground calibration grid; S5, homography matrix calculation: calculate the homography matrix of image pixel distance and real physical world distance according to the ground calibration grid and the physical world grid; S6, object distance calculation: According to the pixel coordinates of the center of the object, the perspective transformation is performed through the homography matrix to obtain the Euclidean distance between objects relative to the real physical world; S7, object angle calculation: based on the angle distortion of the aircraft coordinate system itself, the angle between the objects is calculated according to the pixel coordinates of the objects; S8. Output the distance and angle between the two objects according to the pixel coordinates of the center points of the two objects in the airport apron image.

2. The method for recognizing the depth of field of a monocular camera apron according to claim 1, characterized in that: In S2: based on the contrast between the brick seams of the airport apron and the ground background, edge detection is performed to filter the ground background.

3. The method for recognizing the depth of field of a monocular camera apron according to claim 1, characterized in that: In S3: select the line connecting the pixel coordinate points of the wingtips on both sides of the wing as the x-axis, and the line connecting the pixel coordinate points of the nose and tail as the y-axis, and generate the mathematical expression of the aircraft coordinate system.

4. The method for recognizing the depth of field of a monocular camera apron according to claim 1, characterized in that: In S4: First, complete the missing marking points, then convert them into physical world coordinate points with the lower left corner point as the origin, and set the distance between points to the apron physical point ruler specification in S1.

5. A monocular camera apron depth recognition system, characterized in that: include: A specification preprocessing module, which uses a monocular camera to obtain an airport apron image, determines the airport apron physical point and line specifications on the airport apron image, and obtains the physical point and line related dimensions of the corresponding airport; The grid calibration module generates the ground calibration grid through edge detection and physical point and line marking method; A coordinate system construction module marks the aircraft reference points and constructs the aircraft coordinate system based on the aircraft reference points; The physical world grid generation module generates the corresponding physical world grid according to the ground calibration grid; Matrix construction module, homography matrix calculation: Calculate the homography matrix of image pixel distance and real physical world distance based on the ground calibration grid and the physical world grid; Distance analysis module, object distance calculation: Based on the pixel coordinates of the object center, the perspective transformation is performed through the homography matrix to obtain the Euclidean distance between objects relative to the real physical world; Angle analysis module, object angle calculation: Based on the angle distortion of the aircraft coordinate system itself, the angle between objects is calculated according to the pixel coordinates of the objects; The interaction module outputs the distance and angle between the two objects based on the pixel coordinates of the center points of the two objects in the airport apron image.

6. The monocular camera apron depth of field recognition system according to claim 5, characterized in that: In the grid calibration module: edge detection is performed based on the contrast between the brick seams of the airport apron and the ground background, and the ground background is filtered.

7. The monocular camera apron depth of field recognition system according to claim 5, characterized in that: In the coordinate system construction module: select the line connecting the pixel coordinate points of the wingtips on both sides of the wing as the x-axis, and the line connecting the pixel coordinate points of the nose and tail as the y-axis to generate the mathematical expression of the aircraft coordinate system.

8. The monocular camera apron depth recognition system according to claim 5, characterized in that: In the physical world grid generation module: first, complete the missing marker points, then convert them into physical world coordinate points with the lower left corner point as the origin, and set the distance between points to the apron physical point ruler specification in the specification preprocessing module.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the monocular camera apron depth of field recognition method described in any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the monocular camera apron depth of field recognition method as described in any one of claims 1 to 4 is implemented.

Images