Airport apron vehicle-mounted FOD recognition method based on image recognition technology
By applying image recognition technology on vehicle-mounted equipment and combining multiple algorithms for FOD recognition, the problem of existing radar recognition methods being affected by the environment is solved, and efficient and low-cost FOD detection is achieved.
Patent Information
- Application Number
- CN202411684545.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-03-11
AI Technical Summary
The existing vehicle-mounted FOD recognition methods mainly rely on radar recognition, which is greatly affected by environmental factors such as weather and dust, and is costly and difficult to maintain.
The vehicle-mounted FOD recognition method based on image recognition technology is adopted to analyze, identify and judge FOD by acquiring the image data of the apron, and multiple algorithms such as coarse filtering algorithms, motion trajectory-based filtering algorithms and similarity comparison algorithms are used to reduce false alarms and improve recognition accuracy.
It improves FOD detection efficiency, reduces costs, reduces dependence on manual monitoring, and enhances the applicability and recognition accuracy of the system.
Smart Images
Figure CN119600569A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of aviation technology, and in particular relates to an apron vehicle-mounted FOD recognition method based on image recognition technology. Background Art
[0002] Airport FOD refers to any foreign objects that may endanger the safety of aircraft on the airport runway, taxiway, apron and surrounding areas. FOD stands for Foreign Object Debris, also known as Foreign Object Damage. FOD may be objects of various sizes, including stones, metal fragments, wheel debris, branches, etc. These FOD will cause serious damage to the engine, tires, instruments and other components on the aircraft, and even endanger the lives of passengers and crew members. Therefore, it is very necessary to attach great importance to, strictly prevent and deal with FOD incidents in a timely manner.
[0003] Airport FOD incidents are a major hidden danger affecting aviation operations. Statistics show that there are more than 4,000 FOD-related accidents every year around the world. The direct economic losses and flight delays caused by these incidents cannot be underestimated. Therefore, airport management in various countries attach great importance to strengthening FOD management, formulating corresponding FOD management plans, and implementing effective FOD prevention measures to ensure aviation safety.
[0004] Currently, the identification method for vehicle-mounted FOD is radar identification, but it is greatly affected by environmental factors such as weather and dust, and has very high requirements for radar, resulting in high costs and difficult maintenance. Summary of the invention
[0005] In order to solve the problems existing in the prior art, the present invention provides a pure visual solution for apron vehicle-mounted FOD identification method based on image recognition technology, which aims to improve detection efficiency while reducing costs, and can use existing fixed camera equipment to upgrade the back-end algorithm.
[0006] The technical solution adopted by the present invention is:
[0007] In the first aspect, the present invention provides an apron vehicle-mounted FOD identification method based on image recognition technology, which obtains image data of the apron for analysis, identification and judgment of FOD, and the specific steps are as follows:
[0008] Step 100. First, determine the target area, collect at least two frames of regional images with an interval of time T for the target area, compare the previous and next regional images, and obtain background features after processing through a coarse filtering algorithm;
[0009] Step 200. Then, continue to acquire several frame area images of the target area, select a pixel area excluding background features in each frame area image, and determine candidate motion features through a motion trajectory-based filtering algorithm;
[0010] Step 300. Finally, a similarity comparison algorithm is used to eliminate duplicate targets in the candidate features and background features, and duplicate targets in the candidate features are merged to obtain the final alarm target.
[0011] In combination with the first aspect, the present invention provides a first implementation of the first aspect. In the step 200, at least three frames of regional images are obtained to determine a candidate area excluding background features, and then a distance threshold D1 determined based on the pixel quantity of the regional image is set. The expected motion range is determined by obtaining the motion trajectory in the candidate area. If the maximum distance in the motion range is greater than D1, it is determined to be a candidate feature.
[0012] In combination with the first embodiment of the first aspect, the present invention provides a second embodiment of the first aspect. In the step 200, a distance threshold D2 is further set, and targets with a motion range smaller than D2 in the candidate area are eliminated, and targets with a motion range larger than D2 and smaller than D1 are taken as candidates, and several area images are acquired again and the motion range is re-determined for the candidates, and targets whose re-determined motion range is still smaller than D2 and whose confirmed motion range is smaller or unchanged than the previous motion range are eliminated, and targets whose motion range is larger than D2 and whose motion range is larger than the previous motion range are determined as candidate features.
[0013] In combination with the second implementation of the first aspect, the present invention provides a third implementation of the first aspect, wherein a target whose determined existence motion range remains unchanged or decreases is used as a background feature.
[0014] It should be noted that fixed objects in images are generally used as background features, but some moving objects in the actual environment are also fixed objects, such as all objects that are moved within a certain range by the wind, that is, swaying flags, rotating meteorological detection equipment, wind-blown trees, etc. If these targets are judged as moving targets, it will cause more interference in the entire calculation process, which will not only slow down the calculation process, but may also be identified as alarm targets and cause alarm interference. Therefore, multiple confirmations are performed in the above manner to first reduce the problem of detection errors caused by insufficient number of captured image frames, that is, the motion range of a single detection is not large, but the number of image frames for comparison is increased and combined with the previous comparison, the motion range can be expanded, so that it can be judged as a candidate target. Then, those with a small motion range are eliminated, and those with a large motion range are regarded as candidates, and the moving objects that belong to the interference mentioned above are eliminated.
[0015] In combination with the second implementation of the first aspect, the present invention provides a fourth implementation of the first aspect. In step 100, after the target area is determined, the target area is demarcated as an alarm area, and all targets in the alarm area with a motion range exceeding D2 are used as candidate features.
[0016] It is worth noting that the so-called alarm area refers to an area in the apron or other related areas where there are almost no moving objects or moving objects are not allowed to appear. Once a moving object is detected, it is determined to be a suspicious object for similarity comparison.
[0017] In combination with the second implementation of the first aspect, the present invention provides a fifth implementation of the first aspect, wherein the regional image of the target area is shooting data of a fixed angle and ratio acquired by at least one camera.
[0018] In combination with the second implementation of the first aspect, the present invention provides a sixth implementation of the first aspect, wherein the target area is an area that can be covered by at least one rotatable camera through scanning;
[0019] In step 100, at least two complete target areas are continuously scanned by a camera to obtain a plurality of frames of regional images, and regional images with the same angle but different scanning periods are used as comparison and then processed by a coarse filtering algorithm to obtain background features;
[0020] In step 200, the camera is turned on to scan the target area back and forth at an initial speed. In the real-time image captured by the camera, features other than background features are filtered through a filtering algorithm based on motion trajectories. Once a moving object is determined, a tracking frame with a certain ratio Q to the size of the moving object is set to move with the moving object, and a displacement boundary is determined in the implementation image. When the tracking frame moves to the displacement boundary, the mobile camera follows and shoots until the moving object is determined as a candidate feature.
[0021] It should be noted that the above scheme is calculated based on the regional image obtained by the camera in a fixed position. The real-time picture taken by the movable camera specified in this scheme is also the feedback information obtained after the frames of the real-time regional picture obtained are processed and calculated in the back-end processing module. The actual processing is also calculated based on the frames of the regional picture obtained by the camera, but a tracking picture frame with a prominent mark is displayed in the real-time picture on the feedback monitor for marking.
[0022] In combination with the first aspect, the present invention provides a seventh implementation of the first aspect, wherein the coarse filtering algorithm in step 100 includes a shadow calculation model, and the shadow of the fixed object is filtered as a background feature through the coarse filtering algorithm including the shadow calculation model.
[0023] In combination with the first aspect, the present invention provides an eighth implementation of the first aspect, wherein the coarse filtering algorithm further includes preset common fixed features.
[0024] The so-called common fixed features refer to target objects with conventional features that have been identified through existing OCR recognition software or the image recognition algorithm used in this application, such as various fixed structures, cars, frames, signs, etc., which can quickly identify and filter the features of the picture to improve efficiency.
[0025] In combination with the first aspect, the present invention provides a ninth implementation of the first aspect, wherein the regional image of the target area is shooting data acquired by a camera on at least one movable unmanned platform.
[0026] The beneficial effects of the present invention are:
[0027] (1) The present invention uses image recognition technology to automatically monitor the apron area, reducing reliance on manual monitoring and improving monitoring efficiency and coverage;
[0028] (2) The present invention can effectively distinguish non-threatening targets such as fixed objects, shadows, and common fixed features by applying multiple algorithms such as a coarse filtering algorithm, a motion trajectory-based filtering algorithm, and a similarity comparison algorithm, thereby reducing false alarms;
[0029] (3) The present invention can more accurately identify moving FOD by continuously collecting multiple frames of images and performing comparative analysis. It can also improve the accuracy of identifying real threat targets by dynamically adjusting thresholds and tracking motion trajectories. Moreover, whether it is a fixed camera or a rotatable camera, or a camera installed on a movable unmanned platform, it can be flexibly configured according to actual conditions, thereby enhancing the applicability of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a flow chart of this application. DETAILED DESCRIPTION
[0031] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments.
[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.
[0033] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0034] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0035] In the description of this application, it should be noted that if the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the product of the application is usually placed when in use. It is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application. In addition, if the terms "first", "second", etc. appear in the description of this application, they are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0036] In addition, if the terms "horizontal" or "vertical" appear in the description of this application, it does not mean that the components are required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.
[0037] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0038] Embodiment 1:
[0039] The present embodiment discloses an apron vehicle-mounted FOD identification method based on image recognition technology, which obtains image data of the corresponding area based on a camera device installed at the apron, and then analyzes the image data to determine whether FOD exists and whether the alarm condition is met.
[0040] It should be noted that an identification method is provided in this embodiment, which is not limited to the vehicle-mounted camera platform, and includes fixed and mobile camera equipment. The method is programmed into the processing module of the corresponding camera equipment in an automated program, including a single camera with embedded software, and also includes several camera platforms that interact with data through a unified back-end server.
[0041] The method is as follows:
[0042] First, determine the target area, collect two frames of regional images with an interval of at least T for the target area, compare the previous and next regional images, and obtain background features after processing through a coarse filtering algorithm;
[0043] Then, continue to obtain several frames of regional images of the target area, select a pixel area excluding background features in each frame of the regional image, and determine the candidate features of the motion through a filtering algorithm based on the motion trajectory, wherein at least more than three frames of regional images are obtained to determine the candidate area excluding the background features, and then set a distance threshold D1 determined based on the pixel quantity of the regional image screen, and determine the expected motion range by obtaining the motion trajectory in the candidate area, and if the maximum distance in the motion range is greater than D1, it is determined as a candidate feature;
[0044] Finally, the similarity comparison algorithm is used to eliminate duplicate targets in candidate features and background features, and the duplicate targets in candidate features are merged to obtain the final alarm target.
[0045] The regional image is a single-frame image captured by the camera device. The camera device can take photos separately as regional images for identification, and take photos at intervals of the set T time period to ensure that within the minimum moving period, moving objects that meet the requirements can be determined as FOD.
[0046] Furthermore, a distance threshold D2 is set, and targets with a motion range smaller than D2 in the candidate area are eliminated, and targets with a motion range larger than D2 and smaller than D1 are taken as candidates, and several area images are acquired again and the motion range is redetermined for the candidates, and targets with a redetermined motion range still smaller than D2 and a confirmed motion range that is smaller or unchanged than the previous motion range are eliminated, and targets with a motion range larger than D2 and a motion range larger than the previous motion range are determined as candidate features, and targets with a determined motion range that remains unchanged or decreases are taken as background features. After the target area is determined, the target area is demarcated as an alarm area, and all targets with a motion range exceeding D2 in the alarm area are taken as candidate features.
[0047] Furthermore, the present embodiment also provides a method for monitoring based on a rotatable fixed scanning camera. The fixed scanning camera is set at a fixed position to scan the fixed angle of the entire apron at a constant and adjustable speed as planned to obtain video data. It can be implemented directly through software programming using existing surveillance cameras, or a separate camera module that meets the requirements can be set up to run independently.
[0048] Specifically, the target area is determined, and the target area is an area that can be covered by at least one rotatable camera through scanning. First, the camera continuously scans at least two complete target areas to obtain several frames of regional images, and the regional images with the same angle but different scanning periods are used for comparison and then processed through a coarse filtering algorithm to obtain background features.
[0049] Then the camera is turned on to scan the target area back and forth at the initial speed. In the real-time image obtained by the camera, other features except background features are filtered through a filtering algorithm based on the motion trajectory. Once a moving object is determined, a tracking frame with a certain ratio value Q to the size of the moving object is set to move with the moving object, and the displacement boundary is determined in the implementation image. When the tracking frame moves to the displacement boundary, the mobile camera follows and shoots until the moving object is determined as a candidate feature.
[0050] Furthermore, a method for collecting regional images of a target area as shooting data obtained by a camera on at least one mobile unmanned platform is also provided. It is mainly divided into two parts: rough inspection and fine inspection, and all areas of the apron near the equipment are scanned to realize FOD identification.
[0051] The coarse inspection part obtains the intended target area through background modeling, and then uses a fast filtering algorithm based on motion trajectory to obtain the candidate area. The fine inspection part filters out false alarms through a similarity comparison algorithm to obtain the final alarm target.
[0052] Although the target candidate area obtained by the preliminary detection is relatively reliable and can suppress 99% of false alarms, the false alarm rate is high due to the large number of false alarms and the small number of normal intrusion events. The goal of this link is to accurately detect the target candidate area. The foreground and background of the target area are sent to the similarity comparison network for judgment. If the judgment results are similar, it is considered to be a false alarm and filtered directly; otherwise, it is considered to be a normal intrusion target and an alarm prompt is given, and it is sent to the classification network for category statistics.
[0053] Among them, the so-called similarity comparison algorithm is implemented by designing a lightweight neural network. The purpose is to compare the similarity between the foreground and background of the alarm area at the high-level semantics, and further filter out the false alarms caused by noise. In practical applications, the similarity comparison algorithm uses the similarity comparison network to solve the disadvantages of directly using the classifier for secondary judgment, making the algorithm more general and applicable to situations containing uncommon objects. The training samples of the similarity comparison algorithm are easy to obtain, only requiring the foreground and background of the moving area, and there is no requirement for the target type.
[0054] The moving targets directly extracted by background modeling usually contain various "noises", such as normal environmental factors like grass, trees, light and shadow, which are the main sources of false alarms. The goal of the preliminary detection is to quickly and effectively exclude most of the noises.
[0055] The tracking tool uses SORT, which is a simple and effective online tracking algorithm. It mainly relies on the object detector and the Kalman filter framework to match the detection box and the prediction box for data association and clean up the lost targets, and then updates the model, repeating this process. Its trajectory Straj = {x1, x2,..., xN}, and xt (t = 1, 2,..., N) is the motion state vector at time t, which can be obtained by the Kalman filter.
[0056] Filter the trajectories that do not meet the following conditions: IOU(boxN, pad(box1, r)) ≤ IOUmaxVa < 1N∑N t = 1{llvtll < Vb(1) where boxt and vt are the circumscribed rectangle box and speed of the moving object at time t, respectively, which can be obtained from xt. pad(x, r) is the boundary expansion function, indicating that the rectangle x is expanded by r pixels in all directions. IOU(x, y) represents the intersection over union of the rectangles x and y. IOUmax, Va, Vb, and r are all constants. 1.94 1.21 1.28 1.24 1.36 1.66 1.36.
[0057] The present invention is not limited to the above optional implementation manners. Anyone can obtain other various forms of products under the inspiration of the present invention. The above specific implementation manners should not be understood as limiting the protection scope of the present invention. The protection scope of the present invention should be defined by the claims, and the description can be used to interpret the claims.
Claims
1. The apron vehicle-mounted FOD identification method based on image recognition technology obtains the image data of the apron for analysis, identification and judgment of FOD, and is characterized by: The specific steps are as follows: Step 100. First, determine the target area, collect at least two frames of regional images with an interval of time T for the target area, compare the previous and next regional images, and obtain background features after processing through a coarse filtering algorithm; Step 200. Then, continue to acquire several frame area images of the target area, select a pixel area excluding background features in each frame area image, and determine candidate motion features through a motion trajectory-based filtering algorithm; Step 300. Finally, a similarity comparison algorithm is used to eliminate duplicate targets in the candidate features and background features, and duplicate targets in the candidate features are merged to obtain the final alarm target.
2. The apron vehicle-mounted FOD identification method based on image recognition technology according to claim 1 is characterized in that: In step 200, at least three frames of regional images are obtained to determine the candidate region excluding the background features, and then a distance threshold D1 determined based on the pixel quantity of the regional image is set, and the expected motion range is determined by obtaining the motion trajectory in the candidate region. If the maximum distance in the motion range is greater than D1, it is determined to be a candidate feature.
3. The apron vehicle-mounted FOD identification method based on image recognition technology according to claim 2 is characterized in that: In step 200, a distance threshold D2 is also set, and targets with a motion range smaller than D2 in the candidate area are eliminated, and targets with a motion range larger than D2 and smaller than D1 are selected as candidates, and several area images are obtained again and the motion range is redetermined for the candidates. Targets with a redetermined motion range still smaller than D2 and confirmed motion ranges that are smaller or unchanged than the previous motion range are eliminated, and targets with a motion range larger than D2 and a motion range larger than the previous motion range are determined as candidate features.
4. The apron vehicle-mounted FOD identification method based on image recognition technology according to claim 3 is characterized in that: Targets whose motion range remains constant or decreases are identified as background features.
5. The apron vehicle-mounted FOD identification method based on image recognition technology according to claim 3 is characterized in that: In step 100, after the target area is determined, the target area is demarcated as an alarm area, and all targets in the alarm area with a motion range exceeding D2 are used as candidate features.
6. The apron vehicle-mounted FOD identification method based on image recognition technology according to claim 3 is characterized in that: The regional image of the target area is shooting data of a fixed angle and ratio acquired by at least one camera.
7. The method for identifying FOD on apron vehicles based on image recognition technology according to claim 3 is characterized in that: The target area is an area that can be covered by at least one rotatable camera through scanning; In step 100, at least two complete target areas are continuously scanned by a camera to obtain a plurality of frames of regional images, and regional images with the same angle but different scanning periods are used as comparison and then processed by a coarse filtering algorithm to obtain background features; In step 200, the camera is turned on to scan the target area back and forth at an initial speed. In the real-time image captured by the camera, features other than background features are filtered through a filtering algorithm based on motion trajectories. Once a moving object is determined, a tracking frame with a certain ratio Q to the size of the moving object is set to move with the moving object, and a displacement boundary is determined in the implementation image. When the tracking frame moves to the displacement boundary, the mobile camera follows and shoots until the moving object is determined as a candidate feature.
8. The method for identifying FOD on apron vehicles based on image recognition technology according to claim 1, characterized in that: The coarse filtering algorithm in step 100 includes a shadow calculation model, and the shadow of the fixed object is filtered as a background feature through the coarse filtering algorithm including the shadow calculation model.
9. The method for identifying FOD on apron vehicles based on image recognition technology according to claim 1, characterized in that: The coarse filtering algorithm also includes preset common fixed features.
10. The method for identifying FOD on apron vehicles based on image recognition technology according to claim 1, characterized in that: The regional image of the target area is shooting data acquired by a camera on at least one movable unmanned platform.