Optical airport runway foreign matter detection method

Abstract

The invention discloses an optical airport runway foreign matter detection method, which comprises the following steps: setting parameters of optical equipment, and carrying out scanning evidence collection on an airport runway area to obtain an initial evidence collection image; establishing a background model and completing primary detection of runway foreign matters; according to the result of the primary detection, secondary confirmation detection of the foreign matter is carried out; comparing the results of the primary detection and the secondary confirmation detection, eliminating interference, and obtaining final foreign matter detection information; the optical equipment is used for scanning the airport runway area according to a preset period, primary monitoring and secondary confirmation monitoring are carried out, foreign matter information is reported finally according to preset conditions, and optical airport runway foreign matter detection is completed. The problems that photoelectric image-based runway foreign matter detection is narrow in coverage range, high in false detection rate, low in robustness and the like are solved, a supervision detection mode is not needed, long-time accumulation of samples of real foreign matters is not needed, and the engineering degree is higher.

Description

Technical Field

[0001] The invention relates to a foreign body detection method, in particular to an optical airport runway foreign body detection method. Background Art

[0002] This section merely provides background information related to the present disclosure and is not necessarily prior art.

[0003] Foreign objects on the runway, also known as airport foreign objects (FOD), refer to foreign substances that can cause damage to aircraft, such as loose metal parts, machine tools, rubber fragments and concrete blocks, etc., which can easily cause damage to aircraft engines and tires, and even cause safety accidents, endangering safe flight, with disastrous consequences.

[0004] At present, the handling of foreign objects on airport runways mainly relies on the airport's designated patrol rules, and manual patrols are carried out at specified time periods to detect and remove foreign objects. This method increases the workload of airport staff and is unable to detect them in a timely manner, which poses certain hidden dangers. In order to eliminate the impact of foreign objects on flight safety, more and more scholars are applying radar and optoelectronics to the detection of foreign objects on airport runways to ensure the safe operation of aircraft on the airport surface.

[0005] At present, radar is used to detect foreign objects on the runway, whether it is a tower-type radar or an edge-light-type radar. The equipment cost is high and maintenance is difficult. The detection ability will be greatly reduced in rainy days or when the runway is flooded. At the same time, radar detection is not intuitive enough. Therefore, some manufacturers have begun to use optical equipment to detect foreign objects on airport runways. However, some use single-camera video image processing, which has a small coverage range and is easily affected by weather, camera shake, etc. Some have supervised deep learning networks for detection, which does not meet the actual use of the airport. There are problems such as poor robustness and low detection probability for real foreign objects, which cannot meet the engineering needs of the actual site.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0007] Purpose of the invention: The technical problem to be solved by the present invention is to provide an optical airport runway foreign body detection method in view of the deficiencies in the prior art.

[0008] In order to solve the above technical problems, the present invention discloses an optical airport runway foreign body detection method, comprising the following steps:

[0009] Step 1, setting the parameters of the optical equipment, scanning the airport runway area for evidence collection, and obtaining an initial evidence collection image;

[0010] Step 2, establish a background model and complete the initial detection of foreign objects on the runway;

[0011] Step 3: Based on the results of the initial test, a secondary confirmation test for foreign matter is performed;

[0012] Step 4: Compare the results of the initial test and the secondary confirmation test, eliminate interference, and obtain the final foreign body detection information;

[0013] Step 5, use the optical device to scan the airport runway area according to a preset period, and perform initial monitoring and secondary confirmation monitoring, and finally report foreign object information according to preset conditions to complete the optical airport runway foreign object detection.

[0014] Furthermore, the scanning and evidence collection of the airport runway area described in step 1 specifically includes:

[0015] Step 1-1, measuring the two end positions of the optical device and the airport runway area; wherein the two end positions of the airport runway area are respectively taken as the lower edge position of the left end of the runway area and the upper edge position of the right end of the runway area, and the longitude and latitude coordinates of the optical device are calculated as (la cammer ,lon cammer ), the latitude and longitude coordinates of the two end positions of the airport runway area are (la left ,lon left ) and (la right ,lon right ), la represents latitude information, lon represents longitude information;

[0016] Step 1-2, for the three longitude and latitude coordinates obtained in step 1-1, calculate the horizontal azimuth angle α between the two ends of the runway area and the optical device. The specific method is as follows:

[0017]

[0018] Step 1-3, aim the optical device at the farthest runway area, select a magnification value Z to complete clear observation of the runway area, and obtain the horizontal field of view angle p of the optical device based on the magnification value Z;

[0019] Step 1-4, based on the horizontal field of view angle p obtained in step 1-3, calculate the number of forensic preset positions N of the optical device. The specific method is as follows:

[0020]

[0021] Among them, Ceil represents the upward rounding function;

[0022] Step 1-5, rotate the optical device to the edge of one end of the runway area, according to the multiple value Z set in step 1-3, each horizontal rotation angle p is used as a forensic preset position, and the shooting positions are recorded as 1, 2, ..., N, rotate N times and shoot forensic images;

[0023] Step 1-6, numbering the evidence images taken in step 1-5, wherein the numbers correspond to the evidence preset position numbers one by one, and the evidence images are recorded as img1 to imgN, and the runway area is manually delineated in the evidence images.

[0024] Furthermore, the establishment of the background model and the completion of the initial detection of foreign objects on the runway described in step 2 specifically include:

[0025] Step 2-1, under the condition that there is no foreign matter in the runway area, take a group of forensic images without foreign matter, recorded as Back1, Back2, ..., BackN;

[0026] Step 2-2, the optical device captures a forensic image according to the method described in step 1-5. When the optical device rotates to the capturing position M, a forensic image imgM is captured, where 1<=M<=N;

[0027] Step 2-3, using the Gaussian mixture model, perform differential processing on imgM and the background model BackM of the corresponding shooting position to obtain the foreground image FroundM;

[0028] Step 2-4, using median filtering to perform noise reduction processing on the foreground image FroundM to obtain a noise reduction image BlurFroundM;

[0029] Step 2-5, perform a closing operation on the denoised image BlurFroundM to obtain a closed image FinalM, and then complete the initial detection of foreign objects on the runway through outer contour search and based on the runway area delineated in step 1-6, and obtain the preliminary scale and position information rect_gmmM of the foreign objects in the forensic image imgM numbered M.

[0030] Furthermore, the secondary confirmation detection of foreign matter described in step 3 includes:

[0031] According to the preliminary scale and position information rect_gmmM obtained in step 2, the image of the foreign body area is extracted by zooming in according to the preset scale factor, and the extracted image is processed using the K-Means image segmentation method. Then, the image is binarized based on the pixel features of the central area to obtain a binary image. Finally, the binary image is processed using the morphological opening operation to segment the foreground target and complete the secondary confirmation detection of the foreign body.

[0032] Further, the secondary confirmation detection of foreign matter described in step 3 specifically includes:

[0033] Step 3-1, according to the preliminary scale and position information rect_gmmM of the foreign object in the forensic image obtained in step 2-5, the target area range rect_roiM is set, the center of the foreign object is the center of the rectangle, and the width and height of the rectangle are set to n times the width and height of the foreign object respectively;

[0034] Step 3-2, extracting and segmenting the forensic image imgM obtained in step 2-2 according to the target area rect_roiM set in step 3-1 to obtain a segmented area image img_roiM;

[0035] Step 3-3, use the K-Means method to process the segmented region image img_roiM to obtain the segmented image img_kmeansM;

[0036] Step 3-4, obtain the pixel value of the center position of the segmented image img_kmeansM, and set the pixel value of the position with the same pixel value in the image img_kmeansM to 255, and set the pixel value of the position with the same pixel value to 0, thereby generating a binary image img_binaM;

[0037] Step 3-5, use the closing operation to process the binary image img_binaM to generate the detection image img_finalM;

[0038] Step 3-6, detect and locate the foreign object in the detection image img_finalM through the contour search algorithm, and obtain the secondary scale and position information rect_secondM of the foreign object in the forensic image imgM.

[0039] Furthermore, the comparison process of the results of the initial test and the secondary confirmation test described in step 4 specifically includes:

[0040] Step 4-1, calculate the IOU intersection rate of the preliminary scale and position information rect_gmmM obtained in step 2-5 and the secondary scale and position information rect_secondM obtained in step 3-6 iou ;

[0041] Step 4-2, calculate the first area m_area_gmmM of the preliminary scale and position information rect_gmmM and the second area m_area_secondM of the secondary scale and position information rect_secondM, and calculate the area ratio:

[0042] rate area= max_area / min_area

[0043] where max_area and min_area are the maximum and minimum values of the first area m_area_gmmM and the second area m_area_secondM, respectively;

[0044] Step 4-3: Based on the preliminary scale and position information rect_gmmM and the secondary scale and position information rect_secondM, calculate the distance dis between the center points of the foreign objects in the two detection results;

[0045] Step 4-4: Based on the IOU intersection over union rate iou , area ratio rate area and the center point distance dis, perform authenticity discrimination of the foreign object, specifically as follows:

[0046] When rate iou > iouval, rate area < areval, and distance < disval, it is determined that the detection result is true, and the secondary scale and position information rect_secondM is used as the final foreign object detection information rect_finalM of the image numbered M, otherwise it is determined as interference and eliminated, where iouval, areval, and disval are preset thresholds.

[0047] Further, the final reporting of foreign object information described in Step 5 specifically includes:

[0048] Step 5-1: Set the final foreign object detection information of the forensic image numbered M as rect_finalM, set the corresponding quality count m_quality, and set its initial value to 1;

[0049] Step 5-2: During the process of scanning at the preset period, set the new final foreign object detection information of the newly obtained forensic image img_newM numbered M as rect_final_newM;

[0050] Step 5-3: Compare and process the final foreign object detection information rect_finalM in Step 4-4 with the new final foreign object detection information rect_final_newM. The comparison and processing method is the same as the comparison method described in Step 4. If the determination result is true, add 1 to the quality count m_quality, otherwise subtract 1. At the same time, set the corresponding quality count for the final foreign object detection information rect_final_newM as the newly emerged candidate foreign object target;

[0051] Step 5-4: When m_quality < min_quality, foreign object targets are deleted; when m_quality > max_quality, the final foreign object detection information rect_finalM and the image number M where it is located are reported, where min_quality is the preset minimum quality coefficient and max_quality is the preset maximum quality coefficient;

[0052] Step 5-5: When scanning in the next cycle, for newly emerging candidate foreign object targets, process them according to Steps 5-1 to 5-4.

[0053] Furthermore, in Step 4-4, the threshold iouval is set to 0.8, the threshold areval is set to 2, and the threshold disval is set to 20.

[0054] Furthermore, in Step 5-4, the minimum quality coefficient min_quality is set to 0, and the maximum quality coefficient max_quality is set to 2.

[0055] Furthermore, the optical device described in Step 1 is a pan-tilt camera.

[0056] Beneficial effects:

[0057] 1. The present invention solves problems such as narrow coverage, high false alarm rate, and low robustness in runway foreign object detection based on optoelectronic images. At the same time, the present invention is an unsupervised detection mode, does not require long-term accumulation of samples of real foreign objects, and has a higher degree of engineering.

[0058] 2. The present invention has been tested for performance in various scenarios, can correctly detect and locate foreign objects in images in environments such as night and strong light. The average detection time for each image is 10 ms. The foreign object detection system composed of a single pan-tilt camera can cover a 1-km runway area, and can detect foreign object targets as small as 2 cm, fully verifying the effectiveness of the present invention. Description of the drawings

[0059] The following further specifically describes the present invention in conjunction with the drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0060] Figure 1 is the system flow schematic diagram of the present invention.

[0061] Figure 2 is the initial foreign object detection effect diagram of the present invention.

[0062] Figure 3 is the secondary foreign object detection effect diagram of the present invention.

[0063] Figure 4 It is a foreign matter comparison effect diagram of the present invention.

[0064] Figure 5 It is a flow chart of the multi-frame accumulation principle of the present invention. DETAILED DESCRIPTION

[0065] The present invention proposes an optical airport runway foreign body detection method, comprising the following steps:

[0066] Step 1: Set the scanning preset position and optical zoom value of the PTZ camera, then scan back the runway area for evidence collection according to the set conditions, number the evidence pictures according to the preset positions, and manually delineate the runway area based on the images;

[0067] Step 2: For each numbered runway image without foreign objects, a background model is established. Using the Gaussian mixture model (Wang Rongrong, Wu Jingjing, Weng Chenyi. Moving target segmentation algorithm based on improved Gaussian mixture model [J]. Light Machinery Engineering, 2022, (05): 34-42)), the new image obtained by the retracement is differentially compared with the corresponding background model, and then the foreground target is segmented based on the designated runway area to achieve the initial detection of foreign objects;

[0068] Step 3, based on the location information and size information of the foreign objects initially detected in each numbered image, the image of the corresponding area is extracted after being magnified according to a certain proportional coefficient, and the extracted image is processed using the K-Means image segmentation algorithm (Wang Liying, Li Jiaying, Yuan Yue. Improved K-means clustering algorithm and its application in image segmentation [J]. China Information Industry, 2024(06):252-256), and then the image is binarized based on the pixel features of the central area position (Liu Pengyuan, Tian Rui, Zhou Yuanfeng, et al. Binarization of smart meter images based on adaptive hybrid threshold [J]. Computer Applications and Software, 2023(01):210-215), and finally the binary image is processed using the morphological opening operation (Lu Zongqi, Zhu Yu. Fast algorithm for digital morphological erosion and dilation operation [J]. National Image and Graphics Academic Conference, 2006(13):306-311), the foreground target is segmented, and the secondary confirmation detection of foreign objects is realized;

[0069] Step 4, compare and process the target information detected twice in each numbered image, eliminate interference, and obtain real foreign body information;

[0070] Step 5: After multiple cycles of accumulation and correlation confirmation of the real foreign object information obtained in each numbered image, the foreign object information that meets the conditions is reported, and the targets that do not meet the conditions are deleted to eliminate interference;

[0071] Step 6: The PTZ camera performs a backscan to collect evidence according to the preset position, numbers each image, and processes it according to the above process to achieve accurate detection and reporting of foreign objects in the covered runway area.

[0072] Step 1 includes:

[0073] Step 1-1, use differential GPS to measure the positions of the PTZ camera and the two ends of the runway area to be covered, where the two ends of the runway are respectively the bottom edge of the left runway and the top edge of the right runway, and obtain the position information of the three, where the latitude and longitude coordinate information of the camera is (la cammer ,lon cammer ), the two ends of the runway are (la left ,lon left ) and (la right ,lon right ), la represents latitude information, and lon represents longitude information.

[0074] Step 1-2, for the three coordinate information obtained in step 1-1, calculate the horizontal azimuth angle α between the two ends of the runway and the camera. The specific method is as follows:

[0075]

[0076] Step 1-3, aim the camera at the farthest runway area, adjust the camera parameters, select a suitable magnification value Z to achieve clear observation of the runway target, and based on the Z value, obtain the camera's horizontal field of view angle p according to the factory settings of the PTZ camera;

[0077] Step 1-4, based on the horizontal field of view angle p set in step 1-3, calculate the required number of preset positions. The specific method is as follows:

[0078]

[0079] The Ceil function represents rounding up, and N represents the calculated rotation preset position of the PTZ camera required to cover the selected runway;

[0080] Step 1-5, rotate the PTZ camera to the edge of the runway area to be covered. According to the multiple Z set in step 1-3, each horizontal rotation angle p is a forensic preset position, and rotate N times in total. During the retrace forensics, manually fine-tune the pitch angle of each preset position so that the runway is in the best position in the image.

[0081] Step 1-6, numbering the images obtained from the N preset positions set in step 1-5, wherein the set number value range is 1-N, the numbered images are img1-imgN, and the numbers correspond to the preset positions one by one.

[0082] Step 2 includes:

[0083] Step 2-1, using the picture without foreign objects for each number to establish the background model of the runway area corresponding to the number, the background model is Back1-BackN;

[0084] Step 2-2, when the PTZ camera scans back according to the preset position set in step 1-5, when scanning back to position M, the latest current image imgM is obtained, where imgM represents the image numbered M, where 1 = <M<=N;

[0085] Step 2-3, using the Gaussian mixture model (Wang Rongrong, Wu Jingjing, Weng Chenyi. Moving target segmentation algorithm based on improved Gaussian mixture model [J]. Light Machinery Engineering, 2022, (05): 34-42)), perform difference processing between imgM and the corresponding background model BackN to obtain the foreground image FroundN;

[0086] Step 2-4, using median filtering (Zhou Yuye, Yuan Hongwu, Feng Rui, et al. Research on fast enhancement method of foggy polarization image based on median filtering and polarization modulation [J]. Optical Technology, 2024(01): 106-111) to reduce the noise of the foreground image FroundN to obtain the image BlurFroundN;

[0087] In step 2-5, the image BlurFroundN is processed by closing operation (Lu Zongqi, Zhu Yu. Fast algorithm for digital morphology erosion and dilation operation [J]. National Image and Graphics Academic Conference, 2006(13):306-311) to obtain the image FinalN, and then the image is processed by contour search algorithm (Kang Jie, Ma Zhen. Crop positioning point extraction algorithm based on contour search [J]. Science Technology and Engineering, 2018,18(30):105-111), and based on the runway area artificially demarcated in step 1-6, the initial detection of foreign objects is realized, and the preliminary scale and position information rect_gmmM of the foreign objects in the image number M is obtained.

[0088] Step 3 includes:

[0089] Step 3-1, according to the preliminary scale and position information rect_M of the foreign body in the image obtained in step 2-5, set the target area range rect_roiM, take the center position of the foreign body as the center of the rectangle, and set the width and height of the rectangle to n times the width and height of the foreign body respectively;

[0090] Step 3-2, extracting and segmenting the image imgM obtained by the backscanning forensics in step 2-2 according to the target area rect_roiM set in step 3-1 to obtain a segmented area image img_roiM;

[0091] Step 3-3, use the K-Means algorithm (Wang Liying, Li Jiaying, Yuan Yue. Improved K-means clustering algorithm and its application in image segmentation [J]. China Information Industry, 2024(06): 252-256) to process the segmented image img_roiM obtained in step 3-2 to obtain the segmented image img_kmeansM;

[0092] Step 3-4, obtain the pixel value of the center position of the segmented image img_kmeansM in step 3-3, and set the pixel value of the position with the same pixel value in the image img_kmeansM to 255, and set the pixel value of the position with the same pixel value to 0, thereby generating a binary image img_binaM;

[0093] Step 3-5, use the closing operation (Lu Zongqi, Zhu Yu. Fast algorithm of digital morphology erosion and dilation operation [J]. National Image and Graphics Academic Conference, 2006 (13): 306-311) to process the binary image img_binaM obtained in step 3-4, remove interference, and connect the broken parts to generate the detection image img_finalM.

[0094] Step 4 includes:

[0095] Step 4-1, calculate the IOU intersection rate of the two detected targets of the preliminary scale and position information rect_gmmM obtained in step 2-5 and the secondary scale and position information rect_secondM obtained in step 3-6 iou ;

[0096] Step 4-2, calculate the area m_area_gmmM of the target of the preliminary scale and position information rect_gmmM obtained in step 2-5, and calculate the area m_area_secondM of the secondary scale and position information rect_secondM obtained in step 3-6, and calculate the area ratio rate area =max_area / min_area, where max_area is the maximum value of m_area_gmmM and m_area_secondM, and min_area is the minimum value;

[0097] Step 4-3, based on the preliminary scale and position information rect_gmmM obtained in step 2-5 and the secondary scale and position information rect_secondM obtained in step 3-6, calculate the distance dis between the center points of the foreign body targets detected twice;

[0098] Step 4-4, based on the IOU intersection-over-union ratio calculated in step 4-1 iou and the area ratio calculated in step 4-2 areaAnd based on the center point distance dis calculated in step 4-3, the authenticity of the foreign object is determined. When rate iou >iouval, rate area <areval, distance < disval, it is determined that the detection result meets the requirements, and the final foreign object detection information rect_finalM of the image numbered M is obtained. Otherwise, it is determined as interference and excluded. Generally, iouval takes a value of 0.8, areval takes a value of 2, and disval takes a value of 20.

[0099] Step 5 includes:

[0100] Step 5-1, taking the final foreign object detection information rect_finalM of the image numbered M in step 4-4 as an example for the cumulative reporting of foreign objects in each numbered image, a corresponding quality count m_quality is added to rect_finalM, and its initial value is set to 1;

[0101] Step 5-2, during the rescan process, if a new image img_newM numbered M is obtained again, after processing img_newM through steps 2, 3, and 4, new foreign object detection information rect_final_newM is obtained;

[0102] Step 5-3, comparing the foreign object detection information rect_finalM in step 4-4 with the latest foreign object detection information rect_final_newM in step 5-2, calculating the distance dis_targetM of its target center position, and at the same time calculating the proportionality coefficient rate wide 、rate height , where rate wide = maxwide / minwide, rate height = maxheight / minheight, maxwide is the maximum value of the widths between the two, minwide is the minimum width value, maxheight is the maximum value of the heights between the two, minheight is the minimum value. When dis_targetM < dis_thre, rate wide <rate_wideval, rate height <rate_heightval, at this time, the quality count m_quality added in step 5-1 is incremented by 1. If not satisfied, it is decremented by 1. At the same time, a quality count is added to rect_final_newM according to what is described in step 5-1 as a newly emerging candidate foreign object target, where dis_thre, rate_wideval, and rate_heightval are preset thresholds;

[0103] Step 5-4: Obtain the value of the quality count m_quality added in Step 5-1. When m_quality < min_quality, delete the foreign object target. When m_quality > max_quality, report the foreign object target information and the image number where it is located, where min_quality is the preset minimum quality coefficient and max_quality is the preset maximum quality coefficient;

[0104] Step 5-6: When the pan-tilt camera performs a backscan, process the newly obtained foreign object information based on the number of the image where it is located according to Steps 5-1, 5-2, 5-3, and 5-4, and report the foreign object information that meets the conditions.

[0105] Embodiment:

[0106] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0107] The present invention discloses an optical foreign object detection method for airport runways, as Figure 1 shown, including the following steps:

[0108] S1: Perform backscan picture evidence collection and numbering on the airport runway area:

[0109] Performing backscan picture evidence collection and numbering on the airport runway area is a prerequisite for optical foreign object detection. Only by completing the coverage backscan of the runway area by the cloud and numbering the acquired pictures can the normal detection of foreign objects in each subsequent area be ensured. First, use differential GPS to measure the coordinate information of the positions of the pan-tilt camera and both ends of the runway area to be covered, and calculate the horizontal azimuth angles between both ends of the runway and the camera. The core formula is:

[0110]

[0111] where (la cammer , lon cammer ) is the longitude and latitude coordinate information of the camera, (la left , lon left ) and (la right , lon right ) are the longitude and latitude coordinate information of both ends of the runway area to be covered, la represents latitude information, lon represents longitude information, and α is the horizontal azimuth angle. Then select an appropriate pan-tilt camera zoom value and horizontal field of view angle, and solve the number of preset positions. The core formula is:

[0112]

[0113] Where p is the horizontal field of view of the camera, N is the number of preset positions, and Ceil represents rounding up. Then the PTZ camera is rotated to the edge of the required runway coverage area. Each horizontal rotation angle p is a forensic preset position, and a total of N rotations are made. During the retrace forensics, the pitch angle of each preset position is manually fine-tuned so that the runway is in the best position in the image. The images are numbered from 1 to N, and the runway area is manually delineated based on the observation of each image.

[0114] S2: Initial detection of foreign matter:

[0115] The initial detection of foreign objects is the first step in finding foreign objects in each forensic image. First, use each numbered picture without foreign objects to establish the background model of the runway area corresponding to the number. When the PTZ camera retraces a new cycle, the new picture is obtained, and the Gaussian mixture model is used to perform differential processing with the background model corresponding to the number to obtain the foreground image; the foreground image is denoised using median filtering; the interference target is removed using closed operations and the broken parts are connected; finally, the contour search is used to achieve the initial detection and positioning of the foreign object target based on the designated runway area. The detection effect is as follows: Figure 2 shown.

[0116] S3: Secondary detection of foreign matter:

[0117] Secondary detection of foreign objects is an enhancement of foreign object detection. For each numbered image, based on the first foreign object detection information corresponding to each image, the corresponding image area is extracted by enlarging the width and height in proportion to the size of the foreign object detected for the first time, with the detection target center as the center, and using K-Means to perform cluster segmentation on the extracted image. Then, the image is binarized with the center position pixel after segmentation as the reference value, and the binary image is processed using a closed operation. Finally, the secondary detection and positioning of foreign objects in each numbered image is achieved through contour search. The detection effect is as follows: Figure 3 shown.

[0118] S4: Foreign matter comparison and confirmation:

[0119] Each numbered image is compared with the target information obtained from the two foreign object detections, mainly through the comparison of IOU, area ratio, and center position distance to confirm whether the detected target is a real foreign object or interference information. The final detection and positioning of foreign objects in the airport runway image is achieved. The specific detection process and results are as follows Figure 4 shown.

[0120] S5: Reporting of foreign matter accumulation:

[0121] Foreign matter accumulation and reporting is an important means to reduce the false detection rate of optical foreign matter detection. It mainly adds a mass count to the foreign matter target detected in each numbered image. During the retrace process, when the detection result of the new numbered image is obtained again, it is determined whether it is the same foreign matter based on the proportional coefficient of the center distance, length and width of the two foreign matters before and after. If they are the same, the mass count is increased by 1. If they are different, the mass count of the old foreign matter is reduced by 1, and the above operation is performed on the new foreign matter. When the mass count of the foreign matter is greater than the set threshold, it is reported, and when it is less than the set threshold, it is deleted. The specific process is as follows Figure 5 shown.

[0122] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, the invention content of an optical airport runway foreign body detection method provided by the present invention and some or all of the steps in each embodiment can be executed. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0123] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on such an understanding, the technical solutions in the embodiments of the present invention can be essentially or partly contributed to the prior art in the form of computer programs, i.e., software products, which can be stored in a storage medium and include several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, an MCU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0124] The present invention provides a method and idea for optical airport runway foreign body detection. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. An optical airport runway foreign body detection method, characterized in that: The steps include: Step 1, setting the parameters of the optical equipment, scanning the airport runway area for evidence collection, and obtaining the initial evidence collection image; Step 2, establish a background model and complete the initial detection of foreign objects on the runway; Step 3: Based on the results of the initial test, a secondary confirmation test for foreign matter is performed; Step 4: Compare the results of the initial test and the secondary confirmation test, eliminate interference, and obtain the final foreign body detection information; Step 5, use the optical device to scan the airport runway area according to a preset period, and perform initial monitoring and secondary confirmation monitoring, and finally report foreign object information according to preset conditions to complete the optical airport runway foreign object detection.

2. The optical airport runway foreign body detection method according to claim 1, characterized in that: The scanning and evidence collection of the airport runway area described in step 1 specifically includes: Step 1-1, measuring the two end positions of the optical device and the airport runway area; wherein the two end positions of the airport runway area are respectively taken as the lower edge position of the left end of the runway area and the upper edge position of the right end of the runway area, and the longitude and latitude coordinates of the optical device are calculated as (la cammer ,lon cammer ), the latitude and longitude coordinates of the two end positions of the airport runway area are (la left ,lon left ) and (la right ,lon right ), la represents latitude information, lon represents longitude information; Step 1-2, for the three longitude and latitude coordinates obtained in step 1-1, calculate the horizontal azimuth angle α between the two ends of the runway area and the optical device. The specific method is as follows: Step 1-3, aim the optical device at the farthest runway area, select a magnification value Z to complete clear observation of the runway area, and obtain the horizontal field of view angle p of the optical device based on the magnification value Z; Step 1-4, based on the horizontal field of view angle p obtained in step 1-3, calculate the number of forensic preset positions N of the optical device. The specific method is as follows: Among them, Ceil represents the upward rounding function; Step 1-5, rotate the optical device to the edge of one end of the runway area, according to the multiple value Z set in step 1-3, each horizontal rotation angle p is used as a forensic preset position, and the shooting positions are recorded as 1, 2, ..., N, rotate N times and shoot forensic images; Step 1-6, numbering the evidence images taken in step 1-5, wherein the numbers correspond to the evidence preset position numbers one by one, and the evidence images are recorded as img1 to imgN, and the runway area is manually delineated in the evidence images.

3. The optical airport runway foreign body detection method according to claim 2, characterized in that: The establishment of the background model and the completion of the initial detection of foreign objects on the runway described in step 2 specifically include: Step 2-1, under the condition that there is no foreign matter in the runway area, take a group of forensic images without foreign matter, recorded as Back1, Back2, ..., BackN; Step 2-2, the optical device captures a forensic image according to the method described in step 1-5. When the optical device rotates to the capturing position M, a forensic image imgM is captured, where 1<=M<=N; Step 2-3, using the Gaussian mixture model, perform differential processing on imgM and the background model BackM corresponding to the shooting position to obtain the foreground image FroundM; Step 2-4, using median filtering to perform noise reduction processing on the foreground image FroundM to obtain a noise reduction image BlurFroundM; Step 2-5, perform a closing operation on the denoised image BlurFroundM to obtain a closed image FinalM, and then complete the initial detection of foreign objects on the runway through outer contour search and based on the runway area delineated in step 1-6, and obtain the preliminary scale and position information rect_gmmM of the foreign objects in the forensic image imgM numbered M.

4. The optical airport runway foreign body detection method according to claim 3, characterized in that: The second confirmation test for foreign matter described in step 3 includes: According to the preliminary scale and position information rect_gmmM obtained in step 2, the image of the foreign body area is extracted by zooming in according to the preset scale factor, and the extracted image is processed using the K-Means image segmentation method. Then, the image is binarized based on the pixel features of the central area to obtain a binary image. Finally, the binary image is processed using the morphological opening operation to segment the foreground target and complete the secondary confirmation detection of the foreign body.

5. The optical airport runway foreign body detection method according to claim 4, characterized in that: The secondary confirmation test for foreign matter described in step 3 specifically includes: Step 3-1, according to the preliminary scale and position information rect_gmmM of the foreign object in the forensic image obtained in step 2-5, the target area range rect_roiM is set, the center of the foreign object is the center of the rectangle, and the width and height of the rectangle are set to n times the width and height of the foreign object respectively; Step 3-2, extracting and segmenting the forensic image imgM obtained in step 2-2 according to the target area rect_roiM set in step 3-1 to obtain a segmented area image img_roiM; Step 3-3, use the K-Means method to process the segmented region image img_roiM to obtain the segmented image img_kmeansM; Step 3-4, obtain the pixel value of the center position of the segmented image img_kmeansM, and set the pixel value of the position with the same pixel value in the image img_kmeansM to 255, and set the pixel value of the position with the same pixel value to 0, thereby generating a binary image img_binaM; Step 3-5, use the closing operation to process the binary image img_binaM to generate the detection image img_finalM; Step 3-6, detect and locate the foreign object in the detection image img_finalM through the contour search algorithm, and obtain the secondary scale and position information rect_secondM of the foreign object in the forensic image imgM.

6. The optical airport runway foreign body detection method according to claim 5, characterized in that: The comparison of the results of the initial test and the secondary confirmation test described in step 4 specifically includes: Step 4-1, calculate the IOU intersection rate of the preliminary scale and position information rect_gmmM obtained in step 2-5 and the secondary scale and position information rect_secondM obtained in step 3-6 iou ; Step 4-2, calculate the first area m_area_gmmM of the preliminary scale and position information rect_gmmM and the second area m_area_secondM of the secondary scale and position information rect_secondM, and calculate the area ratio: rate area =max_area / min_area Among them, max_area and min_area are the maximum and minimum values ​​of the first area m_area_gmmM and the second area m_area_secondM respectively; Step 4-3, based on the preliminary scale and position information rect_gmmM and the secondary scale and position information rect_secondM, calculate the center point distance dis of the foreign body in the two detection results; Step 4-4, based on IOU intersection-over-union ratio iou , area ratio area The distance from the center point dis is used to determine the authenticity of the foreign body, as follows: When rate iou > iouval, rate area < areval, distance < disval, the detection result is determined to be true, and the secondary scale and position information rect_secondM are used as the final foreign object detection information rect_finalM of the image numbered M. Otherwise, it is determined as interference and eliminated. Among them, iouval, areval, and disval are preset thresholds.

7. The optical airport runway foreign body detection method according to claim 6, characterized in that: The foreign body information is finally reported according to the preset conditions as described in step 5, specifically including: Step 5-1, set the final foreign body detection information of the forensic image numbered M to rect_finalM, set the corresponding quality count m_quality, and set its initial value to 1; Step 5-2, during the process of scanning at a preset period, assume that the new final foreign object detection information of the newly obtained forensic image img_newM numbered M is rect_final_newM; Step 5-3, compare and process the final foreign object detection information rect_finalM in Step 4-4 with the new final foreign object detection information rect_final_newM. The comparison and processing method is the same as the comparison method described in Step 4. If the judgment result is true, increment the quality count m_quality by 1; otherwise, decrement it by 1. At the same time, set the corresponding quality count for the final foreign object detection information rect_final_newM as the newly emerged candidate foreign object target; Step 5-4, when m_quality < min_quality, delete the foreign object target. When m_quality > max_quality, report the final foreign object detection information rect_finalM and the image number M where it is located, where min_quality is the preset minimum quality coefficient and max_quality is the preset maximum quality coefficient; Step 5-5, during the next cycle of scanning, process the newly emerged candidate foreign object targets according to Steps 5-1 to 5-4.

8. The optical airport runway foreign body detection method according to claim 3, characterized in that: The threshold value iouval described in Step 4-4 is 0.8, the threshold value areval is 2, and the threshold value disval is 20.

9. The optical airport runway foreign body detection method according to claim 4, characterized in that: The minimum quality coefficient min_quality described in Step 5-4 is 0, and the maximum quality coefficient max_quality is 2.

10. The optical airport runway foreign body detection method according to claim 1, characterized in that: The optical device described in Step 1 is a pan-tilt camera.

Images