An airport runway intrusion prevention monitoring method and system

Abstract

This application provides a method and system for monitoring intrusion prevention on airport runways. The method includes the following steps: acquiring multiple infrared images of the airport runway; detecting first target information in each infrared image using an inter-frame difference algorithm and second target information in each infrared image using a saliency algorithm, and merging the first and second target information and sending them to a backend server; stitching the multiple infrared images into a panoramic view and encoding the video to form a video stream, then sending it to the backend server; and comparing the first and second target information with legitimate targets using the backend server to monitor for intrusion. The method involves capturing multiple infrared images of the airport runway and stitching them together to create a panoramic view. The inter-frame difference algorithm and saliency algorithm are used to identify large and small targets in each infrared image, allowing them to complement each other and improving monitoring accuracy. Furthermore, this approach also improves the data processing efficiency of the backend server.

Description

Technical Field

[0001] This application relates to the field of security monitoring technology, and in particular to a method and system for monitoring intrusion prevention on airport runways. Background Technology

[0002] Airport runways, as the core area of ​​air transport, have extremely high security requirements, necessitating real-time monitoring and early warning of intruders (such as personnel, vehicles, drones, etc.). However, existing technologies have the following drawbacks:

[0003] (1) Poor environmental adaptability: The runway scene faces complex environments such as low light at night, fog, and strong light. Traditional 8-bit images are prone to losing target details (such as the small temperature difference between the intruder and the background in infrared images), resulting in a high rate of missed detection of moving targets. (2) High false alarm rate: Existing systems often misjudge legally taxiing aircraft as intruders, or generate false alarms due to interference from birds, changes in light and shadow, etc., which increases the burden on airport control. Summary of the Invention

[0004] The purpose of this invention is to provide an airport runway intrusion prevention monitoring method and system to improve the monitoring effect.

[0005] This application provides a method for airport runway intrusion prevention monitoring, which includes the following steps: Simultaneously acquire multiple infrared images of the airport runway; and the multiple infrared images cover different areas of the airport runway. The first target information in each infrared image is detected by the inter-frame difference algorithm, and the second target information in each infrared image is detected by the saliency algorithm. The first target information and the second target information detected by the inter-frame difference algorithm and the saliency algorithm are combined and sent to the backend server. The first target information is the information of objects other than aircraft. The second target information is the overall or partial information of aircraft. The collected multi-channel infrared images are stitched together in a panoramic view and then encoded into a video stream before being sent to the backend server. The backend server compares the first and second target information with legitimate targets and monitors for intrusion based on the comparison results.

[0006] In the above technical solution, multiple infrared images of the airport runway are captured, and a panoramic image is created by stitching these images together. Inter-frame difference and saliency algorithms are used to identify large and small targets in each infrared image, allowing them to complement each other and improving monitoring accuracy. Furthermore, this approach also improves the data processing efficiency of the backend server.

[0007] In one specific implementation scheme, the simultaneous acquisition of multiple infrared images of the airport runway; and the multiple infrared images covering different areas of the airport runway, specifically includes: Multiple infrared images of the airport runway are acquired by deploying multiple infrared cameras around the runway. The multiple infrared cameras are arranged in a fan shape and cover the runway area. The infrared images acquired by the infrared cameras are 16-bit infrared images.

[0008] In a specific implementation scheme, the step of detecting first target information in each infrared image using an inter-frame difference algorithm, detecting second target information in each infrared image using a saliency algorithm, and merging the first and second target information detected by the inter-frame difference algorithm and the saliency algorithm and sending them to the backend server specifically includes: The airport runway area is preset as the region of interest; The first target information in the region of interest of each infrared image is detected by the inter-frame difference algorithm, and the second target information in the region of interest of each infrared image is detected by the saliency algorithm; wherein the first target information and the second target information both include the target position, target width and height, target trajectory and acquisition timestamp.

[0009] In one specific implementation scheme, the step of performing panoramic stitching and video encoding on the acquired multi-channel infrared images before sending them to the backend server specifically includes: The 16-bit infrared image is converted into an 8-bit infrared image through dynamic range compression, and the image is enhanced by a digital enhancement algorithm during the conversion. Eight-bit infrared images are stitched together to form a panoramic image and a video stream is generated through hardware encoding and sent to the backend server. The timestamps of the first and second target information in each frame of the eight-bit infrared image are the same as the encoded timestamp of that frame.

[0010] In a specific implementation scheme, the step of comparing the first target information and the second target information with legitimate targets through the backend server, and monitoring for intrusion based on the comparison results, specifically includes: Obtain the decoded video stream from the backend server; The decoded image is used to detect targets by training a YOLOv8 model to confirm legitimate targets; The first target information and the second target information are compared with the legitimate target. If either the first target information or the second target information does not correspond to the legitimate target, an alarm is issued.

[0011] In one specific implementation scheme, obtaining the decoded video stream through a backend server specifically includes: Decode the video stream and restore it to an 8-bit image sequence; The first and second target information detected by the inter-frame difference algorithm and the saliency algorithm are stored in the target information cache queue in the backend server.

[0012] In a specific implementation scheme, comparing the first target information and the second target information with legitimate targets, and issuing an alarm when either the first target information or the second target information does not correspond to the legitimate target, specifically includes: For each frame's decoding timestamp, search for the corresponding first or second target information in the target information cache queue; Check whether the coordinates of the moving target in each first target information and second target information are within a set area, wherein the set area is the target range output by the YOLOv8 model; If the moving target is located within the defined area, it is determined to be a legitimate target and filtered; otherwise, it is determined to be an intrusion target and an alarm is triggered.

[0013] Secondly, an airport runway intrusion prevention monitoring system is provided, the system comprising: Data acquisition module: synchronously acquires multiple infrared images of the airport runway; and the multiple infrared images cover different areas of the airport runway; Front-end data processing module: Detects first target information in each infrared image using an inter-frame difference algorithm and second target information using a saliency algorithm. The first and second target information detected by the inter-frame difference and saliency algorithms are merged and sent to the back-end server. The acquired multi-channel infrared images are then stitched together in a panoramic fashion and encoded into a video stream before being sent to the back-end server. The first target information refers to information about objects other than the aircraft; the second target information refers to overall or partial information about the aircraft. The backend server compares the first target information and the second target information with the legitimate target, and monitors whether there is an intrusion based on the comparison results.

[0014] In one specific implementation scheme, the data acquisition module includes: Multiple infrared cameras are arranged around the airport runway; wherein the multiple infrared cameras are arranged in a fan shape and cover the airport runway area, and the infrared images captured by the infrared cameras are 16-bit infrared images.

[0015] In a specific implementation scheme, the backend server is further configured to acquire the decoded video stream; perform target detection on the decoded image using a trained YOLOv8 model to confirm legitimate targets; compare the first target information and the second target information with the legitimate targets, and issue an alarm when either the first target information or the second target information does not correspond to the legitimate target.

[0016] In a specific implementation scheme, the backend server is further configured to, for each frame's decoding timestamp, search for the corresponding first target information or second target information in the target information cache queue; check whether the coordinates of the moving target in each first target information or second target information are within a set area, wherein the set area is the target range output by the YOLOv8 model; if the moving target is within the set area, it is determined to be a legitimate target and filtered; otherwise, it is determined to be an intrusion target and an alarm is triggered.

[0017] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above possible designs.

[0018] Fourthly, embodiments of this application also provide a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to execute any of the above-described possible designs.

[0019] Fifthly, embodiments of this application also provide a computer program, including instructions, which, when run on a computer, cause the computer to perform any of the above-described possible designs. Attached Figure Description

[0020] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments provided according to this disclosure and should not be construed as limiting the scope of this disclosure.

[0021] Figure 1 A flowchart of an airport runway intrusion prevention monitoring method provided in an embodiment of this application; Figure 2 A logic diagram of front-end embedded device processing provided in the embodiments of this application; Figure 3 A flowchart illustrating the processing of the backend server provided in an embodiment of this application; Figure 4 A structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the application will now be described in further detail with reference to the accompanying drawings.

[0023] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this specification should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar words used in one or more embodiments of this specification do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0024] To facilitate understanding of the airport runway intrusion prevention monitoring method provided in this application embodiment, its application scenario is first described. The airport runway intrusion prevention monitoring method provided in this application embodiment is used to monitor intruders on airport runways. Current monitoring methods have poor environmental adaptability. For example, runway scenarios face complex environments such as low light at night, fog, and strong light. Traditional 8-bit images easily lose target details (such as the small temperature difference between intruders and the background in infrared images), resulting in a high false alarm rate for moving target detection. In addition, current monitoring methods have a high false alarm rate. Existing systems often misidentify legally taxiing aircraft as intruders, or generate false alarms due to interference from birds, changes in light and shadow, etc., increasing the burden on airport control. Therefore, this application embodiment provides an airport runway intrusion prevention monitoring method to improve monitoring effectiveness. The following is a detailed description of the method in conjunction with specific accompanying drawings and embodiments.

[0025] like Figure 1 As shown, Figure 1 A flowchart illustrating an airport runway intrusion prevention monitoring method provided in an embodiment of this application is shown. This application provides an airport runway intrusion prevention monitoring method comprising the following steps: Step 001: Simultaneously acquire multiple infrared images of the airport runway; and the multiple infrared images cover different areas of the airport runway; Specifically, multiple infrared cameras deployed around the airport runway collect multi-channel infrared images of the runway; the multiple infrared cameras are arranged in a fan shape and cover the airport runway area, and the infrared images collected by the infrared cameras are 16-bit infrared images.

[0026] In specific setups, infrared cameras are deployed around the airport runway, such as 4-12 16-bit infrared cameras arranged in a fan shape, with their field of view covering the runway area to ensure that the entire airport runway can be monitored.

[0027] The 16-bit infrared images acquired by multiple infrared cameras are obtained in real time. During the acquisition process, high-speed acquisition technology is used to acquire 16-bit images (with grayscale levels of 0-65535) from multiple (4-12) infrared cameras in parallel in real time (e.g., at a frame rate of 25Hz). Each frame is timestamped during acquisition. Using this method, the 16-bit grayscale level can preserve subtle temperature differences between the target and the background, such as the difference in infrared radiation between an intruder and the runway surface.

[0028] Step 002: Detect the first target information in each infrared image using the inter-frame difference algorithm, and detect the second target information in each infrared image using the saliency algorithm. Then, merge the first target information and the second target information detected by the inter-frame difference algorithm and the saliency algorithm and send them to the backend server. Specifically, the airport runway area is first preset as the region of interest; then, the first target information in the region of interest of each infrared image is detected by the inter-frame difference algorithm, and the second target information in the region of interest of each infrared image is detected by the saliency algorithm; wherein, the first target information and the second target information both include the target position, target width and height, target trajectory and acquisition timestamp.

[0029] In specific processing, a collaborative detection method combining "frame difference + saliency algorithm" is performed based on the 16-bit raw data. Since the computational load of 16-bit data increases exponentially, and the airport runway intrusion prevention system focuses on the runway area, the runway area can be pre-set as the region of interest (ROI). Target detection is then performed only on the ROI, which can greatly reduce the computational load to meet the system's real-time requirements for image processing.

[0030] Both the inter-frame difference and saliency algorithms are lightweight and suitable for moving target detection in embedded front-end devices. The inter-frame difference algorithm handles the detection of large targets (first target information), while the saliency algorithm handles the detection of small targets (second target information). The combination of the two algorithms complements each other, overcoming the shortcomings of single algorithms such as missed detections and insufficient accuracy. The first target information refers to information about objects other than aircraft, such as people and vehicles; the second target information is the overall or partial information of the aircraft.

[0031] The output of the moving target detection algorithm includes information such as target location, target width and height, target trajectory, and acquisition timestamp.

[0032] Step 003: The acquired multi-channel infrared images are stitched together in a panoramic manner and then encoded into a video stream before being sent to the backend server.

[0033] Specifically, the 16-bit infrared image is converted into an 8-bit infrared image through dynamic range compression, and the image is enhanced by a digital enhancement algorithm during the conversion. The 8-bit infrared images are then stitched together to form a panoramic image and a video stream is generated through hardware encoding and sent to the backend server. In this process, the timestamps of the first and second target information of each frame in the 8-bit infrared image are the same as the encoded timestamp of that frame.

[0034] In practice, images captured by multiple infrared cameras need to be stitched together into a panoramic image and encoded into H.265 video for display and storage by the user. Currently, video encoding only supports 8-bit and 10-bit images. To reduce the load on the front-end devices, 8-bit video encoding is chosen. Therefore, 16-bit images need to be converted to 8-bit images through dynamic range compression (e.g., linear mapping to 0-255). Simultaneously, a digital enhancement algorithm (DDE) is applied to enhance the image and improve its display quality. Finally, multiple 8-bit images are stitched together into a panoramic image, and a timestamp is added to the panoramic image.

[0035] During video encoding and transmission, the 8-bit panoramic image is hardware-encoded to generate an H.265 video stream. The front-end detection results and the video stream are synchronously transmitted to the back-end server via Ethernet. The target information timestamp of each frame is exactly the same as the encoded timestamp of that frame, which is used for synchronous matching during target filtering on the back-end server.

[0036] Step 004: The backend server compares the first target information and the second target information with the legitimate target, and monitors whether there is an intrusion based on the comparison results.

[0037] Specifically, the following method will be used: Step 1: Obtain the decoded video stream from the backend server; Specifically, the backend server receives the video stream and moving target detection results transmitted from the frontend, and decodes the video stream to restore it into an 8-bit image sequence.

[0038] Step 2: Perform target detection on the decoded image using the trained YOLOv8 model to confirm legitimate targets; Specifically, a pre-trained YOLOv8 model is used to detect targets in the decoded images. This model has been specifically trained and optimized using an airport scene dataset, and all detected targets are legitimate targets, namely "aircraft". Then, based on the decoding timestamp, the moving target information corresponding to the frame is found in the target information queue, and the moving targets within the aircraft's coordinate range are filtered out (because for the airport, aircraft are not considered intrusion targets and do not require alarms).

[0039] The training dataset contains aircraft samples from different aircraft types (such as Boeing 737, Airbus A320, etc.) and different environments (nighttime, foggy days, etc.).

[0040] Step 3: Compare the first target information and the second target information with the legitimate targets. If either the first target information or the second target information does not correspond to a legitimate target, issue an alarm.

[0041] Specifically, for each frame's decoding timestamp, the corresponding first or second target information is searched in the target information cache queue; the coordinates of the moving target in each first and second target information are checked to see if they are within a set area, where the set area is the target range output by the YOLOv8 model; if the moving target is within the set area, it is determined to be a legitimate target and filtered; otherwise, it is determined to be an intrusion target and an alarm is triggered.

[0042] In practice, since the backend server has sufficient computing power, ROI settings can be ignored, and full-frame detection can be performed to improve detection accuracy. For example, an aircraft on the runway may have a part of its body (such as the wing) outside the runway area. If only the ROI area is detected, there is a risk of missing detections.

[0043] In the above technical solution, a series of combined methods, including 16-bit image acquisition and moving target detection (fusion of frame difference and saliency algorithms), 8-bit image panoramic stitching and video encoding, and ROI limiting the detection range, not only ensure the high-precision detection effect of the front end, but also meet the requirements of lightweight front end and avoid insufficient computing resources.

[0044] As described above, the method disclosed in this application involves capturing multiple infrared images of the airport runway and stitching them together to create a panoramic image. Inter-frame difference and saliency algorithms are used to identify large and small targets in each infrared image, allowing them to complement each other and improving monitoring accuracy. Furthermore, this approach also improves the data processing efficiency of the backend server.

[0045] The method provided in this application differs from the processing approach of general backend filtering systems. It classifies and identifies targets transmitted from the front end, and filters targets of specific categories. This invention is specifically designed for airport runway scenarios, utilizing a self-built airport runway image dataset for dedicated training and optimization of the YOLOv8 model, significantly improving detection accuracy. After full-frame detection, only one type of target is detected: aircraft. This result is used solely to filter moving targets from the front end, filtering by comparing the coordinates of the moving targets, eliminating the need for target classification and avoiding filtering failures due to classification errors. The backend server has sufficient computing power, allowing for the use of more powerful target recognition models in the future.

[0046] Furthermore, existing multi-infrared camera systems are based on a back-end processing architecture: infrared images are sent to a back-end server via video encoding, where image processing (target detection, recognition, etc.) is performed. In this architecture, the back-end server receives an 8-bit image after video encoding. The 8-bit image already has a compressed dynamic range, and the video encoding further reduces image contrast and the target's signal-to-noise ratio, making it unsuitable for detecting weak or small targets. In contrast, the solution disclosed in this application improves the comparison effect by performing target information recognition at the front end and then comparing based on timestamps.

[0047] Combined Figure 2 and Figure 3 As shown, Figure 2 A logic block diagram of the front-end embedded device is shown. Figure 3 An embodiment of the present application illustrates a backend server. The system provided in this embodiment is used to implement the above-described method. The front-end embedded device is used to implement steps 001 to 003 of the above method, while the backend server is used to execute step 004 of the above method.

[0048] Combination Figure 2 As shown, the front-end embedded equipment is deployed around the airport runway, equipped with multiple (4-12) 16-bit infrared cameras arranged in a fan shape, covering the runway area. The front-end embedded equipment includes a 16-bit image acquisition module, a moving target detection module, a 16-bit image enhancement and panoramic stitching module, and a video encoding and transmission module. Specific functions include: Real-time acquisition of 16-bit infrared images from multiple cameras: Based on high-speed acquisition technology, 16-bit images (grayscale 0-65535) from multiple (4-12) infrared cameras are acquired in real time (frame rate 25Hz), and each frame is timestamped during acquisition. The 16-bit grayscale level can preserve subtle temperature differences between the target and the background (such as the difference in infrared radiation between an intruder and the runway ground).

[0049] Moving target detection: Target detection is performed using a combined "frame difference + saliency algorithm" based on 16-bit raw data. Since the computational load increases exponentially with 16-bit data, and the airport runway intrusion prevention system focuses on the runway area, the runway area can be pre-defined as the region of interest (ROI). Target detection is then performed only on the ROI, significantly reducing computational load to meet the system's real-time image processing requirements.

[0050] Both the inter-frame difference and saliency algorithms are lightweight algorithms, suitable for moving target detection in front-end embedded devices. The inter-frame difference algorithm handles the detection of large targets, while the saliency algorithm handles the detection of small targets. The combination of the two complements each other, solving the shortcomings of single algorithms such as missed detections and insufficient accuracy.

[0051] The output of the moving target detection algorithm includes information such as target location, target width and height, target trajectory, and acquisition timestamp.

[0052] 16-bit Image Conversion and Panoramic Stitching: Images captured by multiple infrared cameras need to be stitched into a panoramic image and encoded into H.265 video for display and storage. Currently, video encoding only supports 8-bit and 10-bit images. To reduce the load on the front-end devices, we chose 8-bit video encoding. Therefore, 16-bit images need to be converted to 8-bit images through dynamic range compression (e.g., linear mapping to 0-255). Simultaneously, a digital enhancement algorithm (DDE) is applied to enhance the image and improve its display. Finally, multiple 8-bit images are stitched into a panoramic image, and a timestamp is added to the panoramic image.

[0053] Video Encoding and Transmission: 8-bit panoramic images are hardware-encoded into an H.265 video stream. The front-end detection results and the video stream are synchronously transmitted to the back-end server via Ethernet. The target information timestamp of each frame is identical to the encoded timestamp of that frame, used for synchronous matching during target filtering on the back-end server.

[0054] Combination Figure 3 As shown, the backend server is deployed in the airport data center or server room, equipped with high-performance computing cards. The backend system includes a data receiving and video decoding module, and a target detection and filtering module, with the following specific functions: Data reception and video decoding: Receives the video stream and moving target detection results transmitted from the front end, decodes the video stream to restore it into an 8-bit image sequence, and saves the front-end moving target information to the buffer queue.

[0055] Target detection and filtering: A pre-trained YOLOv8 model is used to detect targets in the decoded image. This model has been specifically trained and optimized using an airport scene dataset, and all detected targets are legitimate targets, namely "aircraft". Then, based on the decoding timestamp, the moving target information corresponding to the frame is found in the target information queue, and the moving targets within the aircraft's coordinate range are filtered out (because for the airport, aircraft are not considered intrusion targets and do not require alarm).

[0056] Training dataset: Contains aircraft samples of different models (Boeing 737, Airbus A320, etc.) and different environments (nighttime, foggy days, etc.).

[0057] Filtering logic: The target "aircraft" output by the YOLOv8 model is processed as follows: Based on the decoding timestamp of the frame, the corresponding front-end moving target information is searched in the target information queue. This moving target information may contain multiple moving targets. The coordinates of each moving target are checked to see if they are within the RectPlane, where the RectPlane is the target range output by the YOLOv8 model (the rectangular area where the aircraft is located). If the moving target is within the RectPlane, it is determined to be a legitimate target and filtered; otherwise, it is determined to be an intrusion target, and an alarm is triggered.

[0058] Since the backend server has sufficient computing power, ROI settings can be ignored, and full-frame detection can be performed to improve detection accuracy. For example, an aircraft on the runway may have a part of its body (such as the wing) outside the runway area. If only the ROI area is detected, there is a risk of missing detections.

[0059] This embodiment provides an airport runway intrusion prevention monitoring system. The system includes: a data acquisition module that simultaneously acquires multiple infrared images of the airport runway, covering different areas of the runway; a front-end data processing module that detects first target information in each infrared image using an inter-frame difference algorithm and second target information in each infrared image using a saliency algorithm, and merges the first and second target information detected by the inter-frame difference and saliency algorithms and sends them to a back-end server; and a system that performs panoramic stitching and video encoding of the acquired multiple infrared images to form a video stream, which is then sent to the back-end server. The back-end server compares the first and second target information with legitimate targets and monitors for intrusion based on the comparison results.

[0060] Among them, multiple infrared cameras are deployed around the airport runway; these multiple infrared cameras are arranged in a fan shape and cover the airport runway area, and the infrared images captured by the infrared cameras are 16-bit infrared images.

[0061] The aforementioned data acquisition module may include a 16-bit image acquisition module, while the front-end data processing module includes a moving target detection module, a 16-bit image enhancement and panoramic stitching module, and a video encoding and transmission module. For details, please refer to the above description; further elaboration will not be repeated here.

[0062] The system provided in this application embodiment also includes a backend server, which is used to acquire the decoded video stream; perform target detection on the decoded image using a trained YOLOv8 model to confirm legitimate targets; compare the first target information and the second target information with the legitimate targets, and issue an alarm when either the first target information or the second target information does not correspond to a legitimate target.

[0063] In a specific implementation scheme, the backend server is further configured to store the first and second target information detected by the inter-frame difference algorithm and the saliency algorithm in a target information cache queue in the backend server; for the decoding timestamp of each frame, search for the corresponding first or second target information in the target information cache queue; check whether the coordinates of the moving target in each first and second target information are within a set area, where the set area is the target range output by the YOLOv8 model; if the moving target is within the set area, it is determined to be a legitimate target and filtered; otherwise, it is determined to be an intrusion target and an alarm is triggered.

[0064] The aforementioned backend server may include a data receiving and video decoding module and a target detection and filtering module. For details, please refer to the above description, which will not be repeated here.

[0065] To facilitate understanding of the methods and systems provided in the embodiments of this application, specific examples are given below for illustration.

[0066] Nighttime airport runway intrusion monitoring: Front-end embedded devices: NVIDIA Jetson AGX Xavier, equipped with eight 16-bit infrared cameras (Gaode PLUG612).

[0067] 16-bit image acquisition module: Simultaneously acquires images from 8 infrared cameras, with a single frame image resolution of 640×512, grayscale level of 0-65535, and an acquisition frequency of 25 frames / second.

[0068] Moving target detection module: Inter-frame difference operation: The interval parameter is 1. If the current frame is the nth frame, then the inter-frame difference operation is performed on the nth frame and the (n-2)th frame to obtain the motion region and detect large targets (greater than or equal to 11×11 pixels).

[0069] The saliency algorithm compares the difference between the gray values ​​of each pixel in the 16-bit image and those of its surrounding pixels to calculate the saliency of the pixels. Then, through background modeling and target extraction, it achieves the detection of small targets (less than 11×11 pixels).

[0070] The actual output detection result is the result of merging the two algorithms mentioned above: those with the same or similar coordinates (coordinate deviation less than 5 pixels) are merged, and the others are taken as the union of the detection sets.

[0071] 16-bit image conversion and panoramic stitching module: Calculate the mean and standard deviation of the grayscale values ​​of the 16-bit image, select the grayscale range of the 16-bit image, linearly map (when computing power is sufficient, the digital enhancement algorithm DDE can be used) to 0-255, convert it into an 8-bit image, and then perform panoramic stitching using the pre-calibrated panoramic stitching parameters.

[0072] Video Encoding and Transmission: Utilizing the hardware encoding capabilities of AGX Xavier, 8-bit panoramic images are encoded into H.265 video streams (bitrate 2Mbps). The video streams and front-end moving target information are transmitted separately via TCP connections to the back-end server, without interference.

[0073] Backend server: Kylin Linux Advanced Server V10, configured with Huawei Ascend Atlas300i inference card.

[0074] Data reception and video decoding: The system receives the video stream and moving target detection results transmitted from the front end through two TCP connections. The received video stream is decoded and restored into an 8-bit image sequence. The received moving target information from the front end is saved to the buffer queue.

[0075] Target Detection and Filtering: A pre-trained YOLOv8 model is used to detect targets in the decoded image. The pre-trained model has an input size of 4096×512 and a confidence threshold of 0.9. This model has been specifically optimized for airport scene datasets, and all detected targets are legitimate targets, namely "airplanes". If the pre-trained model detects an airplane, and the target region is RectPlane(Xp, Yp, Wp, Hp), then the corresponding moving target information for that frame is found in the target information queue based on the decoding timestamp. Assuming the moving target information includes two targets: a vehicle and an airplane, with target ranges of Rect1(X1, Y1, W1, H1) and Rect2(X2, Y2, W2, H2) respectively, if the coordinates (X1, Y1) are not within the RectPlane, a vehicle alarm is triggered, while the coordinates (X2, Y2) are within the RectPlane, indicating a legitimate target, the airplane, which is filtered out.

[0076] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above possible designs.

[0077] This application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to execute any of the above-described possible designs.

[0078] This application also provides a computer program 300, including instructions that, when run on a computer, cause the computer to perform any of the above-described possible designs.

[0079] Furthermore, the technical effects of any of the above possible design approaches can be found in [reference needed]. Figure 3The effects of different design approaches in the example section will not be elaborated here.

[0080] Figure 4 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0081] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0082] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system 100 and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0083] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0084] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0085] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0086] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0087] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0088] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this specification, which are not provided in detail for the sake of brevity.

[0089] Additionally, to simplify the description and discussion, and to avoid obscuring one or more embodiments of this specification, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring one or more embodiments of this specification, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which one or more embodiments of this specification will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that one or more embodiments of this specification may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0090] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0091] One or more embodiments in this specification are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this specification should be included within the scope of protection of this disclosure.

Claims

1. A method for monitoring intrusion prevention on airport runways, characterized in that, Includes the following steps: Simultaneously acquire multiple infrared images of the airport runway; and the multiple infrared images cover different areas of the airport runway. The first target information in each infrared image is detected by the inter-frame difference algorithm, and the second target information in each infrared image is detected by the saliency algorithm. The first target information and the second target information detected by the inter-frame difference algorithm and the saliency algorithm are combined and sent to the backend server. The first target information is the information of objects other than aircraft. The second target information is the overall or partial information of aircraft. The collected multi-channel infrared images are stitched together in a panoramic view and then encoded into a video stream before being sent to the backend server. The backend server compares the first and second target information with legitimate targets and monitors for intrusion based on the comparison results.

2. The airport runway intrusion prevention monitoring method according to claim 1, characterized in that, The simultaneous acquisition of multiple infrared images of the airport runway; and the multiple infrared images covering different areas of the airport runway, specifically including: Multiple infrared images of the airport runway are acquired by deploying multiple infrared cameras around the runway. The multiple infrared cameras are arranged in a fan shape and cover the runway area. The infrared images acquired by the infrared cameras are 16-bit infrared images.

3. The airport runway intrusion prevention monitoring method according to claim 2, characterized in that, The process of detecting first target information in each infrared image using an inter-frame difference algorithm and second target information in each infrared image using a saliency algorithm, and then merging the first and second target information detected by the inter-frame difference algorithm and the saliency algorithm and sending them to the backend server, specifically includes: The airport runway area is preset as the region of interest; The first target information in the region of interest (ROI) of each infrared image is detected using an inter-frame difference algorithm, and the second target information in the ROI of each infrared image is detected using a saliency algorithm; wherein... Both the first target information and the second target information include the target location, target width and height, target trajectory, and data collection timestamp.

4. The airport runway intrusion prevention monitoring method according to claim 2, characterized in that, The process of stitching together and encoding the acquired multi-channel infrared images into a panoramic view before sending them to the backend server specifically includes: The 16-bit infrared image is converted into an 8-bit infrared image through dynamic range compression, and the image is enhanced by a digital enhancement algorithm during the conversion. Eight-bit infrared images are stitched together to form a panoramic image and a video stream is generated through hardware encoding and sent to the backend server. The timestamps of the first and second target information in each frame of the eight-bit infrared image are the same as the encoded timestamp of that frame.

5. The airport runway intrusion prevention monitoring method according to any one of claims 1 to 4, characterized in that, The step of comparing the first target information and the second target information with legitimate targets through the backend server, and monitoring for intrusion based on the comparison results, specifically includes: Obtain the decoded video stream from the backend server; The decoded image is used to detect targets by training a YOLOv8 model to confirm legitimate targets; The first target information and the second target information are compared with the legitimate target. If either the first target information or the second target information does not correspond to the legitimate target, an alarm is issued.

6. The airport runway intrusion prevention monitoring method according to claim 5, characterized in that, The process of obtaining the decoded video stream from the backend server specifically includes: Decode the video stream and restore it to an 8-bit image sequence; The first target information and the second target information are stored in the target information cache queue in the backend server.

7. The airport runway intrusion prevention monitoring method according to claim 6, characterized in that, The step of comparing the first target information and the second target information with legitimate targets, and issuing an alarm when either the first target information or the second target information does not correspond to the legitimate target, specifically includes: For each frame's decoding timestamp, search for the corresponding first or second target information in the target information cache queue; Check whether the coordinates of the moving target in each first target information and second target information are within a set area, wherein the set area is the target range output by the YOLOv8 model; If the moving target is located within the defined area, it is determined to be a legitimate target and filtered; otherwise, it is determined to be an intrusion target and an alarm is triggered.

8. An airport runway intrusion prevention monitoring system, characterized in that, include: Data acquisition module: synchronously acquires multiple infrared images of the airport runway; and the multiple infrared images cover different areas of the airport runway; Front-end data processing module: Detects the first target information in each infrared image using the inter-frame difference algorithm and the second target information in each infrared image using the saliency algorithm. Then, it merges the first target information and the second target information detected by the inter-frame difference algorithm and the saliency algorithm and sends them to the back-end server. The collected multi-channel infrared images are stitched together in a panoramic view and then encoded into a video stream before being sent to the backend server; the first target information is information about objects other than the aircraft; the second target information is overall or partial information about the aircraft. The backend server compares the first target information and the second target information with the legitimate target, and monitors whether there is an intrusion based on the comparison results.

9. The airport runway intrusion prevention monitoring system according to claim 8, characterized in that, The backend server is also used to acquire the decoded video stream; perform target detection on the decoded image using a trained YOLOv8 model to confirm legitimate targets; compare the first target information and the second target information with the legitimate targets, and issue an alarm when either the first target information or the second target information does not correspond to the legitimate target.

10. The airport runway intrusion prevention monitoring system according to claim 9, characterized in that, The backend server is also specifically used to search for the corresponding first target information or second target information in the target information cache queue for the decoding timestamp of each frame; check whether the coordinates of the moving target in each first target information and second target information are within a set area, wherein the set area is the target range output by the YOLOv8 model; if the moving target is within the set area, it is determined to be a legal target and filtered; otherwise, it is determined to be an intrusion target and an alarm is triggered.

11. A computer program, characterized in that, The computer program includes instructions that, when run on a computer, cause the computer to perform the method according to any one of claims 1 to 7.

Images