Airport runway intrusion detection system based on deep learning

Abstract

The invention discloses an airport runway intrusion detection system based on deep learning, and is applied to the technical field of target recognition. Comprising a data acquisition module, a data processing module, an airport runway detection module, an intrusion body detection module, a contact point detection module and an intrusion judgment module, wherein the data acquisition module acquires video monitoring data of an airport runway; the data processing module decomposes the video monitoring data into frame images; the airfield runway detection module, the invading body detection module and the contact point detection module are used for detecting the position of an airfield runway, the position of an invading body and the position of a contact point of the invading body and the ground in the image based on the frame image respectively; and the invasion judgment module is used for judging whether an invasion condition exists or not based on a detection result. According to the invention, the intrusion condition of the airport runway is detected, so that the accident occurrence probability of the aircraft in the takeoff process can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of target recognition technology, and more specifically to an airport runway intrusion detection system based on deep learning. Background Art

[0002] Occasionally, there will be intrusion objects that do not belong to the runway surface on the airport runway in the airport operation area, such as batteries, plastic parts, metal plates, etc., and there may be foreign objects that will pose a serious threat to the normal operation of the aircraft. Therefore, the detection of intrusion objects has always been a major challenge in aircraft safety assurance. At present, most airports still use manual inspections to check for foreign objects on the airport runway. However, this method requires a lot of human resources, is costly and inefficient, and cannot be accurately detected after the runway is closed during takeoff and landing. In addition, the manual detection method also affects the personal safety of the staff. Therefore, how to provide an airport runway intrusion detection system based on deep learning is a problem that technicians in this field urgently need to solve. Summary of the invention

[0003] In view of this, the present invention provides an airport runway intrusion detection system based on deep learning, which detects whether there is an intruder and the position of the intruder relative to the airport runway to determine whether there is an intrusion, thereby reducing the probability of accidents.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A deep learning-based airport runway intrusion detection system comprises: a data acquisition module, a data processing module, an airport runway detection module, an intrusion body detection module, a contact point detection module and an intrusion judgment module; wherein the data acquisition module acquires video monitoring data of the airport runway; the data processing module decomposes the video monitoring data into frame images; the airport runway detection module detects the position of the airport runway in an image based on the frame image, the intrusion body detection module detects the position of the intrusion body in an image based on the frame image, the contact point detection module detects the position of the contact point between the intrusion body and the ground in the image based on the frame image, and the intrusion judgment module judges whether there is an intrusion based on the position of the contact point between the intrusion body and the ground, the position of the intrusion body and the position of the airport runway.

[0006] Optionally, the airport runway detection module and the intruder detection module use the same detection model and are improved based on the YOLOv10m model. Specifically, the C2f module in the YOLOv10m model is replaced with a C2f-faster-EMA module, the SPPF module is replaced with a shared convolution module, and the detection network is replaced with a shared detection network.

[0007] Optionally, the C2f-faster-EMA module includes a first CBS unit, a separation layer, a first EMA-faster unit, a second EMA-faster unit, a third EMA-faster unit, a connection layer and a second CBS unit connected in sequence. The connection layer is also connected to the Split layer, the first EMA-faster unit and the second EMA-faster unit respectively. The first CBS unit and the second CBS unit are both composed of a convolutional layer, a batch normalization layer and an activation function connected in sequence.

[0008] Optionally, the shared convolution module includes a first 1×1 convolution layer, a first 3×3 convolution layer, a second 3×3 convolution layer, a third 3×3 convolution layer, a connection unit, and a second 1×1 convolution layer connected in sequence, wherein the first 1×1 convolution layer is also connected to the second 1×1 convolution layer, and the first 3×3 convolution layer and the second 3×3 convolution layer are also connected to the connection unit.

[0009] Optionally, the shared detection network includes three 1×1 shared convolutional layers, two 3×3 shared convolutional layers and three detection heads, wherein the three 1×1 shared convolutional layers are all connected to the first 3×3 shared convolutional layer, the first 3×3 shared convolutional layer is connected to the second 3×3 shared convolutional layer, and the second 3×3 shared convolutional layer is connected to three detection heads.

[0010] Optionally, the training process of the airport runway detection module and the intruder detection module is:

[0011] S1, obtain a video dataset of the airport runway, and decompose the video dataset into frame images;

[0012] S2, classifying and manually annotating the frame images, dividing the frame images into a set of images with intrusion behavior and a set of images without intrusion behavior, and annotating the airport runway and the intrusion object in the frame images;

[0013] S3. Use the image set without intrusion behavior as the training set of the airport runway detection module, and use the image set with intrusion behavior as the training set of the intrusion body detection module to train the airport runway detection module and the intrusion body detection module.

[0014] Optionally, the judgment process of the intrusion judgment module is specifically as follows: if the position of the intruding body does not overlap with the position of the airport runway, it is judged that there is no invasion risk; if the position of the intruding body overlaps with the position of the airport runway but the position of the contact point between the intruding body and the ground is outside the airport runway, it is judged that there is an invasion risk; if the position of the intruding body overlaps with the position of the airport runway and the position of the contact point between the intruding body and the ground is inside the airport runway, it is judged that there is an invasion.

[0015] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides an airport runway intrusion detection system based on deep learning, which has the following beneficial effects:

[0016] 1. The present invention detects intruders through a deep learning model, improves the existing target recognition algorithm, improves the detection efficiency while ensuring the detection accuracy, can quickly and accurately identify intruders on the airport runway, and ensures the operational safety during the airport operation;

[0017] 2. The present invention detects intruders and airport runways separately. Compared with the traditional camera monitoring solution, there is no need to reset the airport runway area after the monitoring angle changes, and there is no need to fix the monitoring points and angles. The airport runway can be monitored in all directions.

[0018] 3. The present invention determines the position of the intruder through the contact point of the intruder and then determines whether there is an intrusion, thereby avoiding false detection due to viewing angle problems and improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0020] Figure 1 It is a schematic diagram of the airport runway intrusion detection system of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] The embodiment of the present invention discloses an airport runway intrusion detection system based on deep learning, such as Figure 1As shown, it includes: a data acquisition module, a data processing module, an airport runway detection module, an intruder detection module, a contact point detection module and an intrusion judgment module; wherein the data acquisition module acquires video monitoring data of the airport runway; the data processing module decomposes the video monitoring data into frame images; the airport runway detection module detects the position of the airport runway in the image based on the frame image, the intruder detection module detects the position of the intruder in the image based on the frame image, the contact point detection module detects the position of the contact point between the intruder and the ground in the image based on the frame image, and the intrusion judgment module judges whether there is an intrusion based on the position of the contact point between the intruder and the ground, the position of the intruder and the position of the airport runway.

[0023] Furthermore, the airport runway detection module and the intruder detection module use the same detection model and are improved based on the YOLOv10m model. Specifically, the C2f module in the YOLOv10m model is replaced by the C2f-faster-EMA module, the SPPF module is replaced by a shared convolution module, and the detection network is replaced by a shared detection network.

[0024] Furthermore, the C2f-faster-EMA module includes a first CBS unit, a separation layer, a first EMA-faster unit, a second EMA-faster unit, a third EMA-faster unit, a connection layer and a second CBS unit which are connected in sequence. The connection layer is also connected to the Split layer, the first EMA-faster unit and the second EMA-faster unit respectively. The first CBS unit and the second CBS unit are both composed of a convolutional layer, a batch normalization layer and an activation function which are connected in sequence.

[0025] In an embodiment of the present invention, the EMA-faster unit includes a partial convolution layer, a CBS unit, a 1×1 convolution layer and an EMA unit connected in sequence. The partial convolution uses 1 / 4 of the input channels for convolution operation, while the remaining channels remain unchanged, which can improve the overall detection efficiency. A 1×1 convolution layer is set after the CBS unit to reduce the number of channels in the output of the CBS unit to the state before its expansion. The C2f-faster-EMA module not only improves the computational efficiency, but also reduces the computational complexity of the original C2f structure.

[0026] Furthermore, the shared convolution module includes a first 1×1 convolution layer, a first 3×3 convolution layer, a second 3×3 convolution layer, a third 3×3 convolution layer, a connection unit, and a second 1×1 convolution layer, which are connected in sequence, wherein the first 1×1 convolution layer is also connected to the second 1×1 convolution layer, and the first 3×3 convolution layer and the second 3×3 convolution layer are also connected to the connection unit.

[0027] In the embodiment of the present invention, the number of trainable parameters is greatly reduced by sharing the convolution module, thereby reducing storage and computing requirements and significantly improving computing efficiency. The two 1×1 convolutional layers can effectively adjust the number of channels and perform feature fusion in the process, which not only retains key feature information but also gives the model greater expressive power while maintaining computing efficiency. Traditional SPPF may cause some details to be lost.

[0028] Furthermore, the shared detection network includes three 1×1 shared convolutional layers, two 3×3 shared convolutional layers and three detection heads, wherein the three 1×1 shared convolutional layers are all connected to the first 3×3 shared convolutional layer, the first 3×3 shared convolutional layer is connected to the second 3×3 shared convolutional layer, and the second 3×3 shared convolutional layer is connected to three detection heads.

[0029] In the embodiment of the present invention, the detection head includes two Conv-Reg units and two Conv-CIs units, each Conv-Reg layer is connected to a Scale unit, and all Conv-Reg units and Conv-CIs units in the three detection heads are connected to the second 3×3 shared convolution layer. By using a shared convolution layer, the shared detection network can ensure that the detection head maintains high accuracy while minimizing the complexity of parameters and calculations.

[0030] Furthermore, the training process of the airport runway detection module and the intruder detection module is as follows:

[0031] S1, obtain a video dataset of the airport runway, and decompose the video dataset into frame images;

[0032] S2, classifying and manually annotating the frame images, dividing the frame images into a set of images with intrusion behavior and a set of images without intrusion behavior, and annotating the airport runway and the intrusion object in the frame images;

[0033] S3. Use the image set without intrusion behavior as the training set of the airport runway detection module, and use the image set with intrusion behavior as the training set of the intrusion body detection module to train the airport runway detection module and the intrusion body detection module.

[0034] In an embodiment of the present invention, under normal circumstances, the midpoint of the bottom edge of the intruder detection frame can be used as its contact point with the ground. However, in some special cases, such as when a person's foot enters an airport runway, judging only by the midpoint of the bottom edge of the detection frame may lead to misjudgment. Therefore, the contact point detection module adopts a key point detection network to identify the key points of the intruder, and uses the key points as contact points to judge the intrusion situation.

[0035] Furthermore, the judgment process of the intrusion judgment module is specifically as follows: if the position of the intruding body does not overlap with the position of the airport runway, it is judged that there is no invasion risk; if the position of the intruding body overlaps with the position of the airport runway but the position of the contact point between the intruding body and the ground is outside the airport runway, it is judged that there is an invasion risk; if the position of the intruding body overlaps with the position of the airport runway and the position of the contact point between the intruding body and the ground is inside the airport runway, it is judged that there is an invasion situation.

[0036] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0037] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An airport runway intrusion detection system based on deep learning, characterized in that: include: A data acquisition module, a data processing module, an airport runway detection module, an intruder detection module, a contact point detection module and an intrusion judgment module; wherein the data acquisition module acquires video monitoring data of the airport runway; the data processing module decomposes the video monitoring data into frame images; the airport runway detection module detects the position of the airport runway in the image based on the frame images, the intruder detection module detects the position of the intruder in the image based on the frame images, the contact point detection module detects the position of the contact point between the intruder and the ground in the image based on the frame images, and the intrusion judgment module judges whether there is an intrusion based on the position of the contact point between the intruder and the ground, the position of the intruder and the position of the airport runway.

2. The airport runway intrusion detection system based on deep learning according to claim 1 is characterized in that: The airport runway detection module and the intruder detection module use the same detection model and are improved based on the YOLOv10m model. Specifically, the C2f module in the YOLOv10m model is replaced by the C2f-faster-EMA module, the SPPF module is replaced by a shared convolution module, and the detection network is replaced by a shared detection network.

3. The airport runway intrusion detection system based on deep learning according to claim 2 is characterized in that: The C2f-faster-EMA module includes a first CBS unit, a separation layer, a first EMA-faster unit, a second EMA-faster unit, a third EMA-faster unit, a connection layer and a second CBS unit connected in sequence. The connection layer is also connected to the Split layer, the first EMA-faster unit and the second EMA-faster unit respectively. The first CBS unit and the second CBS unit are both composed of a convolutional layer, a batch normalization layer and an activation function connected in sequence.

4. The airport runway intrusion detection system based on deep learning according to claim 2 is characterized in that: The shared convolution module includes a first 1×1 convolution layer, a first 3×3 convolution layer, a second 3×3 convolution layer, a third 3×3 convolution layer, a connection unit, and a second 1×1 convolution layer, which are connected in sequence, wherein the first 1×1 convolution layer is also connected to the second 1×1 convolution layer, and the first 3×3 convolution layer and the second 3×3 convolution layer are also connected to the connection unit.

5. The airport runway intrusion detection system based on deep learning according to claim 2 is characterized in that: The shared detection network includes three 1×1 shared convolutional layers, two 3×3 shared convolutional layers and three detection heads, where the three 1×1 shared convolutional layers are all connected to the first 3×3 shared convolutional layer, the first 3×3 shared convolutional layer is connected to the second 3×3 shared convolutional layer, and the second 3×3 shared convolutional layer is connected to three detection heads.

6. The airport runway intrusion detection system based on deep learning according to claim 1 is characterized in that: The training process of the airport runway detection module and the intruder detection module is as follows: S1, obtain a video dataset of the airport runway, and decompose the video dataset into frame images; S2, classifying and manually annotating the frame images, dividing the frame images into a set of images with intrusion behavior and a set of images without intrusion behavior, and annotating the airport runway and the intrusion object in the frame images; S3. Use the image set without intrusion behavior as the training set of the airport runway detection module, and use the image set with intrusion behavior as the training set of the intrusion body detection module to train the airport runway detection module and the intrusion body detection module.

7. The airport runway intrusion detection system based on deep learning according to claim 1 is characterized in that: The judgment process of the intrusion judgment module is specifically as follows: if the position of the intruding body does not overlap with the position of the airport runway, it is judged that there is no invasion risk; if the position of the intruding body overlaps with the position of the airport runway but the position of the contact point between the intruding body and the ground is outside the airport runway, it is judged that there is an invasion risk; if the position of the intruding body overlaps with the position of the airport runway and the position of the contact point between the intruding body and the ground is inside the airport runway, it is judged that there is an invasion.

Images