Airport runway foreign matter monitoring method and airport runway foreign matter monitoring system based on Leiyu fusion

Abstract

The invention discloses an airport runway foreign matter monitoring method and system based on thunder-vision fusion. The method comprises the following steps: acquiring radar data when a radar monitors an airport runway and visual data when a visual sensor monitors the airport runway; setting a radar signal processing model, and calculating radar response measurement of radar reflection echoes according to the radar data; setting a visual signal processing model, and calculating visual feature measurement of a visual signal according to the visual data; setting a thunder-vision fusion model, fusing the radar response measurement of the radar reflection echo and the visual feature measurement of the visual signal, and calculating the position response value of the foreign matter after thunder-vision fusion; a foreign matter detection model is set, a foreign matter detection value is calculated according to the position response value of the foreign matter subjected to the thunder-sight fusion, and when the foreign matter detection value exceeds a preset foreign matter threshold value, warning information is sent out to prompt workers to conduct inspection and detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of airport foreign object monitoring, and more specifically, relates to a method and system for airport runway foreign object monitoring based on radar and visual fusion. Background Art

[0002] Airport runway foreign object detection is a crucial step in ensuring safe aircraft takeoff and landing. Various technologies and methods are typically used to monitor runway foreign objects to prevent them from posing a threat to aircraft. Foreign objects can include birds, debris, trash, or other objects that, if not removed promptly, can cause accidents during takeoff or landing.

[0003] Foreign object detection on airport runways is often done using two methods: visual image recognition and radar technology. Visual image recognition relies on real-time photography of the runway by high-resolution cameras or drones, combined with image processing and artificial intelligence algorithms to automatically identify foreign objects. The advantages of this method are high detection accuracy, strong target visualization, and the ability to accurately identify the type and shape of foreign objects. However, it is significantly affected by lighting, weather, and obstructions, and is particularly prone to false detection or missed detection in rainy, foggy, nighttime, or low-contrast scenes. Radar technology uses radar equipment deployed around the runway to analyze object reflection waves. It can maintain stable operation even in low-visibility environments such as fog, haze, rain, and snow, and has good all-weather capabilities. However, radar resolution is relatively low, making it difficult to identify foreign objects that are small in size, have smooth surfaces, or are made of absorbing materials.

[0004] Therefore, there is an urgent need for a technical solution that can combine the two to achieve technical complementarity, thereby improving the accuracy of foreign object monitoring on airport runways. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a method for monitoring foreign objects on airport runways based on radar and visual fusion, comprising:

[0006] Acquire radar data when radar monitors the airport runway and visual data when visual sensors monitor the airport runway;

[0007] Setting a radar signal processing model and calculating a radar response metric of a radar reflection echo based on the radar data;

[0008] Setting a visual signal processing model and calculating a visual feature metric of the visual signal based on the visual data;

[0009] Setting a radar-visual fusion model, fusing the radar response metric of the radar reflection echo and the visual feature metric of the visual signal, and calculating the position response value of the foreign object after the radar-visual fusion;

[0010] A foreign object detection model is set up, and the foreign object detection value is calculated based on the position response value of the foreign object after the radar and visual fusion. When the foreign object detection value exceeds the preset foreign object threshold, an alarm message is issued to prompt the staff to conduct patrol inspection.

[0011] Furthermore, the radar signal processing model includes:

[0012]

[0013] Among them, S radar is the radar response measure of the radar reflection echo, P tx is the radar transmission power, R0 is the reflection coefficient of the foreign object, j is the imaginary unit, f′ is the current frequency of the radar, τ is the propagation delay time of the radar wave, r is the distance between the foreign object and the radar, σ radar is the spatial resolution of the radar, δ is the correction value of the distance between the foreign object and the radar, γ1 is the first adjustment factor of the radar signal processing model, α is the second adjustment factor of the radar signal processing model, θ is the pitch angle of the foreign object relative to the radar, φ is the azimuth angle of the foreign object relative to the radar, β is the third adjustment factor of the radar signal processing model, N is the number of frequency bands, and w i is the weight of the ith frequency band, f i is the center frequency of the ith frequency band.

[0014] Furthermore, the visual signal processing model includes:

[0015]

[0016] Among them, f vision is the visual feature measurement of the visual signal, λ1 is the first adjustment factor of the visual signal processing model, x0 is the horizontal coordinate of the location of the foreign object, y0 is the vertical coordinate of the location of the foreign object, σ image is the standard deviation of the image resolution, λ2 is the second adjustment factor of the visual signal processing model, I(x, y) is the image intensity captured by the visual sensor at position (x, y), η is the third adjustment factor of the visual signal processing model, M is the number of edge information channels, ρ j′ is the weight of the j′th edge information channel, α j′ is the adjustment factor of the j′th edge information channel.

[0017] Furthermore, the radar-visual fusion model includes:

[0018]

[0019] in, is the position response value of the foreign body after the fusion of the radar and vision at time t, w radar is the weight of the radar, w visionis the weight of the visual sensor.

[0020] Furthermore, the foreign body detection model includes:

[0021] When the foreign object is a dynamic foreign object, the foreign object detection model is:

[0022]

[0023] When the foreign object is a static foreign object, the foreign object detection model is:

[0024]

[0025] Among them, Detect1 is the foreign body detection value of dynamic foreign bodies, Detect2 is the foreign body detection value of static foreign bodies, T is the detection cycle, is the speed of the foreign body at time t, γ4 is the first adjustment factor of the foreign body detection model, γ5 is the second adjustment factor of the foreign body detection model, shape target (t) is the volume of the foreign body at time t.

[0026] The present invention also proposes an airport runway foreign object monitoring system based on radar and visual fusion, comprising:

[0027] A data acquisition module is used to acquire radar data when the radar monitors the airport runway and visual data when the visual sensor monitors the airport runway;

[0028] a radar signal processing module, configured to set a radar signal processing model and calculate a radar response metric of a radar reflection echo based on the radar data;

[0029] a visual signal processing module, configured to set a visual signal processing model and calculate a visual feature metric of the visual signal based on the visual data;

[0030] A fusion module is used to set a radar-visual fusion model, fuse the radar response metric of the radar reflection echo and the visual feature metric of the visual signal, and calculate the position response value of the foreign object after the radar-visual fusion;

[0031] The foreign object detection module is used to set the foreign object detection model and calculate the foreign object detection value based on the position response value of the foreign object after the radar and visual fusion. When the foreign object detection value exceeds the preset foreign object threshold, an alarm message is issued to prompt the staff to conduct patrol inspection.

[0032] Furthermore, the radar signal processing model includes:

[0033]

[0034] Among them, S radaris the radar response measure of the radar reflection echo, P tx is the radar transmission power, R0 is the reflection coefficient of the foreign object, j is the imaginary unit, f′ is the current frequency of the radar, τ is the propagation delay time of the radar wave, r is the distance between the foreign object and the radar, σ radar is the spatial resolution of the radar, δ is the correction value of the distance between the foreign object and the radar, γ1 is the first adjustment factor of the radar signal processing model, α is the second adjustment factor of the radar signal processing model, θ is the pitch angle of the foreign object relative to the radar, φ is the azimuth angle of the foreign object relative to the radar, β is the third adjustment factor of the radar signal processing model, N is the number of frequency bands, and w i is the weight of the ith frequency band, f i is the center frequency of the ith frequency band.

[0035] Furthermore, the visual signal processing model includes:

[0036]

[0037] Among them, f vision is the visual feature measurement of the visual signal, λ1 is the first adjustment factor of the visual signal processing model, x0 is the horizontal coordinate of the location of the foreign object, y0 is the vertical coordinate of the location of the foreign object, σ image is the standard deviation of the image resolution, λ2 is the second adjustment factor of the visual signal processing model, I(x,y) is the image intensity captured by the visual sensor at position (x,y), η is the third adjustment factor of the visual signal processing model, M is the number of edge information channels, ρ j′ is the weight of the j′th edge information channel, α j′ is the adjustment factor of the j′th edge information channel.

[0038] Furthermore, the radar-visual fusion model includes:

[0039]

[0040] in, is the position response value of the foreign body after the fusion of the radar and vision at time t, w radar is the weight of the radar, w vision is the weight of the visual sensor.

[0041] Furthermore, the foreign body detection model includes:

[0042] When the foreign object is a dynamic foreign object, the foreign object detection model is:

[0043]

[0044] When the foreign object is a static foreign object, the foreign object detection model is:

[0045]

[0046] Among them, Detect1 is the foreign body detection value of dynamic foreign bodies, Detect2 is the foreign body detection value of static foreign bodies, T is the detection cycle, is the speed of the foreign body at time t, γ4 is the first adjustment factor of the foreign body detection model, γ5 is the second adjustment factor of the foreign body detection model, shape target (t) is the volume of the foreign body at time t.

[0047] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0048] Through the above technical solution, the present invention can detect foreign objects in the airport runway. When foreign objects that hinder safety are found, an alarm message is issued in time to remind airport staff to conduct safety inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of the method of embodiment 1 of the present invention;

[0050] Figure 2 This is a system structure diagram of Example 2 of the present invention. DETAILED DESCRIPTION

[0051] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0052] The method provided by the present invention can be implemented in the following terminal environment, wherein the terminal may include one or more of the following components: a processor, a storage medium, and a display screen. The storage medium stores at least one instruction, which is loaded and executed by the processor to implement the method described in the following embodiments.

[0053] A processor can include one or more processing cores. It connects various components within the terminal using various interfaces and circuits. It executes instructions, programs, code sets, or instruction sets stored in storage media, and accesses data stored in storage media to perform various terminal functions and process data.

[0054] The storage medium may include a random access memory (RAM) or a read-only memory (ROM). The storage medium may be used to store instructions, programs, codes, code sets, or instructions.

[0055] The display is used to show the user interface of each application.

[0056] In addition, those skilled in the art will appreciate that the structure of the terminal described above does not limit the terminal. The terminal may include more or fewer components, or a combination of certain components, or a different arrangement of components. For example, the terminal may also include a radio frequency circuit, an input unit, a sensor, an audio circuit, a power supply, and other components, which will not be described in detail here.

[0057] Example 1

[0058] like Figure 1 As shown, an embodiment of the present invention proposes a method for monitoring foreign objects on an airport runway based on radar and visual fusion, comprising:

[0059] Step 101, obtaining radar data when a radar monitors an airport runway and visual data when a visual sensor monitors an airport runway;

[0060] Step 102: Setting a radar signal processing model and calculating a radar response metric of the radar reflection echo based on the radar data;

[0061] Specifically, the radar signal processing model includes:

[0062]

[0063] Among them, S radar is the radar response measure of the radar reflection echo, P tx is the radar transmission power, R0 is the reflection coefficient of the foreign object, j is the imaginary unit, f′ is the current frequency of the radar, τ is the propagation delay time of the radar wave, r is the distance between the foreign object and the radar, σ radar is the spatial resolution of the radar, δ is the correction value of the distance between the foreign object and the radar, γ1 is the first adjustment factor of the radar signal processing model, α is the second adjustment factor of the radar signal processing model, θ is the pitch angle of the foreign object relative to the radar, φ is the azimuth angle of the foreign object relative to the radar, β is the third adjustment factor of the radar signal processing model, N is the number of frequency bands, and w i is the weight of the ith frequency band, f i is the center frequency of the ith frequency band.

[0064] Step 103: setting a visual signal processing model and calculating a visual feature metric of the visual signal based on the visual data;

[0065] Specifically, the visual signal processing model includes:

[0066]

[0067] Among them, f visionis the visual feature measurement of the visual signal, λ1 is the first adjustment factor of the visual signal processing model, x0 is the horizontal coordinate of the location of the foreign object, y0 is the vertical coordinate of the location of the foreign object, σ image is the standard deviation of the image resolution, λ2 is the second adjustment factor of the visual signal processing model, I(x, y) is the image intensity captured by the visual sensor at position (x, y), η is the third adjustment factor of the visual signal processing model, M is the number of edge information channels, ρ j′ is the weight of the j′th edge information channel, α j′ is the adjustment factor of the j′th edge information channel.

[0068] Step 104: Setting a radar-visual fusion model, fusing the radar response metric of the radar reflection echo and the visual feature metric of the visual signal, and calculating the position response value of the foreign object after the radar-visual fusion;

[0069] Specifically, the radar-visual fusion model includes:

[0070]

[0071] in, is the position response value of the foreign body after the fusion of the radar and vision at time t, w radar is the weight of the radar, w vision is the weight of the visual sensor.

[0072] Step 105: Set a foreign object detection model and calculate a foreign object detection value based on the position response value of the foreign object after the radar and visual fusion. When the foreign object detection value exceeds the preset foreign object threshold, an alarm message is issued to prompt the staff to conduct a patrol inspection.

[0073] Specifically, the foreign body detection model includes:

[0074] When the foreign object is a dynamic foreign object, the foreign object detection model is:

[0075]

[0076] When the foreign object is a static foreign object, the foreign object detection model is:

[0077]

[0078] Among them, Detect1 is the foreign body detection value of dynamic foreign bodies, Detect2 is the foreign body detection value of static foreign bodies, T is the detection cycle, is the speed of the foreign body at time t, γ4 is the first adjustment factor of the foreign body detection model, γ5 is the second adjustment factor of the foreign body detection model, shape target (t) is the volume of the foreign body at time t.

[0079] Example 2

[0080] like Figure 2 As shown, an embodiment of the present invention further provides an airport runway foreign object monitoring system based on radar and visual fusion, comprising:

[0081] A data acquisition module is used to acquire radar data when the radar monitors the airport runway and visual data when the visual sensor monitors the airport runway;

[0082] a radar signal processing module, configured to set a radar signal processing model and calculate a radar response metric of a radar reflection echo based on the radar data;

[0083] Specifically, the radar signal processing model includes:

[0084]

[0085] Among them, S radar is the radar response measure of the radar reflection echo, P tx is the radar transmission power, R0 is the reflection coefficient of the foreign object, j is the imaginary unit, f′ is the current frequency of the radar, τ is the propagation delay time of the radar wave, r is the distance between the foreign object and the radar, σ radar is the spatial resolution of the radar, δ is the correction value of the distance between the foreign object and the radar, γ1 is the first adjustment factor of the radar signal processing model, α is the second adjustment factor of the radar signal processing model, θ is the pitch angle of the foreign object relative to the radar, φ is the azimuth angle of the foreign object relative to the radar, β is the third adjustment factor of the radar signal processing model, N is the number of frequency bands, and w i is the weight of the ith frequency band, f i is the center frequency of the ith frequency band.

[0086] a visual signal processing module, configured to set a visual signal processing model and calculate a visual feature metric of the visual signal based on the visual data;

[0087] Specifically, the visual signal processing model includes:

[0088]

[0089] Among them, f vision is the visual feature measurement of the visual signal, λ1 is the first adjustment factor of the visual signal processing model, x0 is the horizontal coordinate of the location of the foreign object, y0 is the vertical coordinate of the location of the foreign object, σ iamge is the standard deviation of the image resolution, λ2 is the second adjustment factor of the visual signal processing model, I(x, y) is the image intensity captured by the visual sensor at position (x, y), η is the third adjustment factor of the visual signal processing model, M is the number of edge information channels, ρj′ is the weight of the j′th edge information channel, α j′ is the adjustment factor of the j′th edge information channel.

[0090] A fusion module is used to set a radar-visual fusion model, fuse the radar response metric of the radar reflection echo and the visual feature metric of the visual signal, and calculate the position response value of the foreign object after the radar-visual fusion;

[0091] Specifically, the radar-visual fusion model includes:

[0092]

[0093] in, is the position response value of the foreign body after the fusion of the radar and vision at time t, w radar is the weight of the radar, w vision is the weight of the visual sensor.

[0094] The foreign object detection module is used to set the foreign object detection model and calculate the foreign object detection value based on the position response value of the foreign object after radar and visual fusion. When the foreign object detection value exceeds the preset foreign object threshold, an alarm message is issued to prompt the staff to conduct patrol inspection.

[0095] Specifically, the foreign body detection model includes:

[0096] When the foreign object is a dynamic foreign object, the foreign object detection model is:

[0097]

[0098] When the foreign object is a static foreign object, the foreign object detection model is:

[0099]

[0100] Among them, Detect1 is the foreign body detection value of dynamic foreign bodies, Detect2 is the foreign body detection value of static foreign bodies, T is the detection cycle, is the speed of the foreign body at time t, γ4 is the first adjustment factor of the foreign body detection model, γ5 is the second adjustment factor of the foreign body detection model, shape target (t) is the volume of the foreign body at time t.

[0101] Example 3

[0102] An embodiment of the present invention further provides a storage medium storing a plurality of instructions, wherein the instructions are used to implement the airport runway foreign object monitoring method based on radar and visual fusion.

[0103] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0104] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: Step 101, acquiring radar data when a radar monitors an airport runway and visual data when a visual sensor monitors an airport runway;

[0105] Step 102: Setting a radar signal processing model and calculating a radar response metric of the radar reflection echo based on the radar data;

[0106] Specifically, the radar signal processing model includes:

[0107]

[0108] Among them, S radar is the radar response measure of the radar reflection echo, P tx is the radar transmission power, R0 is the reflection coefficient of the foreign object, j is the imaginary unit, f′ is the current frequency of the radar, τ is the propagation delay time of the radar wave, r is the distance between the foreign object and the radar, σ radar is the spatial resolution of the radar, δ is the correction value of the distance between the foreign object and the radar, γ1 is the first adjustment factor of the radar signal processing model, α is the second adjustment factor of the radar signal processing model, θ is the pitch angle of the foreign object relative to the radar, φ is the azimuth angle of the foreign object relative to the radar, β is the third adjustment factor of the radar signal processing model, N is the number of frequency bands, and w i is the weight of the ith frequency band, f i is the center frequency of the ith frequency band.

[0109] Step 103: setting a visual signal processing model and calculating a visual feature metric of the visual signal based on the visual data;

[0110] Specifically, the visual signal processing model includes:

[0111]

[0112] Among them, f vision is the visual feature measurement of the visual signal, λ1 is the first adjustment factor of the visual signal processing model, x0 is the horizontal coordinate of the location of the foreign object, y0 is the vertical coordinate of the location of the foreign object, σ image is the standard deviation of the image resolution, λ2 is the second adjustment factor of the visual signal processing model, I(x,y) is the image intensity captured by the visual sensor at position (x,y), η is the third adjustment factor of the visual signal processing model, M is the number of edge information channels, ρj′ is the weight of the j′th edge information channel, α j′ is the adjustment factor of the j′th edge information channel.

[0113] Step 104: Setting a radar-visual fusion model, fusing the radar response metric of the radar reflection echo and the visual feature metric of the visual signal, and calculating the position response value of the foreign object after the radar-visual fusion;

[0114] Specifically, the radar-visual fusion model includes:

[0115]

[0116] in, is the position response value of the foreign body after the fusion of the radar and vision at time t, w radar is the weight of the radar, w vision is the weight of the visual sensor.

[0117] Step 105: Set a foreign object detection model and calculate a foreign object detection value based on the position response value of the foreign object after the radar and visual fusion. When the foreign object detection value exceeds the preset foreign object threshold, an alarm message is issued to prompt the staff to conduct a patrol inspection.

[0118] Specifically, the foreign body detection model includes:

[0119] When the foreign object is a dynamic foreign object, the foreign object detection model is:

[0120]

[0121] When the foreign object is a static foreign object, the foreign object detection model is:

[0122]

[0123] Among them, Detect1 is the foreign body detection value of dynamic foreign bodies, Detect2 is the foreign body detection value of static foreign bodies, T is the detection cycle, is the speed of the foreign body at time t, γ4 is the first adjustment factor of the foreign body detection model, γ5 is the second adjustment factor of the foreign body detection model, shape target (t) is the volume of the foreign body at time t.

[0124] Example 4

[0125] An embodiment of the present invention also proposes an electronic device, including a processor and a storage medium connected to the processor, wherein the storage medium stores multiple instructions, which can be loaded and executed by the processor to enable the processor to execute the airport runway foreign object monitoring method based on radar and visual fusion.

[0126] Specifically, the electronic device of this embodiment may be a computer terminal, which may include: one or more processors, and a storage medium.

[0127] The storage medium can be used to store software programs and modules, such as the corresponding program instructions / modules of the method for monitoring foreign objects on airport runways based on radar-visual fusion in an embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the storage medium, thereby realizing the above-mentioned method for monitoring foreign objects on airport runways based on radar-visual fusion. The storage medium may include high-speed random access memory and may also include non-volatile storage media, such as one or more magnetic storage systems, flash memory, or other non-volatile solid-state storage media. In some instances, the storage medium may further include storage media remotely located relative to the processor, and these remote storage media may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0128] The processor may call the information and application stored in the storage medium through the transmission system to perform the following steps: Step 101, obtaining radar data when the radar monitors the airport runway and visual data when the visual sensor monitors the airport runway;

[0129] Step 102: Setting a radar signal processing model and calculating a radar response metric of the radar reflection echo based on the radar data;

[0130] Specifically, the radar signal processing model includes:

[0131]

[0132] Among them, S radar is the radar response measure of the radar reflection echo, P tx is the radar transmission power, R0 is the reflection coefficient of the foreign object, j is the imaginary unit, f′ is the current frequency of the radar, τ is the propagation delay time of the radar wave, r is the distance between the foreign object and the radar, σ radar is the spatial resolution of the radar, δ is the correction value of the distance between the foreign object and the radar, γ1 is the first adjustment factor of the radar signal processing model, α is the second adjustment factor of the radar signal processing model, θ is the pitch angle of the foreign object relative to the radar, φ is the azimuth angle of the foreign object relative to the radar, β is the third adjustment factor of the radar signal processing model, N is the number of frequency bands, and w i is the weight of the ith frequency band, f i is the center frequency of the ith frequency band.

[0133] Step 103: setting a visual signal processing model and calculating a visual feature metric of the visual signal based on the visual data;

[0134] Specifically, the visual signal processing model includes:

[0135]

[0136] Among them, f vision is the visual feature measurement of the visual signal, λ1 is the first adjustment factor of the visual signal processing model, x0 is the horizontal coordinate of the location of the foreign object, y0 is the vertical coordinate of the location of the foreign object, σ image is the standard deviation of the image resolution, λ2 is the second adjustment factor of the visual signal processing model, I(x, y) is the image intensity captured by the visual sensor at position (x, y), η is the third adjustment factor of the visual signal processing model, M is the number of edge information channels, ρ j′ is the weight of the j′th edge information channel, α j′ is the adjustment factor of the j′th edge information channel.

[0137] Step 104: Setting a radar-visual fusion model, fusing the radar response metric of the radar reflection echo and the visual feature metric of the visual signal, and calculating the position response value of the foreign object after the radar-visual fusion;

[0138] Specifically, the radar-visual fusion model includes:

[0139]

[0140] in, is the position response value of the foreign body after the fusion of the radar and vision at time t, w radar is the weight of the radar, w vision is the weight of the visual sensor.

[0141] Step 105: Set a foreign object detection model and calculate a foreign object detection value based on the position response value of the foreign object after the radar and visual fusion. When the foreign object detection value exceeds the preset foreign object threshold, an alarm message is issued to prompt the staff to conduct a patrol inspection.

[0142] Specifically, the foreign body detection model includes:

[0143] When the foreign object is a dynamic foreign object, the foreign object detection model is:

[0144]

[0145] When the foreign object is a static foreign object, the foreign object detection model is:

[0146]

[0147] Among them, Detect1 is the foreign body detection value of dynamic foreign bodies, Detect2 is the foreign body detection value of static foreign bodies, T is the detection cycle, is the speed of the foreign body at time t, γ4 is the first adjustment factor of the foreign body detection model, γ5 is the second adjustment factor of the foreign body detection model, shape target (t) is the volume of the foreign body at time t.

[0148] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0149] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0150] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the system embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, and can be electrical or other forms.

[0151] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0152] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only storage medium (ROM, Read-Only Memory), random access storage medium (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0154] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A method for monitoring foreign objects on airport runways based on radar and visual fusion, characterized in that: include: Acquire radar data when radar monitors the airport runway and visual data when visual sensors monitor the airport runway; Setting a radar signal processing model and calculating a radar response metric of a radar reflection echo based on the radar data; Setting a visual signal processing model and calculating a visual feature metric of the visual signal based on the visual data; Setting a radar-visual fusion model, fusing the radar response metric of the radar reflection echo and the visual feature metric of the visual signal, and calculating the position response value of the foreign object after the radar-visual fusion; A foreign object detection model is set up, and the foreign object detection value is calculated based on the position response value of the foreign object after the radar and visual fusion. When the foreign object detection value exceeds the preset foreign object threshold, an alarm message is issued to prompt the staff to conduct patrol inspection.

2. The method for monitoring foreign objects on an airport runway based on radar and visual fusion according to claim 1, characterized in that: The radar signal processing model includes: Among them, S radar is the radar response measure of the radar reflection echo, P tx is the radar transmission power, R0 is the reflection coefficient of the foreign object, j is the imaginary unit, f′ is the current frequency of the radar, τ is the propagation delay time of the radar wave, r is the distance between the foreign object and the radar, σ radar is the spatial resolution of the radar, δ is the correction value of the distance between the foreign object and the radar, γ1 is the first adjustment factor of the radar signal processing model, α is the second adjustment factor of the radar signal processing model, θ is the pitch angle of the foreign object relative to the radar, φ is the azimuth angle of the foreign object relative to the radar, β is the third adjustment factor of the radar signal processing model, N is the number of frequency bands, and w i is the weight of the ith frequency band, f i is the center frequency of the ith frequency band.

3. The method for monitoring foreign objects on airport runways based on radar and visual fusion as claimed in claim 2, characterized in that: The visual signal processing model includes: Among them, f vision is the visual feature measurement of the visual signal, λ1 is the first adjustment factor of the visual signal processing model, x0 is the horizontal coordinate of the location of the foreign object, y0 is the vertical coordinate of the location of the foreign object, σ image is the standard deviation of the image resolution, λ2 is the second adjustment factor of the visual signal processing model, I(x, y) is the image intensity captured by the visual sensor at position (x, y), η is the third adjustment factor of the visual signal processing model, M is the number of edge information channels, ρ j′ is the weight of the j′th edge information channel, α j′ is the adjustment factor of the j′th edge information channel.

4. The method for monitoring foreign objects on an airport runway based on radar and visual fusion as claimed in claim 3, characterized in that: The radar-visual fusion model includes: in, is the position response value of the foreign body after the fusion of the radar and vision at time t, w radar is the weight of the radar, w vision is the weight of the visual sensor.

5. The method for monitoring foreign objects on an airport runway based on radar and visual fusion according to claim 4, characterized in that: The foreign body detection model includes: When the foreign object is a dynamic foreign object, the foreign object detection model is: When the foreign object is a static foreign object, the foreign object detection model is: Among them, Detect1 is the foreign body detection value of dynamic foreign bodies, Detect2 is the foreign body detection value of static foreign bodies, T is the detection cycle, is the speed of the foreign body at time t, γ4 is the first adjustment factor of the foreign body detection model, γ5 is the second adjustment factor of the foreign body detection model, shape target (t) is the volume of the foreign body at time t.

6. An airport runway foreign object monitoring system based on radar and visual fusion, characterized in that: include: A data acquisition module is used to acquire radar data when the radar monitors the airport runway and visual data when the visual sensor monitors the airport runway; a radar signal processing module, configured to set a radar signal processing model and calculate a radar response metric of a radar reflection echo based on the radar data; a visual signal processing module, configured to set a visual signal processing model and calculate a visual feature metric of the visual signal based on the visual data; A fusion module is used to set a radar-visual fusion model, fuse the radar response metric of the radar reflection echo and the visual feature metric of the visual signal, and calculate the position response value of the foreign object after the radar-visual fusion; The foreign object detection module is used to set the foreign object detection model and calculate the foreign object detection value based on the position response value of the foreign object after radar and visual fusion. When the foreign object detection value exceeds the preset foreign object threshold, an alarm message is issued to prompt the staff to conduct patrol inspection.

7. The airport runway foreign object monitoring system based on radar and visual fusion according to claim 6, characterized in that: The radar signal processing model includes: Among them, S radar is the radar response measure of the radar reflection echo, P tx is the radar transmission power, R0 is the reflection coefficient of the foreign object, j is the imaginary unit, f′ is the current frequency of the radar, τ is the propagation delay time of the radar wave, r is the distance between the foreign object and the radar, σ radar is the spatial resolution of the radar, δ is the correction value of the distance between the foreign object and the radar, γ1 is the first adjustment factor of the radar signal processing model, α is the second adjustment factor of the radar signal processing model, θ is the pitch angle of the foreign object relative to the radar, φ is the azimuth angle of the foreign object relative to the radar, β is the third adjustment factor of the radar signal processing model, N is the number of frequency bands, and w i is the weight of the ith frequency band, f i is the center frequency of the ith frequency band.

8. The airport runway foreign object monitoring system based on radar and visual fusion as claimed in claim 7, characterized in that: The visual signal processing model includes: Among them, f vision is the visual feature measurement of the visual signal, λ1 is the first adjustment factor of the visual signal processing model, x0 is the horizontal coordinate of the location of the foreign object, y0 is the vertical coordinate of the location of the foreign object, σ image is the standard deviation of the image resolution, λ2 is the second adjustment factor of the visual signal processing model, I(x, y) is the image intensity captured by the visual sensor at position (x, y), η is the third adjustment factor of the visual signal processing model, M is the number of edge information channels, ρ j′ is the weight of the j′th edge information channel, α j′ is the adjustment factor of the j′th edge information channel.

9. The airport runway foreign object monitoring system based on radar and visual fusion as claimed in claim 8, characterized in that: The radar-visual fusion model includes: in, is the position response value of the foreign body after the fusion of the radar and vision at time t, w radar is the weight of the radar, w vision is the weight of the visual sensor.

10. The airport runway foreign object monitoring system based on radar and visual fusion according to claim 9, characterized in that: The foreign body detection model includes: When the foreign object is a dynamic foreign object, the foreign object detection model is: When the foreign object is a static foreign object, the foreign object detection model is: Among them, Detect1 is the foreign body detection value of dynamic foreign bodies, Detect2 is the foreign body detection value of static foreign bodies, T is the detection cycle, is the speed of the foreign body at time t, γ4 is the first adjustment factor of the foreign body detection model, γ5 is the second adjustment factor of the foreign body detection model, shape target (t) is the volume of the foreign body at time t.

Images