A multi-sensor based airport runway foreign object detection system and method

Abstract

The present invention relates to the field of foreign object detection technology, specifically to a multi-sensor-based airport runway foreign object detection system and method, the system comprising a foreign object detection module, an environmental analysis module, a threshold adjustment module, a priority judgment module, a path planning module, and a navigation and execution module. In the present invention, by combining millimeter wave radar and image sensor, foreign objects on the runway can be accurately identified, and the object coordinates are adjusted using real-time environmental data, thereby improving the accuracy of positioning and reducing the possibility of false alarms. By dynamically adjusting environmental sensitivity, it is possible to adapt to environmental changes and maintain efficient operation. Through priority evaluation, resource allocation is made more reasonable, ensuring that key foreign object threats are handled with priority, minimizing safety risks, and optimizing cleaning path planning to reduce interference with airport operations and improve the efficiency of runway use.

Description

Technical Field

[0001] The present invention relates to the technical field of foreign object detection, and in particular to a multi-sensor based airport runway foreign object detection system and method. Background Art

[0002] The field of foreign object detection technology mainly involves the use of various sensors, monitoring equipment and data analysis technologies to identify and locate unexpected objects in security-sensitive and operation-intensive areas. In the airport runway environment, foreign objects include gravel, aircraft parts fragments, tools or other debris, which can cause serious safety hazards during aircraft takeoff and landing.

[0003] Among them, the multi-sensor based airport runway foreign object detection system aims to enhance the detection capability of foreign objects on the airport runway by integrating multiple types of sensors. The purpose of the system is to improve the coverage and accuracy of detection and reduce the risk of accidents caused by undetected runway obstacles. By monitoring the runway conditions in real time, it can promptly warn airport ground personnel and pilots of potential obstacles on the runway, thereby ensuring the safe takeoff and landing of aircraft.

[0004] Traditional foreign object detection systems lack effective environmental adaptability and cannot adjust detection parameters based on real-time environmental data, limiting their application effectiveness in dynamic environments. The lack of a priority judgment mechanism leads to uneven resource allocation, untimely processing, and slow response to emergencies, creating potential safety hazards. For example, at a busy airport, failure to quickly identify and process critical foreign objects that affect aircraft takeoff and landing can lead to flight delays or even accidents, affecting overall safety and efficiency. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a multi-sensor based airport runway foreign object detection system and method.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solution: a multi-sensor based airport runway foreign object detection system, the system comprising;

[0007] The foreign object detection module identifies and locates foreign objects on the airport runway based on the millimeter-wave radar and image sensor data of the airport runway, and obtains the basic object coordinates;

[0008] The environment analysis module adjusts the position deviation of the basic object coordinates based on the basic object coordinates through the data of the light intensity sensor and the temperature and humidity sensor to generate optimized and corrected object coordinates;

[0009] The threshold adjustment module adjusts the detection threshold based on the optimized and corrected object coordinates and combines the current airport runway environment change data to obtain optimized detection parameters;

[0010] The priority judgment module evaluates the threat level of foreign objects on the airport runway based on the optimized and corrected object coordinates, evaluates the priority of foreign object removal, and obtains a priority decision result;

[0011] The path planning module calculates the foreign object cleaning path based on the optimized corrected object coordinates and priority decision results, combined with the real-time airport runway usage status and ground condition data, and adjusts the path to optimize the processing efficiency of foreign objects, and generates an optimized cleaning path;

[0012] The navigation and execution module guides the foreign object cleaning personnel to perform cleaning operations based on the optimized cleaning path, updates the cleaning progress in real time, matches the foreign objects that appear immediately, and generates cleaning task completion information.

[0013] As a further solution of the present invention, the basic object coordinates include the geographic location, size and shape information of the object, the optimized and corrected object coordinates include the adjusted geographic location of the object, the environmental impact correction value and the timestamp, the optimized detection parameters include the adjusted sensitivity level, the environmental adaptation threshold and the image clarity index, the priority decision result includes the urgency of processing the foreign object, the impact rating and the recommended processing time, the optimized cleaning path includes the coordinates of the path starting point and the end point, the estimated cleaning time and the safety assessment level, and the cleaning task completion information includes the cleaning area identification, the completion time and the cleaning quality assessment.

[0014] As a further solution of the present invention, the foreign object detection module includes:

[0015] The feature comparison submodule analyzes the captured images and radar signals based on the millimeter-wave radar and image sensor data of the airport runway, compares the reflection characteristic differences and image contour information in the two data, distinguishes foreign objects from the airport runway background, and obtains foreign object feature data;

[0016] The image optimization submodule adjusts the focal length and shooting angle of the image sensor based on the foreign object feature data to optimize image capture, and processes the image to optimize the clarity and accuracy of the image to obtain an adjusted image;

[0017] Based on the adjusted image, the coordinate positioning submodule identifies the geographical location of the foreign object by analyzing the millimeter wave radar and image sensor data, calculates the coordinate points of the foreign object on the runway, and obtains the basic object coordinates.

[0018] As a further solution of the present invention, the environmental analysis module includes:

[0019] Based on the basic object coordinates, the environmental monitoring submodule collects the illumination, temperature and humidity data of the current airport runway through the illumination intensity sensor and the temperature and humidity sensor, analyzes the impact of environmental parameters on the performance of the millimeter wave radar and image sensor, and generates environmental impact information;

[0020] The data correction submodule applies historical environmental data based on the environmental impact information, compares and corrects the data deviation of the current millimeter wave radar and image sensor, optimizes the accuracy of environmental monitoring, and obtains environmental correction information;

[0021] The coordinate adjustment submodule adjusts the position deviation of the foreign object based on the environmental correction information and according to real-time weather changes, optimizes the accuracy of the foreign object coordinates, matches the current environmental conditions, and generates optimized and corrected object coordinates.

[0022] As a further solution of the present invention, the threshold adjustment module includes:

[0023] The parameter calculation submodule calculates the influence of various environmental factors on the detection performance of millimeter wave radar and image sensor based on the optimized and corrected object coordinates and the current airport runway environment change data, identifies the detection threshold that needs to be adjusted, and obtains the environmental sensitivity index data;

[0024] The threshold optimization submodule analyzes the existing detection thresholds of the millimeter-wave radar and the image sensor based on the environmental sensitivity index data, adjusts and optimizes the detection thresholds of the millimeter-wave radar and the image sensor according to the data fluctuation characteristics under the current light and temperature conditions, optimizes the threshold settings to match the environmental changes, and generates detection threshold adjustment parameters;

[0025] The sensitivity update submodule adjusts the parameters based on the detection threshold, updates the sensitivity settings of the millimeter-wave radar and the image sensor, adjusts the sensitivity of the millimeter-wave radar and the image sensor to match the actual environment, optimizes the accuracy and response speed of foreign object recognition, and obtains optimized detection parameters.

[0026] As a further solution of the present invention, the priority determination module includes:

[0027] The threat assessment submodule analyzes the size, shape and material reflection characteristics of each foreign object based on the optimized and corrected object coordinates, identifies the potential hazard level of each foreign object according to the physical characteristics of the foreign object and the position of the foreign object on the runway, and generates an object threat level;

[0028] The impact analysis submodule analyzes the impact of foreign objects on the safety of aircraft takeoff and landing based on the threat level of the object, including the possibility of foreign objects damaging aircraft tires and causing flight delays. It evaluates the impact of foreign objects by simulating multiple scenarios and obtains flight safety impact assessment results.

[0029] The priority sorting submodule is based on the flight safety impact assessment results and adopts a multi-attribute decision analysis method to analyze the clearance priorities of various foreign objects. According to the risks and impacts of the foreign objects, combined with the frequency of runway use and the location of the foreign objects, the priority of foreign object handling is sorted to obtain a priority decision result.

[0030] As a further solution of the present invention, the multi-attribute decision analysis method is according to the formula:

[0031]

[0032] Calculate the priority value for foreign object handling, where A priority value for foreign object processing, For the The weight of the attribute, For the The standardized value of the attribute, Score the impact of foreign objects on the flight's proximity, Score the impact of foreign objects on runway efficiency, is the total number of attributes, is the weight coefficient of temporal proximity, is the weight coefficient of runway utilization efficiency.

[0033] As a further solution of the present invention, the path planning module includes:

[0034] The path calculation submodule analyzes the cleaning path from the current position to each foreign object position based on the optimized corrected object coordinates and priority decision results, using real-time airport runway usage status and ground condition data, combined with the flight area and ground obstacles, and generates a basic cleaning route;

[0035] The safety assessment submodule checks the multiple areas that the cleaning route passes through based on the basic cleaning route, evaluates the distance between the cleaning route and the flight operation area, ground obstacles and risk factors, adjusts the cleaning path to avoid risk areas, optimizes the safety of the cleaning operation, and generates a safety verification path;

[0036] The path adjustment submodule adjusts the path length and steering angle based on the safety verification path, optimizes the foreign object cleaning time and distance, analyzes the impact of the foreign object cleaning process on the flight, optimizes the distribution and sequence of path points, and generates an optimized cleaning path.

[0037] A multi-sensor based airport runway foreign object detection method, the multi-sensor based airport runway foreign object detection method is performed based on the multi-sensor based airport runway foreign object detection system, comprising the following steps:

[0038] S1: Based on the millimeter-wave radar and image sensor data of the airport runway, foreign object feature capture and positioning are performed to generate basic object coordinates;

[0039] S2: Based on the basic object coordinates, the influence of the environment on the sensor performance is analyzed through the data of the light intensity sensor and the temperature and humidity sensor, the foreign object coordinates are adjusted, and the optimized corrected object coordinates are generated;

[0040] S3: Based on the optimized and corrected object coordinates, using the current environmental change data, calculating the environmental sensitivity index, adjusting the detection threshold, and obtaining the adjusted detection parameters;

[0041] S4: Based on the adjusted detection parameters and the optimized corrected object coordinates, assess the threat level of the foreign object and generate a priority decision result;

[0042] S5: Based on the priority decision result and the optimized corrected object coordinates, a cleaning path is calculated, a safe path that avoids the flight area is selected, and an optimized cleaning path is generated;

[0043] S6: Based on the optimized cleaning path, guide the foreign object cleaning personnel to perform the cleaning task, update the navigation information in real time, match the site condition changes and the foreign objects identified in real time, and obtain the cleaning navigation instructions;

[0044] S7: Based on the cleaning navigation instruction, monitor the cleaning quality and progress, identify the removal of foreign objects, and generate cleaning task completion information.

[0045] Compared with the prior art, the advantages and positive effects of the present invention are:

[0046] In the present invention, by combining millimeter-wave radar and image sensor, foreign objects on the runway can be accurately identified, and the object coordinates can be adjusted using real-time environmental data, thereby improving the accuracy of positioning and reducing the possibility of false alarms. By dynamically adjusting environmental sensitivity, it can adapt to environmental changes and maintain high-efficiency operation. Through priority evaluation, resource allocation is more reasonable, ensuring that key foreign object threats are given priority and safety risks are minimized. Optimized clearing path planning reduces interference with airport operations and improves the efficiency of runway use. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a system flow chart of the present invention;

[0048] Figure 2 It is a schematic diagram of the system framework of the present invention;

[0049] Figure 3 This is a flow chart of the foreign object detection module of the present invention;

[0050] Figure 4 It is a flow chart of the environmental analysis module of the present invention;

[0051] Figure 5 is a flow chart of a threshold adjustment module of the present invention;

[0052] Figure 6 It is a flow chart of the priority determination module of the present invention;

[0053] Figure 7 This is a flow chart of the path planning module of the present invention;

[0054] Figure 8 It is a flow chart of the navigation and execution module of the present invention;

[0055] Fig. 9 It is a schematic diagram of the method steps of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0057] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0058] Embodiment 1

[0059] See also Figure 1 ,The present invention provides a technical solution: a multi-sensor based airport runway foreign object detection system, the system includes a foreign object detection module, an environment analysis module, a threshold adjustment module, a priority judgment module, a path planning module and a navigation and execution module;

[0060] The foreign object detection module identifies and locates foreign objects on the airport runway based on the millimeter-wave radar and image sensor data of the airport runway. It refines the object features by comparing the radar echo and image data, adjusts the focal length and shooting angle to optimize the location of foreign objects, and obtains the basic object coordinates.

[0061] The environmental analysis module analyzes the current environmental conditions of the airport runway based on the basic object coordinates, the light intensity sensor, and the temperature and humidity sensor data. It adjusts the position deviation of the basic object coordinates according to the real-time weather conditions and generates optimized and corrected object coordinates.

[0062] The threshold adjustment module calculates the environmental sensitivity index based on the optimized and corrected object coordinates and the current airport runway environment change data, adjusts the detection threshold, matches the current light and temperature conditions, and obtains the optimized detection parameters;

[0063] The priority judgment module evaluates the threat level of foreign objects on the airport runway based on the optimized and corrected object coordinates. It analyzes the potential impact of foreign objects on flight operations based on the size, shape and reflection characteristics of the foreign objects, evaluates the priority of foreign object removal, and obtains the priority decision result.

[0064] The path planning module calculates the foreign object cleaning path based on the optimized and corrected object coordinates and priority decision results, combined with the real-time airport runway usage status and ground condition data, selects a safe path to avoid the flight area, and adjusts the path to optimize the processing efficiency of foreign objects, generating an optimized cleaning path;

[0065] The navigation and execution module guides the foreign object cleaning personnel to perform cleaning operations based on the optimized cleaning path, updates the cleaning progress in real time, adjusts the cleaning path, matches the foreign objects that appear immediately, and generates cleaning task completion information after completing the cleaning of the designated area.

[0066] The basic object coordinates include the object's geographic location, size and shape information; the optimized and corrected object coordinates include the adjusted object's geographic location, environmental impact correction value and timestamp; the optimized detection parameters include the adjusted sensitivity level, environmental adaptation threshold and image clarity index; the priority decision results include the urgency of foreign object processing, impact rating and recommended processing time; the optimized cleaning path includes the path start and end coordinates, estimated cleaning time and safety assessment level; the cleaning task completion information includes the cleaning area identification, completion time and cleaning quality assessment.

[0067] See also Figure 2 and Figure 3 ,The foreign object detection module includes a feature comparison submodule, an image optimization submodule, and a coordinate positioning submodule;

[0068] The feature comparison submodule analyzes the captured images and radar signals based on the millimeter-wave radar and image sensor data of the airport runway, compares the reflection characteristic differences and image contour information in the two data, distinguishes foreign objects from the airport runway background, and obtains the specific process of foreign object feature data as follows;

[0069] The feature comparison submodule is based on the millimeter-wave radar and image sensor data of the airport runway. The data set consists of reflection intensity data R and image data set I. The radar data R is preliminarily enhanced to eliminate the noise introduced by environmental factors such as rain and fog. The enhancement algorithm is based on dynamic threshold adjustment, where the threshold is automatically adjusted according to the current environmental conditions. The formula is T=m(R)+σ(R), where m(R) represents the mean of the radar data and σ(R) represents the standard deviation. The processed radar data is synchronized with the image data I, and the contour information C is extracted from the image using edge detection technology. Edge detection is implemented through the improved Canny algorithm, which is used for high-contrast image feature extraction. The algorithm outputs the edge intensity matrix E, which includes the intensity value of each edge pixel. The radar data and image contour data are combined. By calculating the angle difference between the radar reflection vector and the image contour vector, the difference between the foreign object and the airport runway background can be determined. A weight parameter λ is introduced to adjust the proportion of reflection characteristics and contour information to optimize the comparison results. The comparison formula is D=sum(λ*abs(RE)), where sum represents the sum of the elements in the vector, and abs represents the absolute value function. The foreign object feature data is obtained, including information such as the type, size, and reflection characteristics of the foreign object.

[0070] The image optimization submodule adjusts the focal length and shooting angle of the image sensor based on the foreign object feature data, optimizes image capture, and processes the image to optimize the clarity and accuracy of the image. The specific process of obtaining the adjusted image is as follows;

[0071] The image optimization submodule adjusts the focal length and shooting angle of the image sensor based on the feature data of foreign objects. The purpose of the adjustment is to maximize the image quality of the target object and ensure that its features can be captured more accurately. The calculation of the focal length f and angle θ depends on the size and estimated distance of the foreign object. The calculation formula is f=d*tan(θ / 2), where d is the distance between the foreign object and the sensor, and θ is the required viewing angle range. After adjusting the focal length, the captured image is post-processed, including sharpening and contrast enhancement, to improve the clarity and detail of the image. Image sharpening is performed through the Laplace operator, which can enhance the edge information of the image and make the outline of the foreign object clearer. Contrast enhancement is achieved through adaptive histogram equalization, which can provide the best image contrast under different lighting conditions to obtain the adjusted image.

[0072] Based on the adjusted image, the coordinate positioning submodule analyzes the millimeter wave radar and image sensor data to identify the geographical location of the foreign object, calculate the coordinate points of the foreign object on the runway, and obtain the specific process of the basic object coordinates as follows;

[0073] The coordinate positioning submodule determines the precise geographic location of the foreign object based on the adjusted image combined with the millimeter wave radar data. The time difference between the landmarks in the image and the radar signal is used to estimate the distance of the foreign object. The calculation of the geographic coordinates uses an improved triangulation method, which includes the measurement values ​​of the angle α and the distance d. The formula is x=d*cos(α), y=d*sin(α), where (x, y) represents the coordinate point of the foreign object on the runway, which can effectively obtain the coordinate point of the foreign object and obtain the coordinates of the basic object.

[0074] See also Figure 2 and Figure 4 ,The environmental analysis module includes an environmental monitoring submodule, a data correction submodule, and a coordinate adjustment submodule;

[0075] Based on the basic object coordinates, the environmental monitoring submodule collects the illumination, temperature and humidity data of the current airport runway through the illumination intensity sensor and the temperature and humidity sensor, analyzes the impact of environmental parameters on the performance of millimeter-wave radar and image sensor, and generates the specific process of environmental impact information as follows;

[0076] The environmental monitoring submodule deploys light intensity sensors and temperature and humidity sensors at designated locations based on the basic object coordinates. Monitor the light, temperature, and humidity conditions of the airport runway. The light intensity sensor records the light data, while the temperature and humidity sensor records the temperature and humidity data respectively. Each data is processed by the sliding average method to smooth occasional fluctuations. The calculation formula is "smoothed data = (sum of data) / number of samples". The processed data is used to evaluate the impact of environmental parameters on the performance of millimeter wave radar and image sensor. The impact of light on the image sensor is quantified by the light sensitivity adjustment coefficient, which automatically adjusts the exposure setting of the sensor. The impact of temperature and humidity on the radar is compensated by adjusting the propagation speed and attenuation coefficient of the signal. The compensation formula is "adjusted speed = standard speed * (1 + temperature impact coefficient temperature value + humidity impact coefficient humidity value)", which generates environmental impact information.

[0077] The data correction submodule uses historical environmental data based on environmental impact information to compare and correct the data deviation of the current millimeter-wave radar and image sensor, optimize the accuracy of environmental monitoring, and obtain the specific process of environmental correction information:

[0078] The data correction submodule uses the stored historical environmental data for comparative analysis based on environmental impact information to calculate the deviation of the current millimeter-wave radar and image sensor data. The deviation calculation uses the empirical correction formula "corrected data = current data + correction coefficient * (historical average data - current data)" to adjust and optimize data accuracy. The corrected data is used to improve the accuracy of environmental monitoring and obtain environmental correction information, which is used to optimize the accuracy of data processing.

[0079] The coordinate adjustment submodule adjusts the position deviation of foreign objects based on environmental correction information and real-time weather changes, optimizes the accuracy of foreign object coordinates, matches current environmental conditions, and generates the specific process of optimizing and correcting object coordinates as follows;

[0080] The coordinate adjustment submodule adjusts the deviation of the position of foreign objects based on environmental correction information and real-time weather changes. A dynamic adjustment algorithm is used to adjust the coordinate points of foreign objects based on real-time weather data. The coordinate adjustment formula is "adjusted coordinates = current coordinates + adjustment factor * (correction information calculated coordinates - current coordinates)", where the adjustment factor is dynamically calculated based on environmental influencing factors. The adjusted coordinates reflect the influence of changes in temperature, humidity and light intensity, ensuring that the position information of foreign objects remains highly accurate under any environmental conditions, generating optimized and corrected object coordinates, and providing accurate and real-time spatial data for airport operation management.

[0081] See also Figure 2 and Figure 5 ,The threshold adjustment module includes a parameter calculation submodule, a threshold optimization submodule, and a sensitivity updating submodule;

[0082] The parameter calculation submodule is based on optimizing and correcting the object coordinates, combined with the current airport runway environment change data, to calculate the impact of various environmental factors on the detection performance of millimeter-wave radar and image sensor, identify the detection threshold that needs to be adjusted, and obtain the specific process of environmental sensitivity index data as follows;

[0083] The parameter calculation submodule evaluates the influence of various environmental factors on the detection performance of millimeter-wave radar and image sensors based on the optimization and correction of object coordinates and the current environmental change data of the airport runway. Environmental factors include light intensity, temperature, humidity, etc., which affect the sensitivity of the equipment and data quality. The environmental sensitivity analysis algorithm is adopted, and the calculation formula is "environmental sensitivity index = ∑ (environmental factor weight × environmental factor impact)", where the environmental factor weight is set according to the influence of historical data on equipment performance, and which detection thresholds need to be adjusted according to the current environmental conditions. The environmental sensitivity index data is obtained, which describes the specific impact of each environmental factor on equipment performance and provides a basis for threshold optimization.

[0084] The threshold optimization submodule analyzes the existing detection thresholds of the millimeter-wave radar and image sensor based on the environmental sensitivity index data, adjusts and optimizes the detection thresholds of the millimeter-wave radar and image sensor according to the data fluctuation characteristics under the current light and temperature conditions, optimizes the threshold settings to match environmental changes, and generates the specific process of detection threshold adjustment parameters as follows;

[0085] The threshold optimization submodule analyzes the existing detection thresholds of millimeter wave radar and image sensor based on the environmental sensitivity index data. Considering the data fluctuation characteristics under the current lighting and temperature conditions, the threshold adjustment value = current threshold + adjustment coefficient × (target threshold - current threshold) is used, where the adjustment coefficient is dynamically calculated according to the environmental sensitivity index to ensure that the threshold matches the environmental changes, optimize the detection threshold setting, and ensure the best performance of the device under different environmental conditions. Generate the detection threshold adjustment parameters, which describe the adjusted threshold to adapt to the current and expected environmental conditions.

[0086] The sensitivity update submodule adjusts the parameters based on the detection threshold, updates the sensitivity settings of the millimeter-wave radar and image sensor, adjusts the sensitivity of the millimeter-wave radar and image sensor to match the actual environment, optimizes the accuracy and response speed of foreign object recognition, and obtains the specific process of optimizing the detection parameters as follows;

[0087] The sensitivity update submodule receives the detection threshold adjustment parameter and updates the sensitivity settings of the millimeter-wave radar and image sensor accordingly. The sensitivity update value = current sensitivity + adjustment factor × (adaptation sensitivity - current sensitivity) is adopted. The adjustment factor is determined based on the detection threshold adjustment parameter and the actual environment data to obtain the optimized detection parameters, adjust the sensitivity of the millimeter-wave radar and image sensor to perfectly match the actual environment, optimize the accuracy and response speed of foreign object recognition, and can quickly and accurately recognize and respond to the presence of foreign objects.

[0088] See also Figure 2 and Figure 6 ,The priority judgment module includes a threat assessment submodule, an impact analysis submodule, and a priority sorting submodule;

[0089] The threat assessment submodule analyzes the size, shape and material reflection characteristics of each foreign object based on the optimized and corrected object coordinates. It identifies the potential hazard level of each foreign object based on its physical characteristics and location on the runway. The specific process of generating the object threat level is as follows:

[0090] The threat assessment submodule analyzes the size, shape and material reflective characteristics of each foreign object based on the optimized and corrected object coordinates. The volume and surface characteristics of the foreign object are calculated using the volume calculation formula "volume = length × width × height". The surface characteristic analysis is performed based on the reflectivity coefficient of the material. According to the physical characteristics of the foreign object and its specific location on the runway, the module applies a risk assessment algorithm with the calculation formula "threat level = impact coefficient × (position coefficient + characteristic coefficient)". The impact coefficient is based on a comprehensive assessment of the object's size and reflective characteristics, the position coefficient is dynamically adjusted based on the distance between the object and the critical area of ​​the runway, and the characteristic coefficient takes into account the hazardous properties of the object's material. The generated object threat level describes the potential hazard level of each foreign object, providing a basis for impact analysis.

[0091] The impact analysis submodule analyzes the impact of foreign objects on aircraft takeoff and landing safety based on the threat level of the object, including the possibility of foreign objects damaging aircraft tires and causing flight delays. The specific process of evaluating the impact of foreign objects by simulating multiple scenarios and obtaining the flight safety impact assessment results is as follows;

[0092] The impact analysis submodule analyzes the potential impact of foreign objects on aircraft takeoff and landing safety based on the threat level of the object. Consider the damage that foreign objects may cause to aircraft tires and the flight delays that may result. The damage possibility analysis uses the probability calculation formula "damage possibility = damage sensitivity × threat level", where damage sensitivity is determined based on aircraft type and tire specifications. The flight delay analysis is based on delay sensitivity and foreign object threat level, and the calculation formula is "delay probability = delay sensitivity × threat level". By simulating multiple scenarios to evaluate the impact of foreign objects, such as changes in the location of foreign objects or changes in the impact under different weather conditions, the flight safety impact assessment results are generated. The results help airport management understand the impact of foreign objects on airport operation safety and formulate countermeasures.

[0093] The priority sorting submodule is based on the flight safety impact assessment results and uses a multi-attribute decision analysis method to analyze the removal priorities of various foreign objects. According to the risk and impact of foreign objects, combined with the frequency of runway use and the location of foreign objects, the priority of handling foreign objects is sorted. The specific process of obtaining the priority decision result is as follows;

[0094] The priority sorting submodule uses a multi-attribute decision analysis method to sort the handling priorities of various foreign objects based on the flight safety impact assessment results. The risk and impact of foreign objects are analyzed, and the priority sorting is performed based on the frequency of runway use and the specific location of foreign objects to generate priority decision results. The results provide a scientific basis for airport ground handling and safety management departments.

[0095] Multi-attribute decision analysis method, according to the formula:

[0096]

[0097] Calculate the priority value for foreign object handling, where A priority value for foreign object processing, For the The weight of each attribute is used to measure its relative importance in the overall priority calculation. For the Standardized values ​​of each attribute ensure that the attribute values ​​are compared and calculated on the same scale. Score the impact of foreign objects on the approach time of flights to measure the potential impact of foreign objects on upcoming flights. Score the impact of foreign objects on runway efficiency, taking into account the impact of the location of foreign objects on airport operating efficiency, is the total number of attributes, including the size, shape, material, location, mobility, color, reflective properties, and potential hazards of the foreign object. is the weight coefficient of time proximity, adjust The degree of influence in the overall priority calculation, is the weight coefficient of runway utilization efficiency, adjust The degree of influence in the overall priority calculation.

[0098] The specific implementation process of the improved formula is as follows:

[0099] Based on the flight safety impact assessment results, the risk and impact of each foreign object are standardized. The risk of foreign objects includes the physical damage caused to the aircraft. The degree of interference of foreign objects to airport operations is considered to determine the impact. The priority score is calculated based on the frequency of runway use and the location of foreign objects. The proximity time of foreign objects to flights is considered. For example, if a foreign object is close to the next flight landing, its removal priority will be increased. Indicates that the impact of foreign objects on runway efficiency, such as a foreign object located in a critical part of the runway, resulting in a long runway closure, can be solved by Determined through historical data analysis or expert evaluation and The specific value of ensures the accuracy and adaptability of the calculation, and calculates the foreign object processing priority value through the formula , generating improved priority decision results.

[0100] See also Figure 2 and Figure 7 ,The path planning module includes a path calculation submodule, a safety assessment submodule, and a path adjustment submodule;

[0101] The path calculation submodule is based on the optimized and corrected object coordinates and priority decision results, uses real-time airport runway usage status and ground condition data, combines the flight area and ground obstacles, analyzes the cleaning path from the current position to each foreign object location, and generates the specific process of the basic cleaning route:

[0102] The path calculation submodule plans the cleaning path based on the optimized corrected object coordinates and priority decision results. Considering factors such as the flight area and ground obstacles such as maintenance facilities and ground vehicles, the path planning algorithm is used to calculate the optimal cleaning path from the current position to each foreign object. The path planning algorithm adopts a cost minimization strategy, and the calculation formula is "path cost = path length × ground condition coefficient + obstacle avoidance cost". The ground condition coefficient reflects the flatness and passability of the ground. The obstacle avoidance cost is dynamically adjusted according to the type and size of the obstacle to generate a basic cleaning route, which describes the shortest and safest path required to reach the location of each foreign object, ensuring that the cleaning work can be carried out efficiently.

[0103] The safety assessment submodule is based on the basic cleaning route, checks the multiple areas that the cleaning route passes through, evaluates the distance between the cleaning route and the flight operation area, ground obstacles and risk factors, adjusts the cleaning path to avoid risk areas, optimizes the safety of the cleaning operation, and generates the specific process of the safety verification path:

[0104] The safety assessment submodule checks multiple areas that the cleaning route passes through based on the basic cleaning route. The distance between the cleaning route and the flight operation area, ground obstacles and risk factors are assessed using the risk assessment formula "risk value = distance factor × distance assessment + obstacle factor × obstacle assessment". The distance factor is adjusted based on the proximity of the cleaning path to the flight operation area, and the obstacle factor is adjusted based on the potential harmfulness of the obstacles. Based on the risk assessment results, the cleaning path is adjusted to avoid high-risk areas and optimize the safety of the cleaning operation. The generated safety verification path avoids potential risks more accurately and ensures that the cleaning operation does not interfere with the normal operation of the airport and the safe takeoff and landing of aircraft.

[0105] The path adjustment submodule adjusts the path length and steering angle based on the safety verification path, optimizes the time and distance of foreign object cleaning, analyzes the impact of the foreign object cleaning process on the flight, optimizes the distribution and sequence of path points, and generates the specific process of optimizing the cleaning path:

[0106] The path adjustment submodule makes minor adjustments to the path length and steering angle based on the safety verification path. The path length and necessary steering angle are analyzed, and the path optimization algorithm is used. The formula "optimized path = original path length - reducible length × length adjustment coefficient + steering adjustment cost" is used. The length adjustment coefficient is dynamically calculated based on the actual ground conditions and the location of foreign objects. The steering adjustment cost takes into account the difficulty and frequency of steering. The impact of the foreign object cleaning process on flight operations is analyzed to ensure that the cleaning operation does not cause unnecessary delays. By optimizing the distribution and sequence of path points, an optimized cleaning path is generated, which shortens the cleaning time, reduces the moving distance, and improves the overall efficiency and safety of airport operations.

[0107] See also Figure 2and Figure 8 ,The navigation and execution module includes a cleaning navigation submodule, a cleaning monitoring submodule, and a task confirmation submodule;

[0108] The cleaning navigation submodule guides foreign object cleaning personnel and equipment to the location of foreign objects based on the optimized cleaning path, cleans foreign objects, updates navigation information in real time, matches site condition changes and instantly identified foreign objects, optimizes the accuracy and response speed of the cleaning process, and obtains the specific process of real-time navigation instructions as follows;

[0109] The cleaning navigation submodule guides foreign object cleaning personnel and equipment to the location of foreign objects based on the optimized cleaning path. Using the real-time navigation system, navigation instructions are issued according to the optimized cleaning path. The instructions include the turning angle, the distance to the next turning point, and the estimated arrival time. At the same time, the site condition change data and the newly identified foreign object information are received to dynamically adjust the navigation route. The adjustment formula is "adjusted path = current path + change factor × adjustment coefficient". The change factor includes new obstacle information or environmental changes. The adjustment coefficient is dynamically calculated based on real-time data to optimize the accuracy and response speed of the cleaning process, ensuring that the cleaning personnel and equipment can reach the destination efficiently and accurately. The generated real-time navigation instructions provide detailed guidance for the cleaning operation to ensure the effective removal of foreign objects.

[0110] The cleaning monitoring submodule monitors the position and progress of the cleaning process based on real-time navigation instructions. Through image sensor data, it detects the working status and effect of foreign object cleaning personnel and cleaning equipment in real time, adjusts the cleaning plan, and responds to problems encountered in actual cleaning. The specific process of obtaining real-time cleaning data is as follows;

[0111] The cleaning monitoring submodule monitors the position and progress of the cleaning process based on real-time navigation instructions, and uses image sensor data to detect the working status and effect of foreign object cleaning personnel and equipment in real time. Through image recognition technology, the cleaning effect is evaluated and the cleaning plan is adjusted in real time. If the image data shows that the cleaning effect does not meet expectations, the plan adjustment algorithm immediately generates new cleaning instructions, with the formula "adjusted plan = current plan + adjustment demand × response speed", to quickly respond to problems encountered in actual cleaning, such as incomplete cleaning or newly discovered foreign objects. Real-time cleaning data is obtained, including the completion status and time of each cleaning point, to provide managers with accurate cleaning progress updates.

[0112] The task confirmation submodule is based on real-time cleaning data. After the cleaning task is completed, it checks the foreign objects in the designated cleaning area, identifies whether there are any omissions of foreign objects, verifies the completion and accuracy of the cleaning task, and obtains the specific process of the cleaning task completion information as follows;

[0113] The task confirmation submodule is based on real-time cleaning data. After the cleaning task is completed, it checks for foreign objects in the designated cleaning area. Using image recognition technology, the cleaned area is carefully checked to identify whether there are any missed foreign objects. The inspection formula is "missing identification = image analysis × sensitivity", where image analysis relies on high-resolution images, and sensitivity is adjusted according to the size and material of the foreign objects. The verification of completion and accuracy is carried out by comparing the image data of the area before and after cleaning to ensure that all designated foreign objects have been completely removed. The generated cleaning task completion information includes a detailed status report of the cleaning area and the issues that need to be addressed. It ensures the safe use of the airport runway and prevents any safety hazards caused by incomplete cleaning of foreign objects.

[0114] See also Fig. 9 A multi-sensor based airport runway foreign object detection method, the multi-sensor based airport runway foreign object detection method is performed based on the multi-sensor based airport runway foreign object detection system, comprising the following steps:

[0115] S1: Based on the millimeter-wave radar and image sensor data of the airport runway, foreign object feature capture and positioning are performed to generate basic object coordinates;

[0116] S2: Based on the basic object coordinates, the light intensity sensor and temperature and humidity sensor data are used to analyze the impact of the environment on sensor performance, adjust the coordinates of foreign objects, optimize the errors caused by environmental changes, and generate optimized and corrected object coordinates;

[0117] S3: Based on the optimized and corrected object coordinates, the environmental sensitivity index is calculated using the current environmental change data, the detection threshold is adjusted to match the current light and temperature conditions, and the adjusted detection parameters are obtained;

[0118] S4: Based on the adjusted detection parameters and optimized corrected object coordinates, assess the threat level of foreign objects, analyze the size, shape and reflection characteristics of foreign objects, conduct flight safety impact assessment, and generate priority decision results;

[0119] S5: Based on the priority decision results and the optimized corrected object coordinates, the cleaning path is calculated, a safe path that avoids the flight area is selected, the airport runway usage status and ground conditions are analyzed, and an optimized cleaning path is generated;

[0120] S6: Based on the optimized cleaning path, guide the foreign object cleaning personnel to perform the cleaning task, update the navigation information in real time, match the site condition changes and the foreign objects identified in real time, and obtain the cleaning navigation instructions;

[0121] S7: Based on the cleaning navigation instructions, monitor the cleaning quality and progress, conduct cleaning area inspections, identify the removal of foreign objects, and generate cleaning task completion information.

[0122] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A multi-sensor based airport runway foreign object detection system, characterized in that: The system comprises: The foreign object detection module identifies and locates foreign objects on the airport runway based on the millimeter-wave radar and image sensor data of the airport runway, and obtains the basic object coordinates; The environment analysis module adjusts the position deviation of the basic object coordinates based on the basic object coordinates through the data of the light intensity sensor and the temperature and humidity sensor to generate optimized and corrected object coordinates; A threshold adjustment module, based on the optimized and corrected object coordinates and combined with the current airport runway environment change data, adjusts the detection threshold to obtain optimized detection parameters; A priority judgment module, based on the optimized and corrected object coordinates, performs threat level assessment on foreign objects on the airport runway, assesses foreign object removal priorities, and obtains a priority decision result; A path planning module calculates a foreign object cleaning path based on the optimized corrected object coordinates and priority decision results, combined with real-time airport runway usage status and ground condition data, and adjusts the path to optimize the processing efficiency of foreign objects, thereby generating an optimized cleaning path; A navigation and execution module guides foreign object cleaning personnel to perform cleaning operations based on the optimized cleaning path, updates the cleaning progress in real time, matches foreign objects that appear immediately, and generates cleaning task completion information; The environmental analysis module includes: The environmental monitoring submodule collects the illumination, temperature and humidity data of the current airport runway based on the basic object coordinates through the illumination intensity sensor and the temperature and humidity sensor, analyzes the impact of environmental parameters on the performance of the millimeter wave radar and the image sensor, and generates environmental impact information; A data correction submodule, based on the environmental impact information, applies historical environmental data, compares and corrects the data deviation of the current millimeter wave radar and image sensor, optimizes the accuracy of environmental monitoring, and obtains environmental correction information; The coordinate adjustment submodule adjusts the position deviation of the foreign object based on the environmental correction information and according to real-time weather changes, optimizes the accuracy of the foreign object coordinates, matches the current environmental conditions, and generates optimized and corrected object coordinates.

2. The multi-sensor based airport runway foreign object detection system according to claim 1, characterized in that: The basic object coordinates include the object's geographic location, size and shape information, the optimized and corrected object coordinates include the adjusted object's geographic location, environmental impact correction value and timestamp, the optimized detection parameters include the adjusted sensitivity level, environmental adaptation threshold and image clarity index, the priority decision result includes the urgency of handling the foreign object, the impact rating and the recommended processing time, the optimized cleaning path includes the path starting point and end point coordinates, the estimated cleaning time and the safety assessment level, and the cleaning task completion information includes the cleaning area identification, completion time and cleaning quality assessment.

3. The multi-sensor based airport runway foreign object detection system according to claim 1, characterized in that: The foreign object detection module comprises: The feature comparison submodule analyzes the captured images and radar signals based on the millimeter-wave radar and image sensor data of the airport runway, compares the reflection characteristic differences and image contour information in the two data, distinguishes foreign objects from the airport runway background, and obtains foreign object feature data; An image optimization submodule, based on the foreign object feature data, adjusts the focal length and shooting angle of the image sensor to optimize image capture, and processes the image to optimize the clarity and accuracy of the image to obtain an adjusted image; The coordinate positioning submodule identifies the geographical location of the foreign object based on the adjusted image, calculates the coordinate points of the foreign object on the runway, and obtains the basic object coordinates by analyzing the millimeter wave radar and image sensor data.

4. The multi-sensor based airport runway foreign object detection system according to claim 1, characterized in that: The threshold adjustment module comprises: The parameter calculation submodule calculates the influence of various environmental factors on the detection performance of the millimeter wave radar and the image sensor based on the optimized and corrected object coordinates and the current airport runway environment change data, identifies the detection threshold that needs to be adjusted, and obtains the environmental sensitivity index data; A threshold optimization submodule analyzes the existing detection thresholds of the millimeter-wave radar and the image sensor based on the environmental sensitivity index data, adjusts and optimizes the detection thresholds of the millimeter-wave radar and the image sensor according to the data fluctuation characteristics under the current light and temperature conditions, optimizes the threshold settings to match the environmental changes, and generates detection threshold adjustment parameters; The sensitivity update submodule updates the sensitivity settings of the millimeter-wave radar and the image sensor based on the detection threshold adjustment parameters, adjusts the sensitivity of the millimeter-wave radar and the image sensor to match the actual environment, optimizes the accuracy and response speed of foreign object recognition, and obtains optimized detection parameters.

5. The multi-sensor based airport runway foreign object detection system according to claim 1, characterized in that: The priority determination module comprises: A threat assessment submodule, based on the optimized and corrected object coordinates, analyzes the size, shape and material reflection characteristics of each foreign object, identifies the potential hazard level of each foreign object according to the physical characteristics of the foreign object and the position of the foreign object on the runway, and generates an object threat level; The impact analysis submodule analyzes the impact of foreign objects on the safety of aircraft takeoff and landing based on the threat level of the object, including the possibility of foreign objects damaging aircraft tires and causing flight delays. The impact of foreign objects is evaluated by simulating multiple scenarios to obtain flight safety impact assessment results. The priority sorting submodule, based on the flight safety impact assessment results, adopts a multi-attribute decision analysis method to analyze the clearance priorities of various foreign objects, and sorts the foreign object handling priorities according to the risks and impacts of the foreign objects, combined with the runway use frequency and the location of the foreign objects, to obtain a priority decision result.

6. The airport runway foreign object detection system based on multiple sensors according to claim 5 is characterized in that: The multi-attribute decision analysis method is according to the formula: Calculate the priority value for foreign object handling, where A priority value for foreign object processing, For the The weight of the attribute, For the The standardized value of the attribute, Score the impact of foreign objects on the flight's proximity, Score the impact of foreign objects on runway efficiency, is the total number of attributes, is the weight coefficient of temporal proximity, is the weight coefficient of runway utilization efficiency.

7. The airport runway foreign object detection system based on multiple sensors according to claim 1 is characterized in that: The path planning module includes: A path calculation submodule, based on the optimized corrected object coordinates and priority decision results, uses real-time airport runway usage status and ground condition data, combines flight areas and ground obstacles, analyzes the cleaning path from the current position to each foreign object position, and generates a basic cleaning route; A safety assessment submodule, based on the basic cleaning route, checks multiple areas that the cleaning route passes through, assesses the distance between the cleaning route and the flight operation area, ground obstacles and risk factors, adjusts the cleaning path to avoid risk areas, optimizes the safety of the cleaning operation, and generates a safety verification path; The path adjustment submodule adjusts the path length and steering angle based on the safety verification path, optimizes the time and distance for cleaning foreign objects, analyzes the impact of the foreign object cleaning process on the flight, optimizes the distribution and sequence of path points, and generates an optimized cleaning path.

8. A multi-sensor based airport runway foreign object detection method, characterized in that: The airport runway foreign object detection system based on multiple sensors according to any one of claims 1 to 7 comprises the following steps: Based on the millimeter-wave radar and image sensor data of the airport runway, foreign object features are captured and located to generate basic object coordinates; Based on the basic object coordinates, the influence of the environment on the sensor performance is analyzed through the data of the light intensity sensor and the temperature and humidity sensor, the foreign object coordinates are adjusted, and the optimized and corrected object coordinates are generated; Based on the optimized and corrected object coordinates, using the current environmental change data, calculating the environmental sensitivity index, adjusting the detection threshold, and obtaining the adjusted detection parameters; Based on the adjusted detection parameters and the optimized corrected object coordinates, assessing the threat level of the foreign object and generating a priority decision result; Based on the priority decision results and the optimized corrected object coordinates, a clearing path is calculated, a safe path that avoids the flight area is selected, and an optimized clearing path is generated; Based on the optimized cleaning path, foreign object cleaning personnel are guided to perform cleaning tasks, navigation information is updated in real time, and cleaning navigation instructions are obtained by matching site condition changes and instantly identified foreign objects; Based on the cleaning navigation instructions, the cleaning quality and progress are monitored, the removal of foreign objects is identified, and cleaning task completion information is generated.

Images