Method for identifying fixed fods on a ramp based on image recognition technology

Abstract

The application provides a kind of apron fixed FOD identification method based on image recognition technology, it is related to FOD identification technical field, the method comprises: the visible light image of apron area, infrared thermal imaging and laser radar point cloud data are synchronously collected;The collected data is carried out light equalization and geometric correction preprocessing;Fixed FOD is separated from temporary mobile object by dynamic background modeling, the modeling fuses visible light, infrared features and point cloud spatial distribution;Deep learning model is used to fuse multispectral features and carry out classification and positioning, the model embeds channel attention mechanism;Data is processed through edge computing device, and according to the classification result, real-time trigger hierarchical alarm and record log;The parameters of deep learning model are dynamically updated.The application solves the four core problems of "high false negative rate, high false alarm rate, poor environmental adaptability and insufficient real-time performance" in existing FOD detection through the architecture of "three in one" of multi-modal data fusion + adaptive dynamic modeling + edge intelligent computing.

Description

Technical Field

[0001] This invention relates to the field of FOD identification technology, and specifically to a method for identifying fixed FOD on an apron based on image recognition technology. Background Technology

[0002] Foreign Object Debris (FOD) is one of the major threats to aviation safety, especially small objects such as metal parts and plastic fragments fixed on the tarmac that are easily overlooked.

[0003] Current mainstream detection methods generally include manual inspection and single image sensors. Among them, manual inspection is not only inefficient and has a high false alarm rate, but also cannot cover complex weather conditions (such as rain, fog, and nighttime). As for single sensors, they are easily affected by lighting and shadows, making it difficult to reliably identify small or low-contrast FODs. At the same time, conventional deep learning models (such as YOLO and Faster R-CNN) adapted to existing single image sensors have insufficient generalization ability in dynamic and complex scenes.

[0004] Furthermore, existing technologies lack multimodal data fusion mechanisms and cannot effectively distinguish between fixed FODs and temporary moving objects (such as vehicles and people). Summary of the Invention

[0005] In order to solve at least some of the technical problems in the related technologies, the present invention provides a method for identifying fixed FOD on the apron based on image recognition technology.

[0006] To achieve the above objectives, the technical solution adopted by the present invention includes: This invention provides a method for identifying fixed FOD on aprons based on image recognition technology, comprising the following steps: Step S1: Simultaneously acquire visible light images, infrared thermal images, and lidar point cloud data of the apron area; Step S2: Perform illumination equalization and geometric correction preprocessing on the acquired data; Step S3: Separate fixed FODs from temporary moving objects through dynamic background modeling, wherein the modeling integrates visible light, infrared features and point cloud spatial distribution; Step S4: Use a deep learning model to fuse multispectral features and perform classification and localization. The model incorporates a channel attention mechanism. Step S5: Process the data through edge computing devices, trigger hierarchical alarms in real time based on the classification results, and record logs; Step S6: Dynamically update the deep learning model parameters.

[0007] Optionally, in step S3, the dynamic background modeling employs a joint probability model: In the formula, Indicates that given a feature vector Background under the condition The probability density function, This represents the number of Gaussian distributions in the Gaussian mixture model and , Indicates the first The weight coefficients are distributed according to a Gaussian distribution and satisfy the following conditions: , Indicated by For the mean, Let the probability density function of the Gaussian distribution of the covariance matrix be . For the first The mean of a Gaussian distribution, For the first The covariance matrix of a Gaussian distribution. Represents the pixel feature vector, derived from visible light image features. and infrared thermal imaging features Composition, that is , This represents the weighting coefficient of the spatially uniform distribution term in the point cloud. Represents the spatially uniform distribution term of the point cloud and , Let V be the volume of the space containing the point cloud.

[0008] Optionally, in step S3, the exclusion of the temporary moving object satisfies the following condition: In the formula, Indicates the length of the observation time window. Indicates time The coordinate set of the collected lidar point cloud, Indicates time The coordinate set of the collected lidar point cloud, Indicates time interval, This is the displacement threshold.

[0009] Optionally, in step S4, the calculation of the channel attention mechanism includes: In the formula, This represents the channel weight vector generated by the channel attention mechanism. This represents the Sigmoid activation function. This represents the weight matrix of the second fully connected layer. Represents the ReLU activation function. This represents the weight matrix of the first fully connected layer. This indicates a global average pooling operation. This represents the input feature map, i.e., the input to the channel attention module. This represents the feature map after recalibration using the channel attention mechanism. The channel index of the feature map.

[0010] Optionally, the multispectral features are fused using the following formula: In the formula, This represents the fused feature map. This represents a 1×1 convolution operation, used for feature fusion and dimensionality reduction. This represents a feature map extracted from a visible light image. This represents a feature map extracted from an infrared thermal imaging image. This represents a feature map extracted from depth data (converted from LiDAR point clouds).

[0011] Optionally, the total latency of processing data via the edge computing device satisfies: In the formula, This represents the total system latency on the edge computing device. This indicates the time consumed during the image preprocessing stage. This represents the number of floating-point operations required for model inference. This indicates the computing power of the edge device's GPU. This indicates the time consumption of the non-maximum suppression operation.

[0012] Optionally, the model parameters are dynamically updated through online learning: An update is triggered when the false positive rate is greater than 5% for 30 consecutive frames. In the formula, This represents the updated model parameters. Indicates the current model parameters. Indicates the learning rate and , This represents the gradient of the loss function with respect to the model parameters. Represents the loss function. This indicates a newly collected dataset used for online learning.

[0013] Beneficial effects: 1. Through the above technical solution, this invention solves the four core problems of existing FOD detection, namely "high false alarm rate, high false alarm rate, poor environmental adaptability and insufficient real-time performance", by adopting a three-in-one architecture of "multimodal data fusion + adaptive dynamic modeling + edge intelligent computing".

[0014] Specifically, firstly, step S1 of this invention constructs a multispectral-spatial-thermal radiation joint sensing system by synchronizing visible light images, infrared thermal imaging, and lidar point cloud data. Step S2, through illumination equalization and geometric correction preprocessing, eliminates illumination / shadow interference and improves data consistency. This not only effectively achieves all-weather coverage but also significantly enhances anti-interference capabilities.

[0015] Second, step S3 of this invention fuses visible light, infrared, and point cloud spatial distributions through dynamic background modeling to construct a spatiotemporal joint probability model. This not only enables accurate separation of fixed / moving targets but also effectively reduces the false alarm rate.

[0016] Third, through the channel attention mechanism and multispectral feature fusion in step S4 and the dynamic updating of model parameters in step S6, this invention can not only effectively enhance the detection capability of small targets, but also enhance the generalization capability of dynamic scenes.

[0017] Fourth, step S5 of the present invention, through edge computing device processing and hierarchical alarm, can not only achieve millisecond-level response, but also effectively optimize resources.

[0018] 2. Other beneficial effects or advantages of the present invention will be described in detail in the specific embodiments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] in: Figure 1 This is a schematic flowchart of a fixed FOD identification method for aprons based on image recognition technology provided by an exemplary embodiment of the present invention; Figure 2 This is a schematic diagram of a system for implementing the fixed FOD identification method for aprons based on image recognition technology, provided by an exemplary embodiment of the present invention; Figure 3This is a schematic diagram of a multimodal image fusion process provided by an exemplary embodiment of the present invention; Figure 4 This is a schematic diagram illustrating dynamic background modeling and FOD separation provided by an exemplary embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a deep learning model provided in an exemplary embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0022] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0023] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] like Figures 1 to 5 As shown, according to a first aspect of the present invention, a method for identifying fixed FOD on an apron based on image recognition technology is provided, comprising the following steps: Step S1: Simultaneously acquire visible light images, infrared thermal images, and lidar point cloud data of the apron area; Step S2: Perform illumination equalization and geometric correction preprocessing on the acquired data; Step S3: Separate fixed FODs from temporary moving objects through dynamic background modeling, and model and fuse visible light, infrared features and point cloud spatial distribution; Step S4: Use a deep learning model to fuse multispectral features and perform classification and localization, embedding a channel attention mechanism into the model; Step S5: Process the data through edge computing devices, trigger hierarchical alarms in real time based on the classification results, and record logs; Step S6: Dynamically update the deep learning model parameters.

[0025] Through the above technical solutions, this invention solves the four core problems of existing FOD detection—high false alarm rate, high false alarm rate, poor environmental adaptability, and insufficient real-time performance—by adopting a three-in-one architecture of "multimodal data fusion + adaptive dynamic modeling + edge intelligent computing".

[0026] Specifically, firstly, step S1 of this invention constructs a multispectral-spatial-thermal radiation joint sensing system by synchronizing visible light images, infrared thermal imaging, and lidar point cloud data. Step S2 eliminates light / shadow interference and improves data consistency through illumination equalization and geometric correction preprocessing. This not only effectively achieves all-weather coverage (infrared penetrates rain, fog, and dust; lidar ignores changes in illumination; visible light preserves details; the three complement each other to completely solve detection blind spots in complex weather conditions such as nighttime and rain / fog), but also effectively enhances anti-interference capabilities (significantly reducing the impact of sudden changes in illumination, shadows, and reflections on the identification of small / low-contrast FODs through multi-source data correction).

[0027] Second, step S3 of this invention fuses visible light, infrared, and point cloud spatial distributions through dynamic background modeling to construct a spatiotemporal joint probability model. This not only enables accurate separation of fixed / moving targets but also effectively reduces the false alarm rate.

[0028] Third, through the channel attention mechanism in step S4 and multispectral feature fusion, and the dynamic updating of model parameters in step S6, this invention can not only effectively enhance the detection capability of small targets (channel attention can strengthen the thermal radiation features in the infrared channel and the texture features in the visible light channel, thereby improving the detection rate of small FODs such as screws and gravel), but also enhance the generalization capability of dynamic scenes (online learning enables the model to adapt to unknown scenarios such as airport pavement wear, seasonal changes, and new FODs, avoiding the failure problem of the static model of traditional YOLO / Faster R-CNN).

[0029] Fourth, step S5 of the present invention, through edge computing device processing and hierarchical alarm, can not only achieve millisecond-level response (through edge GPU computing power and lightweight preprocessing, the time from data acquisition to alarm trigger is less than 100ms to meet the real-time standard of aviation safety), but also effectively achieve resource optimization (hierarchical alarm to significantly reduce the operation and maintenance load, for example, metal parts → emergency alarm, leaves → log recording).

[0030] In one embodiment of the present invention, in step S3, the dynamic background modeling employs a joint probability model: In the formula, Indicates that given a feature vector Background under the condition The probability density function, This represents the number of Gaussian distributions in the Gaussian mixture model and , Indicates the first The weight coefficients are distributed according to a Gaussian distribution and satisfy the following conditions: , Indicated by For the mean, Let the probability density function of the Gaussian distribution of the covariance matrix be . For the first The mean of a Gaussian distribution, For the first The covariance matrix of a Gaussian distribution. Represents the pixel feature vector, derived from visible light image features. and infrared thermal imaging features Composition, that is , This represents the weighting coefficient of the spatially uniform distribution term in the point cloud. Represents the spatially uniform distribution term of the point cloud and , Let V be the volume of the space containing the point cloud.

[0031] In this implementation, for the calculation formula In particular, These are pixel feature vectors, which are visible light image features. and infrared thermal imaging features The joint feature vector is used to fuse texture and thermal radiation features; It is the number of Gaussian distributions ( These are used to model three states: background, foreground (FOD), and shadow / noise, respectively. The point cloud spatial uniform distribution term is used to characterize the invariance of the spatial position of an object (fixed FOD position remains unchanged). The point cloud constraint weight coefficient (usually taken as 0.3 to 0.5) is used to balance the contributions of spectral features and spatial location.

[0032] In this embodiment, the method of the present invention can not only minimize the false alarm rate and achieve sub-pixel level small target detection and localization, but also effectively maintain robustness in dynamic scenes. In addition, it can ensure computational efficiency and meet real-time requirements.

[0033] The technical principles of this embodiment will be explained below.

[0034] 1. How this invention solves the problem of lighting / shadow interference (a defect of traditional GMM): Traditional Gaussian Mixture Models (GMMs) rely solely on RGB or infrared features, which can easily misclassify shadows as foreground objects in strong shadow conditions. This invention, however, addresses this issue by... Dual-spectral joint modeling, in which visible light Capture texture, infrared Sensing temperature, shadows appear as dark areas in visible light, but their temperature is consistent with the environment → this is represented by the covariance matrix in the joint distribution. The negative correlation allows the probability of the shaded area to be reduced. It is significantly higher than the true FOD, thus reducing the shadow misjudgment rate.

[0035] 2. How does this invention exclude temporary moving objects (i.e., moving objects such as vehicles and people that have similar spectral characteristics to FOD in a single frame image)? This invention introduces point cloud spatial constraint terms. Thus, for a fixed FOD, its position remains unchanged, corresponding to The spatial distribution is consistent across consecutive frames, that is, The value is stable; however, for moving objects, their position changes, and the corresponding... Dramatic fluctuations will occur in consecutive frames, resulting in overall... The probability decreases. Based on this, we can combine the judgment logic (such as the exclusion conditions for temporary moving objects as described below). By excluding point clouds identified as moving objects, false alarms caused by moving objects can be effectively reduced.

[0036] 3. How this invention enhances the detection rate of small-sized FOD: Its essence lies in solving the problem that small FODs (such as bolts) are easily overwhelmed by noise in traditional background modeling due to their low pixel ratio. This invention will... The value is set to A Gaussian distribution is specifically used to model small targets (weights). (Adaptive adjustment), so that the lidar is sensitive to tiny protrusions (e.g., those with a height greater than 0.5 cm are detected), through... This method maps physical size constraints to the image domain to improve the detection rate of small-sized FODs.

[0037] The above implementation method will be described below with reference to a specific application scenario.

[0038] Scenario: Inspection of metal screws on the apron after rain.

[0039] The challenges of the scenario are: water accumulation and reflection interfere with visible light, and low-temperature rainwater reduces infrared contrast.

[0040] The solution proposed in this invention is as follows: First, for the visible light channel... In terms of light equalization, reflections are suppressed; secondly, for the infrared channel... In addition, because metal screws absorb heat quickly, there is a certain temperature difference between them and cold water; furthermore, for point clouds... In terms of screw protrusion height, it is approximately 0.8 cm. The probability of triggering the item is increased.

[0041] Result: Screw It was determined to be a foreground FOD, while the surrounding waterlogged area... It was determined to be the background.

[0042] In one embodiment of the present invention, in step S3, the exclusion of the temporarily moved object satisfies the following condition: In the formula, Indicates the length of the observation time window. Indicates time The coordinate set of the collected lidar point cloud, Indicates time The coordinate set of the collected lidar point cloud, Indicates time interval, This is the displacement threshold.

[0043] In this implementation, for the calculation formula In particular, In time The collected lidar point cloud coordinate set is used to characterize the absolute position of an object in physical space; The observation time window length (usually 30 to 60 seconds) is used to cover the transit cycle of typical moving objects (vehicles / personnel) at the airport; It is the point cloud acquisition time interval, used to balance real-time performance and displacement accuracy; It is the Euclidean distance, used to calculate the maximum spatial displacement (neither point displacement nor average displacement) between point clouds. The displacement threshold is a rigid standard for determining the mobility of an object.

[0044] In this embodiment, the method of the present invention, through the maximum displacement criterion and absolute spatial measurement, can effectively solve the vehicle / personnel interference problem, and at the same time, through point cloud displacement stability ( This ensures that the true FOD is not filtered out incorrectly. Furthermore, the lidar can penetrate rain, fog, and smoke, ensuring that the criteria can still be reliably executed under complex weather conditions. In addition, the algorithm complexity is only O(N), and it can complete the cloud computing of tens of thousands of points within 5ms on an edge GPU, effectively guaranteeing real-time performance.

[0045] The technical principles of this embodiment will be explained below.

[0046] 1. How does this invention solve the interference from moving objects (e.g., vehicles, people, etc.)? In existing technologies, image-based optical flow methods or inter-frame difference methods are prone to misclassifying slowly moving objects (such as luggage carts) as stationary FODs. This invention effectively solves the problem of false positives for moving objects. Specifically, taking vehicles and people as examples, for moving vehicles, the contact point between their tires and the ground is significant, causing the overall tire point cloud displacement to be much greater than the displacement threshold. This allows for precise labeling to exclude vehicles; for moving individuals, footsteps cause foot point cloud displacements far exceeding the displacement threshold. This allows for real-time filtering to exclude individuals.

[0047] 2. How this invention prevents fixed FOD from being mistakenly removed: Strong winds cause objects like leaves or fallen rubber pads to vibrate in FOD (Form of Object) states. Traditional methods may misinterpret these as moving objects due to pixel fluctuations. However, in the method of this invention, the entire displacement of the leaf or rubber pad (not a single point) is considered. Based on the maximum displacement criterion of the present invention (the displacement of a single point may be large, but the maximum displacement of the overall point cloud is still lower than the threshold), it is possible to prevent fixed FODs from being mistakenly cleared.

[0048] 3. How to ensure robustness in detecting small-sized FOD: In existing technologies, when a vehicle passes around a small-sized FOD (e.g., a nut), traditional methods will lose the FOD due to image occlusion. However, in this invention, after identifying a moving object, it can be marked (see the process description of vehicle marking in "How to Solve Moving Objects" above), and its occlusion area can be dynamically masked. In this way, the displacement of the occluded FOD point cloud within the time window remains 0, maintaining its fixed properties, thereby ensuring robustness in detecting small-sized FODs.

[0049] For a comparison of this implementation method with the existing optical flow method, please refer to Table 1 below. Table 1. Comparison of the present invention with optical flow. Comparison Projects Optical flow method Point cloud displacement criterion of the present invention Slowly moving luggage cart Mistakenly identified as stationary (movement speed < threshold) Correctly exclude (wheel displacement > τ) Metal pads in strong winds Misjudged as movement (pixel jitter) Correctly preserve (point cloud displacement < τ) Screws under vehicle Missed detection (features disappear due to occlusion) Continuous tracking (point cloud position remains unchanged) Ground reflection on rainy days Numerous false alarms (optical flow noise) Unaffected (laser penetrates rain and fog) computational complexity High (requires dense matching) Low (point cloud distance calculation can be accelerated in parallel) In one embodiment of the present invention, in step S4, the calculation of the channel attention mechanism includes: In the formula, This represents the channel weight vector generated by the channel attention mechanism. This represents the Sigmoid activation function. This represents the weight matrix of the second fully connected layer. Represents the ReLU activation function. This represents the weight matrix of the first fully connected layer. This indicates a global average pooling operation. This represents the input feature map, i.e., the input to the channel attention module. This represents the feature map after recalibration using the channel attention mechanism. The channel index of the feature map.

[0050] In this implementation, for the calculation formula In terms of, among them The input feature map is the original representation of multimodal features (containing visible light / infrared / depth features). It is a global average pooling operation, which compresses spatial information and generates channel descriptors; These are the weights of the first fully connected layer, and their function is to reduce dimensions and extract the nonlinear relationships between channels; It is the ReLU activation function, which introduces nonlinearity and suppresses invalid channels; These are the weights of the second fully connected layer, whose function is to restore the channel dimension and learn the channel importance. It is the Sigmoid activation function, which normalizes the weights to... The importance of the identification channel; It is the channel weight vector, which quantifies the contribution of each channel to FOD detection; It is a recalibrated feature map, whose function is to enhance the features of key channels and suppress noise channels.

[0051] In this embodiment, the method of the present invention can not only effectively improve the detection rate of small FODs, but also resolve multimodal conflicts and achieve adaptive environment switching.

[0052] Specifically, regarding the improvement of the detection rate of minute FODs, in existing related technologies, bolts with a diameter of less than 2cm are visible in the visible light channel. It occupies only 2×2 pixels, and at the same time, it is in the infrared channel. Because the low emissivity of metals is easily masked by environmental thermal noise, existing technologies have a low detection rate for minute FODs. However, in this invention, in, This is the visible light channel weight vector. This is the infrared channel weight vector. This is the depth channel weight vector.

[0053] Let's take two exemplary application scenarios as examples: Scenario 1: Nighttime metal gasket inspection: Weighting is automatically increased (due to large temperature differences in the environment). Automatically reducing light intensity (insufficient light) enhances infrared signatures and suppresses visible light noise, enabling accurate detection of metal pads at night.

[0054] Scenario 2: Inspection of transparent plastic sheets in rainy weather: Automatic weight boosting (point cloud height anomaly) Automatically reduces (reflective interference) and uses depth features to dominate recognition, thereby achieving accurate detection of transparent plastic sheets in rainy weather.

[0055] Regarding the resolution of multimodal conflicts, existing technologies exhibit that water reflections appear as a bright area in the visible light channel (similar to metallic FOD) but as a low temperature in the infrared channel (the opposite of real metallic FOD), thus creating a multimodal feature conflict. This invention addresses this conflict through an attention mechanism. For example, firstly... By calculating the channel mean, it can be learned that for waterlogged areas, the mean of the visible light channel is high while the mean of the infrared channel is low, while for real metal, the mean of both the visible light channel and the infrared channel is simultaneously high. (Fully connected layer) , This allows the system to learn the conflict pattern and output the average values ​​of the visible light channel and infrared channel with lower synchronization. In this way, the conflict feature is suppressed, and false alarms can be effectively avoided.

[0056] In one embodiment of the present invention, the multispectral features of the present invention are fused using the following formula: In the formula, This represents the fused feature map. This represents a 1×1 convolution operation, used for feature fusion and dimensionality reduction. This represents a feature map extracted from a visible light image. This represents a feature map extracted from an infrared thermal imaging image. This represents a feature map extracted from depth data (converted from LiDAR point clouds).

[0057] In this embodiment, the present invention achieves cross-modal feature interaction by fusing the channel stitching results of visible light features, infrared features, and depth features through 1×1 convolution operations (automatically learning weight allocation through convolution kernels). This allows metal parts to be primarily identified by infrared features in rainy and foggy weather, while transparent plastic fragments are located with enhanced depth features, improving the detection rate of small FODs. Furthermore, by performing only linear combination of channel dimensions (without spatial convolution), the computational load is reduced to 1 / 9 of that of traditional 3×3 convolution, and the edge fusion latency is compressed to <5ms (meeting aviation safety real-time standards). At the same time, by preserving the original resolution of the multimodal features and pixel alignment, conflicting modal weights are automatically suppressed in interference scenarios such as water accumulation and reflection, reducing the false alarm rate. In addition, the depth features provide absolute spatial support, so even if the optical sensor fails (such as blinding due to strong light), the system availability can still be maintained through point cloud height anomalies.

[0058] In one embodiment of the present invention, the total latency of data processing via edge computing devices satisfies: In the formula, This represents the total system latency on the edge computing device. This indicates the time consumed during the image preprocessing stage. This represents the number of floating-point operations required for model inference. This indicates the computing power of the edge device's GPU. This indicates the time consumption of the non-maximum suppression operation.

[0059] In this embodiment, by setting the total latency formula in the edge computing device and constraining the preprocessing latency and NMS operation latency, not only can the total system latency be compressed to less than 25ms, but the end-to-end response triggered by the FOD alarm can also meet relevant standards, thereby reducing the possibility of major accidents caused by FOD.

[0060] In one embodiment of the present invention, the model parameters are dynamically updated through online learning: An update is triggered when the false positive rate is greater than 5% for 30 consecutive frames. In the formula, This represents the updated model parameters. Indicates the current model parameters. Indicates the learning rate and , This represents the gradient of the loss function with respect to the model parameters. Represents the loss function. This indicates a newly collected dataset used for online learning.

[0061] In this implementation, by employing an online learning update strategy and setting a false positive rate > 5% for consecutive frames as the trigger condition, the model can not only automatically adapt to seasonal changes in airport pavement (such as glare from rainwater accumulation and asphalt shrinkage in winter), but also maintain a low false alarm rate during continuous operation. Furthermore, the learning rate... This ensures the model's stable convergence characteristics, thereby reducing the frequency of manual parameter tuning and lowering the difficulty and cost of operation and maintenance.

[0062] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for identifying fixed FOD on aprons based on image recognition technology, characterized in that, Includes the following steps: Step S1: Simultaneously acquire visible light images, infrared thermal images, and lidar point cloud data of the apron area; Step S2: Perform illumination equalization and geometric correction preprocessing on the acquired data; Step S3: Separate fixed FODs from temporary moving objects through dynamic background modeling, wherein the modeling integrates visible light, infrared features and point cloud spatial distribution; Step S4: Use a deep learning model to fuse multispectral features and perform classification and localization. The model incorporates a channel attention mechanism. Step S5: Process the data through edge computing devices, trigger hierarchical alarms in real time based on the classification results, and record logs; Step S6: Dynamically update the deep learning model parameters; In step S3, the dynamic background modeling employs a joint probability model: In the formula, Indicates that given a feature vector Background under the condition The probability density function, This represents the number of Gaussian distributions in the Gaussian mixture model and , Indicates the first The weight coefficients are distributed according to a Gaussian distribution and satisfy the following conditions: , Indicated by For the mean, Let the probability density function of the Gaussian distribution of the covariance matrix be . For the first The mean of a Gaussian distribution, For the first The covariance matrix of a Gaussian distribution. Represents the pixel feature vector, derived from visible light image features. and infrared thermal imaging features Composition, that is , This represents the weighting coefficient of the spatially uniform distribution term in the point cloud. Represents the spatially uniform distribution term of the point cloud and , The volume of the space containing the point cloud; In step S3, the exclusion of the temporary moving object satisfies the following condition: In the formula, Indicates the length of the observation time window. Indicates time The coordinate set of the collected lidar point cloud, Indicates time The coordinate set of the collected lidar point cloud, Indicates time interval, This is the displacement threshold.

2. The method for identifying fixed FOD on an apron based on image recognition technology according to claim 1, characterized in that, In step S4, the calculation of the channel attention mechanism includes: , In the formula, This represents the channel weight vector generated by the channel attention mechanism. This represents the Sigmoid activation function. This represents the weight matrix of the second fully connected layer. Represents the ReLU activation function. This represents the weight matrix of the first fully connected layer. This indicates a global average pooling operation. This represents the input feature map, i.e., the input to the channel attention module. This represents the feature map after recalibration using the channel attention mechanism. The channel index of the feature map.

3. The method for identifying fixed FOD on an apron based on image recognition technology according to claim 1, characterized in that, The multispectral features are fused using the following formula: In the formula, This represents the fused feature map. This represents a 1×1 convolution operation, used for feature fusion and dimensionality reduction. This represents a feature map extracted from a visible light image. This represents a feature map extracted from an infrared thermal imaging image. This represents a feature map extracted from depth data converted from LiDAR point clouds.

4. The method for identifying fixed FOD on an apron based on image recognition technology according to claim 1, characterized in that, The total latency of data processing via edge computing devices satisfies: , , In the formula, This represents the total system latency on the edge computing device. This indicates the time consumed during the image preprocessing stage. This represents the number of floating-point operations required for model inference. This indicates the computing power of the edge device's GPU. This indicates the time consumption of the non-maximum suppression operation.

5. The method for identifying fixed FOD on an apron based on image recognition technology according to claim 1, characterized in that, The model parameters are dynamically updated through online learning. ； An update is triggered when the false positive rate is greater than 5% for 30 consecutive frames. In the formula, This represents the updated model parameters. Indicates the current model parameters. Indicates the learning rate and , This represents the gradient of the loss function with respect to the model parameters. Represents the loss function. This indicates a newly collected dataset used for online learning.

Images