An airport intelligent safety monitoring and early warning method
By normalizing the RGB channels and dynamically compressing the range of RGB images of airport pavement, combined with multi-scale saliency map analysis of red-green and yellow-blue contrast channels and a dual-channel mask-guided neural network, the false alarm problem caused by texture and lighting changes in airport pavement detection is solved, and efficient and accurate detection of small foreign objects is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHENGDU SHUANGLIU INT AIRPORT
- Filing Date
- 2026-05-08
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies for airport pavement inspection are easily affected by the inherent texture of the pavement and changes in lighting, resulting in low inspection accuracy.
By employing RGB channel normalization and dynamic range compression, red-green and yellow-blue contrast channel images are extracted, and two-dimensional discrete Fourier transform is performed to generate multi-scale contrast color saliency maps. The relative deviation coefficient is calculated and the saliency maps are fused. A dual-channel mask guides the neural network for defect detection.
It significantly improves the detection accuracy of small foreign objects, effectively suppresses pavement background interference, and enhances the accuracy and robustness of detection.
Smart Images
Figure CN122157171A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, specifically to an intelligent security monitoring and early warning method for airports. Background Technology
[0002] Small foreign objects scattered on airport pavements (including runways, taxiways, and aprons) have always been a major risk source of concern in the field of aviation safety. These objects may originate from bolts or rivets left behind during aircraft maintenance, or metal parts that fall off during baggage handling. Although they are usually small in size, during high-speed takeoffs, landings, and taxiing, such foreign objects can easily cause serious accidents such as engine blade damage or tire blowouts if they are sucked into the engine or run over by the tires. Therefore, efficient and accurate detection and early warning of small foreign objects on the pavement are a key link in the airport safety monitoring system.
[0003] Existing technologies utilize high-definition RGB cameras to continuously acquire pavement images. By employing background subtraction (comparing the current frame pixel-by-pixel with a pre-stored clean pavement background template) or inter-frame subtraction (analyzing brightness differences between adjacent frames), the foreground region of suspected foreign objects is extracted. Subsequently, by combining grayscale thresholding, morphological filtering (such as opening operations to remove isolated noise points), and simple screening rules based on geometric features such as area and aspect ratio, candidate targets are located and alarms are triggered.
[0004] However, airport pavement surfaces commonly exhibit tire tracks, worn-out road markings, and shadows caused by changes in lighting. These disturbances, during differential processing, can manifest grayscale or edge features similar to real foreign objects, causing the system to easily misidentify non-foreign objects such as tire tracks and stains as hazardous targets. Therefore, existing technologies are susceptible to interference from the inherent texture of the pavement and changes in lighting, resulting in low detection accuracy. Summary of the Invention
[0005] To address the aforementioned shortcomings in existing technologies, this invention provides an intelligent airport safety monitoring and early warning method that solves the problem of low detection accuracy caused by the inherent texture of the pavement and changes in lighting.
[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is: an intelligent airport security monitoring and early warning method, comprising the following steps: S1. Normalize each channel of the airport pavement RGB image, and then perform range dynamic compression to obtain a dynamically compressed image. S2. Extract the red-green contrast channel image and the yellow-blue contrast channel image from the dynamically compressed image, and perform two-dimensional discrete Fourier transform on each image to extract the multi-scale amplitude spectrum residuals and obtain the multi-scale contrast color saliency map. S3. Generate an energy saliency map based on the multi-scale contrastive color saliency map, calculate the relative deviation coefficient of the pixels, and fuse the multi-scale information to obtain a fused saliency map; S4. Screen candidate outliers from the fused saliency map and construct a risk candidate mask; S5. A dual-channel mask-guided neural network is used to process the red-green contrast channel image, the yellow-blue contrast channel image, and their respective risk candidate masks to obtain the defect detection results of the airport pavement, and risk warnings are given based on the defect detection results.
[0007] Furthermore, S1 includes the following sub-steps: S11. Normalize each channel of the airport pavement RGB image to obtain normalized R, G and B channel images; S12. On each normalized channel plot, the difference between the maximum channel value and the minimum channel value is taken as the original channel range. S13. Calculate the ratio of the preset target range to the original range of the channel, and constrain the ratio to the minimum value of 1 to obtain the dynamic scaling ratio of the corresponding channel. S14. Use dynamic scaling to compress and correct the channel range of the corresponding channel, and superimpose a fixed dark area compensation offset to obtain the channel value after dynamic compression. S15. Arrange the dynamically compressed channel values according to the corresponding pixels to obtain the corresponding channel compressed image. Combine the three channel compressed images to form a dynamically compressed image.
[0008] Furthermore, S2 includes the following sub-steps: S21. Extract the red-green contrast channel image and the yellow-blue contrast channel image from the dynamically compressed image; S22. Perform two-dimensional discrete Fourier transform on the two comparison channel images respectively to obtain two spectrum images; S23. Extract the amplitude spectrum and phase spectrum from the spectrogram, and perform logarithmic processing on the amplitude spectrum to obtain the logarithmic amplitude spectrum; S24. Taking each frequency point in the logarithmic amplitude spectrum as the center, calculate the mean logarithmic amplitude of the multi-scale neighborhood at the center, and take the mean logarithmic amplitude of the corresponding scale as the new logarithmic amplitude of the frequency point at the center to obtain the multi-scale background amplitude spectrum. S25. Subtract the background amplitude spectrum at each scale from the logarithmic amplitude spectrum to obtain the amplitude spectrum residual at the corresponding scale; S26. Reconstruct the amplitude spectrum residuals and phase spectra of each scale belonging to the same contrast channel image to obtain a multi-scale contrastive color saliency map of the same contrast channel image.
[0009] Furthermore, S3 includes the following sub-steps: S31. Squaring the pixel values of the contrasting color saliency map at each scale yields the contrasting color energy saliency map at the corresponding scale. S32. Take the average of each energy value on the saliency map of the opposing color energy at each scale to obtain the global average energy. S33. On the same scale of the contrasting color energy saliency map, calculate the relative deviation coefficient of the pixel based on the difference between each energy value and the global average energy. S34. Based on the multi-scale contrastive color energy saliency map and the relative deviation coefficient, the fusion saliency map is obtained.
[0010] Furthermore, the formula for calculating the relative deviation coefficient of pixels in S33 is as follows: , in, For the first Significance of opposing color energies at each scale The relative deviation coefficient of the position, For the first Significance of opposing color energies at each scale The energy value of the location, For the first Significance of opposing color energies at each scale The corresponding global average energy, || is used for absolute value operations. The x-coordinate of the pixel is The x-coordinate of the pixel is The scale number; The expression for obtaining the fused saliency map in S34 is: , in, To merge saliency maps in The significance of the location, Select the maximum value for the same pixel location at three scales.
[0011] Furthermore, S4 includes the following sub-steps: S41. Take the average of all saliency values on the fused saliency map to obtain the global saliency mean; S42. Mark pixels with a saliency value greater than the global saliency mean as candidate outliers on the fused saliency map; S43. Set the significance value of isolated candidate outliers to 0, set the significance value of the remaining isolated candidate outliers to 1, and set the significance value of the remaining positions to 0 to obtain the risk candidate mask.
[0012] Furthermore, the dual-channel mask-guided neural network in S5 includes: a red-green mask-guided enhancement channel, a yellow-blue mask-guided enhancement channel, an intensity competition fusion module, a multi-scale receptive field module, and an output unit; S5 includes the following steps: S51. In the red-green mask-guided enhancement channel, the feature map of the red-green contrast channel is enhanced by using the red-green risk candidate mask, and then the feature map is obtained by residual connection. S52. In the yellow-blue mask-guided enhancement channel, the feature map of the yellow-blue contrast channel is enhanced by using the yellow-blue risk candidate mask, and then the feature map is obtained by residual connection. S53. The red-green mask-guided enhancement feature map and the yellow-blue mask-guided enhancement feature map are weighted using the intensity competition fusion module to obtain the mask-guided enhancement feature fusion map; S54. A multi-scale receptive field module is used to extract multi-scale feature maps from the mask-guided enhanced feature fusion map. S55. The output unit outputs the defect detection results of the airport pavement based on multi-scale feature maps.
[0013] Furthermore, the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel have the same structure, both including: a first convolutional block, a second convolutional block, a third convolutional block, a multiplier M1, and an adder A1; The input of the first convolutional block serves as the first input of the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel, and its output is connected to the input of the second convolutional block and the first input of adder A1, respectively. The input of the third convolutional block serves as the second input of the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel; The first input of multiplier M1 is connected to the output of the second convolution block, its second input is connected to the output of the third convolution block, and its output is connected to the second input of adder A1. The output of adder A1 serves as the output of the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel.
[0014] Furthermore, the expression for the intensity competition fusion module is: , , in, The first feature map output by the intensity competition fusion module The position of the first The eigenvalues of the channel, The feature map output by the red-green mask-guided enhancement feature map is the first... The L2 norm of all channels at each position. The first feature map output by the yellow-blue mask-guided enhancement feature map. The L2 norm of all channels at each position. It is a natural constant. The feature map output by the red-green mask-guided enhancement feature map is the first... The proportion of feature intensity at each location For location number, The feature map output by the red-green mask-guided enhancement feature map is the first... The position of the first The eigenvalues of the channel, The first feature map output by the yellow-blue mask-guided enhancement feature map. The position of the first The characteristic values of the channel.
[0015] Furthermore, the multi-scale receptive field module includes: a first dilated convolutional block, a second dilated convolutional block, a third dilated convolutional block, and a Concat layer; The input of the first dilated convolutional block is connected to the input of the second dilated convolutional block and the input of the third dilated convolutional block, respectively, and serves as the input of the multi-scale receptive field module. The input of the Concat layer is connected to the output of the first dilated convolution block, the output of the second dilated convolution block, and the output of the third dilated convolution block, respectively.
[0016] The beneficial effects of this invention are as follows: 1. This invention first significantly reduces the impact of uneven lighting and exposure deviations on image contrast through RGB channel normalization and dynamic range compression. Then, by introducing saliency analysis based on visually contrasting color channels (red-green, yellow-blue) and combining it with multi-scale amplitude spectrum residual extraction, it effectively suppresses common non-foreign object interference on airport pavements, such as tire marks, worn-out road markings, and shadows. These interferences are easily misidentified as foreground foreign objects in conventional difference methods, but this invention utilizes color contrast and frequency domain residual characteristics to significantly reduce their response in the saliency map, thereby significantly improving detection accuracy.
[0017] 2. This invention uses a multi-scale method to extract salient features at different spatial scales, and then fuses them into an energy saliency map and a relative deviation coefficient. This can enhance the response capability to small, low-contrast foreign objects (such as small bolts, rivets, and metal scraps) and avoid them being submerged or missed by the background due to their small size or low contrast.
[0018] 3. This invention uses a dual-channel mask to guide the neural network to enhance the red-green and yellow-blue contrast channel images and their risk candidate masks, which can automatically focus on the real foreign object area, suppress the pavement background and interfering textures, achieve accurate detection and risk warning, and improve the detection accuracy. Attached Figure Description
[0019] Figure 1 A flowchart of an intelligent security monitoring and early warning method for airports; Figure 2 A schematic diagram of a dual-channel mask-guided neural network; Figure 3 Schematic diagrams of red-green mask-guided enhancement channels and yellow-blue mask-guided enhancement channels; Figure 4 This is a schematic diagram of the structure of a multi-scale receptive field module; Figure 5 This is a schematic diagram of the output unit. Detailed Implementation
[0020] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0021] like Figure 1 As shown, an intelligent security monitoring and early warning method for airports includes the following steps: S1. Normalize each channel of the airport pavement RGB image, and then perform range dynamic compression to obtain a dynamically compressed image. S2. Extract the red-green contrast channel image and the yellow-blue contrast channel image from the dynamically compressed image, and perform two-dimensional discrete Fourier transform on each image to extract the multi-scale amplitude spectrum residuals and obtain the multi-scale contrast color saliency map. S3. Generate an energy saliency map based on the multi-scale contrastive color saliency map, calculate the relative deviation coefficient of the pixels, and fuse the multi-scale information to obtain a fused saliency map; S4. Screen candidate outliers from the fused saliency map and construct a risk candidate mask; S5. A dual-channel mask-guided neural network is used to process the red-green contrast channel image, the yellow-blue contrast channel image, and their respective risk candidate masks to obtain the defect detection results of the airport pavement, and risk warnings are given based on the defect detection results.
[0022] In this embodiment, S1 includes the following sub-steps: S11. Normalize each channel of the airport pavement RGB image to obtain normalized R, G and B channel images; S12. On each normalized channel plot, the difference between the maximum channel value and the minimum channel value is taken as the original channel range. S13. Calculate the ratio of the preset target range to the original range of the channel, and apply a minimum constraint between this ratio and the value 1 to obtain the dynamic scaling ratio of the corresponding channel: ,in, for The dynamic scaling ratio of the channel. To select the minimum value, To set a target range, for The original channel range was very poor. A constant used to avoid a denominator of zero, , Corresponding to the R channel, Corresponding to the G channel, Corresponding to channel B; S14. Compress and correct the channel range of the corresponding channel using a dynamic scaling ratio, and overlay a fixed dark area compensation bias to obtain the dynamically compressed channel values: ,in, RGB image of airport pavement The dynamically compressed channel value of the location. RGB image of airport pavement After position normalization Channel value, Minimum after normalization Channel value, Offset for dark area compensation. The x-coordinate of the pixel is The ordinate of the pixel; S15. Arrange the dynamically compressed channel values according to their corresponding pixels to obtain the corresponding channel compressed image. Combine the three channel compressed images to form a dynamically compressed image, which includes: the R channel compressed image. G-channel compression diagram and B channel compression map .
[0023] In this embodiment, the Min-Max normalization method is used in S11. The preset target range is set to 0.25 to balance the preservation of the contrast of real foreign objects and the suppression of the interference of the road surface background. The dark area compensation bias is set to 0.1 to prevent the compressed image from being too dark and to preserve the weak information of the shadow area.
[0024] This invention achieves unified pixel value distribution across the RGB three channels by independently normalizing each channel, eliminating baseline differences in brightness between channels and mitigating overall color shift caused by variations in natural light intensity and shooting angle. Furthermore, based on the contrast ratio of each channel (dynamic scaling), it selectively compresses high-contrast areas (such as extreme changes caused by strong lighting or exposure deviations) while fully preserving low-contrast areas (such as foreign objects or subtle textures in shadows). Simultaneously, a fixed dark area compensation bias is applied to prevent loss of dark area information. This effectively suppresses the interference of uneven lighting, shadows, and exposure deviations on image contrast while retaining key differences between real foreign objects and the background.
[0025] In this embodiment, S2 includes the following sub-steps: S21. Extract the red-green contrast channel image and the yellow-blue contrast channel image from the dynamically compressed image; S22. Perform two-dimensional discrete Fourier transform on the two comparison channel images respectively to obtain two spectrum images; S23. Extract the amplitude spectrum and phase spectrum from the spectrogram, and perform logarithmic processing on the amplitude spectrum to obtain the logarithmic amplitude spectrum: ,in, The amplitude spectrum is logarithmic. For amplitude spectrum, For the frequency index in the horizontal direction, For frequency indexing in the vertical direction, It is a logarithmic function; S24. Taking each frequency point in the logarithmic amplitude spectrum as the center, calculate the mean logarithmic amplitude of the multi-scale neighborhood at the center, and take the mean logarithmic amplitude of the corresponding scale as the new logarithmic amplitude of the frequency point at the center to obtain the multi-scale background amplitude spectrum. S25. Subtract the background amplitude spectrum at each scale from the logarithmic amplitude spectrum to obtain the amplitude spectrum residual at the corresponding scale; S26. Reconstruct the amplitude spectrum residuals and phase spectra of each scale belonging to the same contrast channel image to obtain a multi-scale contrastive color saliency map of the same contrast channel image.
[0026] In this embodiment, the expression for the red-green contrast channel image is: , in, Red-green contrast channel chart The pixel value of the location, For dynamically compressed images The R channel value of the location, For dynamically compressed images The G channel value of the location.
[0027] The expression for the yellow-blue contrast channel chart is: , in, Yellow-blue contrast channel image The pixel value of the location, For dynamically compressed images The B channel value of the location.
[0028] In this embodiment, the multi-scale neighborhood range includes: 3×3, 5×5, and 7×7. S24 specifically includes: taking each frequency point in the logarithmic amplitude spectrum as the center, calculating the logarithmic amplitude mean of the 3×3 neighborhood range at the center, and using the logarithmic amplitude mean of 3×3 as the new logarithmic amplitude of the frequency point at the center to obtain the background amplitude spectrum corresponding to 3×3; taking each frequency point in the logarithmic amplitude spectrum as the center, calculating the logarithmic amplitude mean of the 5×5 neighborhood range at the center, and using the logarithmic amplitude mean of 5×5 as the new logarithmic amplitude of the frequency point at the center to obtain the background amplitude spectrum corresponding to 5×5; taking each frequency point in the logarithmic amplitude spectrum as the center, calculating the logarithmic amplitude mean of the 7×7 neighborhood range at the center, and using the logarithmic amplitude mean of 7×7 as the new logarithmic amplitude of the frequency point at the center to obtain the background amplitude spectrum corresponding to 7×7.
[0029] In this embodiment, the reconstruction formula in S26 is: , in, For the first A scale-dependent color saliency map For the first Amplitude spectrum residuals at each scale, The time corresponds to 3×3. The time corresponds to 5×5. The time corresponds to 7×7. For phase spectrum, The imaginary unit, It is a natural constant. This is the inverse Fourier transform.
[0030] This invention introduces visually contrasting color channels of red-green (RG) and yellow-blue ((R+G)-B) to significantly distinguish between real foreign objects (with clear color contrast) and inherent pavement textures such as tire marks and worn road markings (with weak color contrast). Combined with frequency domain amplitude spectrum residual analysis, it utilizes the characteristic that background textures exhibit regular and predictable spectral components in the frequency domain, while randomly occurring foreign objects exhibit anomalous mutation components. By subtracting the multi-scale background amplitude spectrum from the logarithmic amplitude spectrum, regular background interference is effectively stripped away, allowing real foreign objects to obtain a high response in the saliency map, while the response of regular textures such as tire marks and road markings is significantly suppressed.
[0031] This invention employs multi-scale neighborhoods (3×3, 5×5, 7×7) to calculate the background amplitude spectrum, enabling the simultaneous capture of frequency domain features at different spatial scales: small-scale neighborhoods are sensitive to minute foreign objects, while large-scale neighborhoods are sensitive to larger foreign objects. By reconstructing the residuals of the amplitude spectrum at each scale separately, it ensures that significant residual responses can be obtained at at least one scale for both small targets such as bolts and rivets and larger targets, avoiding omissions due to mismatched target sizes.
[0032] In this embodiment, S3 includes the following sub-steps: S31. Squaring the pixel values of the contrasting color saliency map at each scale yields the contrasting color energy saliency map at the corresponding scale. S32. Take the average of each energy value on the saliency map of the opposing color energy at each scale to obtain the global average energy. S33. On the same scale of the contrasting color energy saliency map, calculate the relative deviation coefficient of the pixel based on the difference between each energy value and the global average energy. S34. Based on the multi-scale contrastive color energy saliency map and the relative deviation coefficient, the fusion saliency map is obtained.
[0033] In this embodiment, the formula for calculating the relative deviation coefficient of a pixel in step S33 is as follows: , in, For the first Significance of opposing color energies at each scale The relative deviation coefficient of the position, For the first Significance of opposing color energies at each scale The energy value of the location, For the first Significance of opposing color energies at each scale The corresponding global average energy, || is used for absolute value operations. The x-coordinate of the pixel is The x-coordinate of the pixel is The scale number; The expression for obtaining the fused saliency map in S34 is: , in, To merge saliency maps in The significance of the location, Select the maximum value for the same pixel location at three scales.
[0034] This invention achieves contrast stretching by performing a square transformation on the contrast saliency maps at various scales and using nonlinear mapping to make the "stronger become stronger and the weaker become weaker". This further enhances the energy of the high-response region corresponding to the real foreign object, while effectively suppressing the low-response region such as background noise, thereby expanding the separability of foreign objects and background in the energy domain.
[0035] This invention calculates the global average energy of the energy saliency map at each scale and introduces a relative deviation coefficient formula to map the absolute energy deviation to the [0,1) interval. This makes the saliency of each pixel no longer depend on the absolute energy value, but rather on its relative deviation from the global average energy. Then, it employs... The maximum values at three scales are selected to ensure that foreign objects of any size can be retained in the final fusion saliency map as long as they obtain a high energy response and relative deviation coefficient at a scale that matches their size.
[0036] This invention takes at each pixel position This achieves the goal of "selecting the value with the strongest response after weighting bias and energy among the three scales as the final significance," thus ensuring that foreign objects of different sizes can be detected at their most suitable scale. and This is combined with further suppression of background interference.
[0037] In this embodiment, S4 includes the following sub-steps: S41. Take the average of all saliency values on the fused saliency map to obtain the global saliency mean; S42. Mark pixels with a saliency value greater than the global saliency mean as candidate outliers on the fused saliency map; S43. Remove isolated points with no other candidate anomalies in the 8-neighborhood (set the significance value to 0), set the significance value of the remaining candidate anomalies to 1, and set the significance value of non-candidate points to 0, to obtain the risk candidate mask.
[0038] The risk candidate mask obtained through the red-green contrast channel diagram is named the red-green risk candidate mask, and the risk candidate mask obtained through the yellow-blue contrast channel diagram is named the yellow-blue risk candidate mask.
[0039] like Figure 2 As shown, the dual-channel mask-guided neural network in S5 includes: a red-green mask-guided enhancement channel, a yellow-blue mask-guided enhancement channel, an intensity competition fusion module, a multi-scale receptive field module, and an output unit; The first input of the red-green mask-guided enhancement channel is used to input the red-green contrast channel image, and its second input is used to input the red-green risk candidate mask. The first input terminal of the yellow-blue mask-guided enhancement channel is used to input the yellow-blue contrast channel map, and the second input terminal is used to input the yellow-blue risk candidate mask; The input of the intensity competition fusion module is connected to the output of the red-green mask-guided enhancement channel and the output of the yellow-blue mask-guided enhancement channel, respectively, and its output is connected to the input of the multi-scale receptive field module. The output of the multi-scale receptive field module is connected to the input of the output unit, and the output of the output unit serves as the output of the dual-channel mask-guided neural network.
[0040] In this embodiment, S5 includes the following sub-steps: S51. In the red-green mask-guided enhancement channel, the feature map of the red-green contrast channel is enhanced by using the red-green risk candidate mask, and then the feature map is obtained by residual connection. S52. In the yellow-blue mask-guided enhancement channel, the feature map of the yellow-blue contrast channel is enhanced by using the yellow-blue risk candidate mask, and then the feature map is obtained by residual connection. S53. The red-green mask-guided enhancement feature map and the yellow-blue mask-guided enhancement feature map are weighted using the intensity competition fusion module to obtain the mask-guided enhancement feature fusion map; S54. A multi-scale receptive field module is used to extract multi-scale feature maps from the mask-guided enhanced feature fusion map. S55. The output unit outputs the defect detection results of the airport pavement based on multi-scale feature maps.
[0041] This invention enhances the feature maps of corresponding contrast channel images, enabling the neural network to focus on candidate anomaly regions while suppressing feature responses in non-candidate regions. This effectively shields background interference such as pavement printing and marking wear, significantly improving the model's accuracy in identifying real foreign objects. After mask enhancement, residual connections are introduced to superimpose the enhanced feature map with the original feature map. This ensures that subtle textures and edge information in the original contrast channel image are not excessively suppressed, highlighting candidate regions while preserving global contextual information. This enhances the model's feature representation capability and prevents over-reliance on mask information. Further weighting considers both weighted and yellow-blue feature information. When the foreign object is color-biased, the red-green channel is emphasized; when the foreign object is brightness / warm / cool contrast-biased, the yellow-blue channel is emphasized. This achieves full coverage detection of foreign objects with different color characteristics, avoiding missed detections or feature weakening due to fixed channel preferences. Finally, a multi-scale receptive field module is used to extract features of different scales from the fused feature map, enabling the model to simultaneously capture foreign object features of different sizes, such as small bolts, rivets, and larger metal fragments, thereby enhancing the model's robustness to size changes.
[0042] like Figure 3 As shown, the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel have the same structure, both including: a first convolutional block, a second convolutional block, a third convolutional block, a multiplier M1, and an adder A1; The input of the first convolutional block serves as the first input of the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel, and its output is connected to the input of the second convolutional block and the first input of adder A1, respectively. The input of the third convolutional block serves as the second input of the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel; The first input of multiplier M1 is connected to the output of the second convolution block, its second input is connected to the output of the third convolution block, and its output is connected to the second input of adder A1. The output of adder A1 serves as the output of the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel.
[0043] In this embodiment, the kernel size of the first, second, and third convolutional blocks is 3×3, the stride is 1, the padding is 1, and the output channels are 32. The image size of the red-green risk candidate mask, the red-green contrast channel image, the yellow-blue contrast channel image, and the yellow-blue risk candidate mask is H×W. The feature map size output by the first convolutional block is H×W×32, the feature map size output by the second convolutional block is H×W×32, and the feature map size output by the third convolutional block is H×W×32, where H is the height and W is the width.
[0044] The first convolutional block extracts shallow features from the contrast channel map, which are further abstracted by the second convolutional block and then multiplied and modulated with the mask features. Simultaneously, the output of the first convolutional block is directly connected to the adder via a residual path, superimposed on the modulated features. This preserves the integrity of the original contrast channel map while incorporating mask guidance information. Multiplier M1 multiplies the mask features element-wise with the feature map output from the second convolutional block, enhancing features in candidate anomaly regions and suppressing features in non-candidate regions. This allows the neural network to focus on potential foreign object locations and effectively shield against pavement background interference. Adder A1 directly adds the modulated features to the original output of the first convolutional block, forming a residual connection structure. This ensures smooth gradient flow during backpropagation (avoiding gradient vanishing in deep networks) and prevents the subtle textures and edge information in the original contrast channel map from being excessively suppressed by the mask modulation process.
[0045] In this embodiment, the expression for the intensity competition fusion module is: , , in, The first feature map output by the intensity competition fusion module The position of the first The eigenvalues of the channel, The feature map output by the red-green mask-guided enhancement feature map is the first... The L2 norm of all channels at each position. The first feature map output by the yellow-blue mask-guided enhancement feature map. The L2 norm of all channels at each position. It is a natural constant. The feature map output by the red-green mask-guided enhancement feature map is the first... The proportion of feature intensity at each location For location number, The feature map output by the red-green mask-guided enhancement feature map is the first... The position of the first The eigenvalues of the channel, The first feature map output by the yellow-blue mask-guided enhancement feature map. The position of the first The eigenvalues of the channel, , ,in, This refers to the channel number on the feature map. This represents the total number of channels on the feature map.
[0046] The feature map size output by the intensity competition fusion module is H×W×32.
[0047] like Figure 4 As shown, the multi-scale receptive field module includes: a first dilated convolutional block, a second dilated convolutional block, a third dilated convolutional block, and a Concat layer; The input of the first dilated convolutional block is connected to the input of the second dilated convolutional block and the input of the third dilated convolutional block, respectively, and serves as the input of the multi-scale receptive field module. The input of the Concat layer is connected to the output of the first dilated convolution block, the output of the second dilated convolution block, and the output of the third dilated convolution block, respectively.
[0048] The first dilated convolutional block has a 3×3 kernel size, a dilation rate of 1, a stride of 2, a padding of 1, and 32 output channels. The second dilated convolutional block has a 3×3 kernel size, a dilation rate of 2, a stride of 2, a padding of 2, and 32 output channels. The third dilated convolutional block has a 3×3 kernel size, a dilation rate of 3, a stride of 2, a padding of 3, and 32 output channels. All three dilated convolutional blocks consist of a dilated convolutional layer, a batch normalization layer, and a ReLU activation layer.
[0049] This invention sets up three parallel dilated convolution branches with void ratios of 1, 2, and 3, respectively. The small receptive field branch is sensitive to micro-foreign objects such as small bolts and metal fragments, while the large receptive field branch can capture the contextual information of larger rubber blocks or foreign object clusters. The three branches work in parallel and complement each other to achieve full coverage detection of foreign objects of different scales.
[0050] The feature map size output by the first, second, and third dilated convolutional blocks is H / 2×W / 2×32, and the feature map size output by the Concat layer is H / 2×W / 2×96.
[0051] like Figure 5 As shown, the output unit comprises, in sequence: an upsampling layer, a fourth convolutional block, a fifth convolutional block, and a sixth convolutional block. The fourth convolutional block has a kernel size of 3×3, a stride of 1, padding of 1, and 64 output channels. The fifth convolutional block has a kernel size of 3×3, a stride of 1, padding of 1, and 32 output channels. The sixth convolutional block has a kernel size of 3×3, a stride of 1, padding of 1, and 1 output channel.
[0052] The first, second, third, fourth, and fifth convolutional blocks each consist of a convolutional layer, a batch normalization layer, and a ReLU activation layer in sequence. The sixth convolutional block consists of a convolutional layer, a batch normalization layer, and a Sigmoid activation layer.
[0053] In this embodiment, a safety risk warning signal is generated and pushed out based on the size and distribution location of the airport pavement defect detection results generated by the output unit.
[0054] In step S1, this invention adaptively suppresses uneven illumination and exposure deviation by performing channel-specific normalization and dynamic range compression on the RGB image, thus reducing interference from inherent pavement textures while preserving the true contrast of foreign objects. In step S2, red-green and yellow-blue contrasting color channels are introduced and combined with multi-scale amplitude spectrum residual analysis to transform the detection from a susceptible grayscale space to a more discriminative color contrasting space. Utilizing the characteristic that foreign objects appear as abrupt changes in a regular background in the frequency domain, regular background responses such as offset printing and markings are effectively removed. In steps S3 and S4, square transformations are used to enhance the foreign object response, calculate the relative deviation coefficient to achieve a globally adaptive threshold, and filter out isolated noise points, generating an accurate risk candidate mask. Finally, in step S5, this mask guides a neural network to perform mask enhancement and intensity competition fusion on the dual-channel features, enabling the model to focus on true candidate regions. This solves the false alarm problem caused by texture similarity in traditional difference methods, significantly improving the accuracy and robustness of small foreign object detection on airport pavements.
[0055] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for intelligent security monitoring and early warning at airports, characterized in that, Includes the following steps: S1. Normalize each channel of the airport pavement RGB image, and then perform range dynamic compression to obtain a dynamically compressed image. S2. Extract the red-green contrast channel image and the yellow-blue contrast channel image from the dynamically compressed image, and perform two-dimensional discrete Fourier transform on each image to extract the multi-scale amplitude spectrum residuals and obtain the multi-scale contrast color saliency map. S3. Generate an energy saliency map based on the multi-scale contrastive color saliency map, calculate the relative deviation coefficient of the pixels, and fuse the multi-scale information to obtain a fused saliency map; S4. Screen candidate outliers from the fused saliency map and construct a risk candidate mask; S5. A dual-channel mask-guided neural network is used to process the red-green contrast channel image, the yellow-blue contrast channel image, and their respective risk candidate masks to obtain the defect detection results of the airport pavement, and risk warnings are given based on the defect detection results.
2. The airport intelligent security monitoring and early warning method according to claim 1, characterized in that, S1 includes the following steps: S11. Normalize each channel of the airport pavement RGB image to obtain normalized R, G and B channel images; S12. On each normalized channel plot, the difference between the maximum channel value and the minimum channel value is taken as the original channel range. S13. Calculate the ratio of the preset target range to the original range of the channel, and constrain the ratio to the minimum value of 1 to obtain the dynamic scaling ratio of the corresponding channel. S14. Use dynamic scaling to compress and correct the channel range of the corresponding channel, and superimpose a fixed dark area compensation offset to obtain the channel value after dynamic compression. S15. Arrange the dynamically compressed channel values according to the corresponding pixels to obtain the corresponding channel compressed image. Combine the three channel compressed images to form a dynamically compressed image.
3. The airport intelligent security monitoring and early warning method according to claim 1, characterized in that, S2 includes the following steps: S21. Extract the red-green contrast channel image and the yellow-blue contrast channel image from the dynamically compressed image; S22. Perform two-dimensional discrete Fourier transform on the two comparison channel images respectively to obtain two spectrum images; S23. Extract the amplitude spectrum and phase spectrum from the spectrogram, and perform logarithmic processing on the amplitude spectrum to obtain the logarithmic amplitude spectrum; S24. Taking each frequency point in the logarithmic amplitude spectrum as the center, calculate the mean logarithmic amplitude of the multi-scale neighborhood at the center, and take the mean logarithmic amplitude of the corresponding scale as the new logarithmic amplitude of the frequency point at the center to obtain the multi-scale background amplitude spectrum. S25. Subtract the background amplitude spectrum at each scale from the logarithmic amplitude spectrum to obtain the amplitude spectrum residual at the corresponding scale; S26. Reconstruct the amplitude spectrum residuals and phase spectra of each scale belonging to the same contrast channel image to obtain a multi-scale contrastive color saliency map of the same contrast channel image.
4. The airport intelligent security monitoring and early warning method according to claim 1, characterized in that, S3 includes the following steps: S31. Squaring the pixel values of the contrasting color saliency map at each scale yields the contrasting color energy saliency map at the corresponding scale. S32. Take the average of each energy value on the saliency map of the opposing color energy at each scale to obtain the global average energy. S33. On the same scale of the contrasting color energy saliency map, calculate the relative deviation coefficient of the pixel based on the difference between each energy value and the global average energy. S34. Based on the multi-scale contrastive color energy saliency map and the relative deviation coefficient, the fusion saliency map is obtained.
5. The airport intelligent security monitoring and early warning method according to claim 4, characterized in that, The formula for calculating the relative deviation coefficient of pixels in S33 is: ， in, For the first Significance of opposing color energies at each scale The relative deviation coefficient of the position, For the first Significance of opposing color energies at each scale The energy value of the location, For the first Significance of opposing color energies at each scale The corresponding global average energy, || is used for absolute value operations. The x-coordinate of the pixel is The x-coordinate of the pixel is The scale number; The expression for obtaining the fused saliency map in S34 is: ， in, To merge saliency maps in The significance of the location, Select the maximum value for the same pixel location at three scales.
6. The airport intelligent security monitoring and early warning method according to claim 1, characterized in that, S4 includes the following steps: S41. Take the average of all saliency values on the fused saliency map to obtain the global saliency mean; S42. Mark pixels with a saliency value greater than the global saliency mean as candidate outliers on the fused saliency map; S43. Set the significance value of isolated candidate outliers to 0, set the significance value of the remaining isolated candidate outliers to 1, and set the significance value of the remaining positions to 0 to obtain the risk candidate mask.
7. The airport intelligent security monitoring and early warning method according to claim 1, characterized in that, The dual-channel mask-guided neural network in S5 includes: a red-green mask-guided enhancement channel, a yellow-blue mask-guided enhancement channel, an intensity competition fusion module, a multi-scale receptive field module, and an output unit; S5 includes the following steps: S51. In the red-green mask-guided enhancement channel, the feature map of the red-green contrast channel is enhanced by using the red-green risk candidate mask, and then the feature map is obtained by residual connection. S52. In the yellow-blue mask-guided enhancement channel, the feature map of the yellow-blue contrast channel is enhanced by using the yellow-blue risk candidate mask, and then the feature map is obtained by residual connection. S53. The red-green mask-guided enhancement feature map and the yellow-blue mask-guided enhancement feature map are weighted using the intensity competition fusion module to obtain the mask-guided enhancement feature fusion map; S54. A multi-scale receptive field module is used to extract multi-scale feature maps from the mask-guided enhanced feature fusion map. S55. The output unit outputs the defect detection results of the airport pavement based on multi-scale feature maps.
8. The airport intelligent security monitoring and early warning method according to claim 7, characterized in that, The red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel have the same structure, both including: a first convolutional block, a second convolutional block, a third convolutional block, a multiplier M1, and an adder A1; The input of the first convolutional block serves as the first input of the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel, and its output is connected to the input of the second convolutional block and the first input of adder A1, respectively. The input of the third convolutional block serves as the second input of the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel; The first input of multiplier M1 is connected to the output of the second convolution block, its second input is connected to the output of the third convolution block, and its output is connected to the second input of adder A1. The output of adder A1 serves as the output of the red-green mask-guided enhancement channel and the yellow-blue mask-guided enhancement channel.
9. The airport intelligent security monitoring and early warning method according to claim 7, characterized in that, The expression for the intensity contention fusion module is: ， ， in, The first feature map output by the intensity competition fusion module The position of the first The eigenvalues of the channel, The feature map output by the red-green mask-guided enhancement feature map is the first... The L2 norm of all channels at each position. The first feature map output by the yellow-blue mask-guided enhancement feature map. The L2 norm of all channels at each position. It is a natural constant. The feature map output by the red-green mask-guided enhancement feature map is the first... The proportion of feature intensity at each location For location number, The feature map output by the red-green mask-guided enhancement feature map is the first... The position of the first The eigenvalues of the channel, The first feature map output by the yellow-blue mask-guided enhancement feature map. The position of the first The characteristic values of the channel.
10. The airport intelligent security monitoring and early warning method according to claim 7, characterized in that, The multi-scale receptive field module includes: a first dilated convolutional block, a second dilated convolutional block, a third dilated convolutional block, and a Concat layer; The input of the first dilated convolutional block is connected to the input of the second dilated convolutional block and the input of the third dilated convolutional block, respectively, and serves as the input of the multi-scale receptive field module. The input of the Concat layer is connected to the output of the first dilated convolution block, the output of the second dilated convolution block, and the output of the third dilated convolution block, respectively.