Foreign Object Detection Method for Large Hangar Door Operation Area Based on Multi-Scale Feature Fusion

Abstract

This invention belongs to the field of computer vision and industrial safety monitoring technology, specifically relating to a foreign object detection method for large hangar door operation areas based on multi-scale feature fusion. The method includes the following steps: S1, acquiring real-time grayscale images of the hangar door operation area floor, preprocessing the real-time grayscale images, and using the Laplacian pyramid algorithm to perform multi-scale decomposition on the preprocessed images to construct a feature pyramid containing multiple feature maps; S2, using the camera's installation geometric parameters, establishing a mapping relationship between the row coordinates of the preprocessed image and the physical scale, and calculating the perspective scale scaling factor at different row coordinates of the preprocessed image. This invention, by reducing the weight of high-frequency layers and utilizing low-frequency structural layers, naturally filters high-frequency texture noise such as oil stains and scratches on the ground, effectively reducing the false alarm rate. Furthermore, it has low computational complexity and can be directly deployed on low-power industrial control computers, reducing engineering implementation costs.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and industrial safety monitoring technology, specifically relating to a method for detecting foreign objects in the work area of ​​a large hangar door based on multi-scale feature fusion. Background Technology

[0002] Large aircraft hangar doors are critical equipment in aircraft maintenance and parking facilities, typically with spans ranging from tens to hundreds of meters and immense weight, operating along ground-mounted guide rails. In practical engineering applications, foreign objects in the ground-mounted guide rail area are a major safety hazard causing doors to derail, jam, or even damage aircraft tires. Therefore, real-time and accurate foreign object detection in the hangar door operating area is of paramount safety importance.

[0003] Currently, foreign object detection in such scenarios mainly relies on wide-angle cameras installed on the top or side of the door for overhead monitoring. Existing technical solutions typically employ general target detection algorithms or traditional background subtraction methods. However, existing technologies have significant limitations in hangar door scenarios: First, due to the high installation height of the cameras and their extremely deep field of view, objects of the same physical size appear very small at the far end of the image due to perspective projection effects, making them easily filtered out by fixed-scale filters. Second, at the near end of the image, background textures on the ground (such as oil stains and scratches) appear large, easily misidentified as large foreign objects. Finally, existing multi-scale feature fusion techniques typically use uniform or learning-based fixed weights across the entire image, failing to dynamically adjust the weights of different frequency band features based on the actual depth position of pixels in physical space, resulting in an inability to balance the detection of small targets at the far end with the suppression of texture interference at the near end.

[0004] To address the aforementioned issues, there is an urgent need for a foreign object detection method that can adapt to scenes with large depth of field and effectively overcome perspective distortion. Summary of the Invention

[0005] This invention provides a method for detecting foreign objects in the work area of ​​a large hangar door based on multi-scale feature fusion, in order to solve the technical problems of missed detection of small targets at a distance and false alarms due to background texture at a near distance caused by the long line of sight in the prior art.

[0006] In a first aspect, the present invention provides a method for detecting foreign objects in the work area of ​​a large hangar door based on multi-scale feature fusion, comprising the following steps:

[0007] S1. Acquire real-time grayscale images of the hangar door operation area floor, preprocess the real-time grayscale images, and use the Laplacian pyramid algorithm to perform multi-scale decomposition on the preprocessed images to construct a feature pyramid containing multi-layer feature maps.

[0008] S2, using the camera's mounting geometry parameters, establishes a mapping relationship between the row coordinates of the preprocessed image and the physical scale, calculates the perspective scale scaling factor at different row coordinates of the preprocessed image, and constructs a spatial adaptive weight field corresponding to each layer of feature map accordingly.

[0009] S3 utilizes a spatial adaptive weight field to perform pixel-level weighted fusion of feature maps at each layer of the feature pyramid, generating a foreign object saliency map that eliminates the influence of perspective scale.

[0010] S4 performs dual-threshold segmentation on the foreign object saliency map, and combines the preset orbital region mask and connected component filtering strategy to determine the location of the foreign object and output the detection results.

[0011] Furthermore, the perspective scale scaling factor at different row coordinates of the preprocessed image is calculated, satisfying the following formula:

[0012]

[0013] in, Indicates the image after preprocessing. The perspective scale scaling factor corresponding to the row pixels; This represents the row coordinate of a pixel in the preprocessed image, with a value range of [value missing]. 0 represents the first row of pixels at the top of the preprocessed image. This represents the last row of pixels at the bottom of the preprocessed image. This indicates the vertical height of the camera lens center from the ground; This represents the equivalent focal length parameter of the camera. This indicates the pitch angle between the camera's optical axis and the horizontal ground. This represents the normalization adjustment coefficient; This represents a pre-defined non-zero positive integer, used to prevent the denominator from being zero.

[0014] Furthermore, the feature is that a spatial adaptive weight field corresponding to each layer of feature maps is constructed, satisfying the following formula:

[0015]

[0016] in, Indicates the first Layer feature maps in image rows The fusion weight at the location; Indicates the first The preset optimal response scale center corresponding to the layer feature map; Indicates the first Scale tolerance of layer weights; Represents extremely small positive numbers. Indicates the image after preprocessing. The perspective scale scaling factor corresponding to the row pixels.

[0017] Furthermore, pixel-level weighted fusion is performed on the feature maps of each layer in the feature pyramid, satisfying the following formula:

[0018]

[0019] in, Coordinates in the graph representing the salience of foreign objects Pixel value at; Indicates the number of layers in the feature pyramid; Indicates the first Layer feature map in coordinates Pixel value at; Indicates the first Layer feature maps in image rows The fusion weight at the location; This represents a fixed, non-zero small constant used to ensure numerical stability. Indicates the first Layer feature maps in image rows The fusion weight at the location.

[0020] Furthermore, the real-time grayscale image is preprocessed, including: adaptive histogram equalization of the acquired real-time grayscale image to enhance image contrast; and smoothing the equalized image using a Gaussian filter to remove random noise generated by the sensor.

[0021] Furthermore, the foreign object saliency map is segmented using a dual threshold method, including:

[0022] Set a high threshold and low threshold ;

[0023] Pixel value higher The pixels were confirmed as strong foreign objects;

[0024] Set pixel value between and Pixels that are between and contain strong foreign objects in their neighborhood are identified as weak foreign objects and retained; otherwise, they are discarded.

[0025] Furthermore, combining preset orbital region masks and connected component filtering strategies, including:

[0026] The binary image after double threshold segmentation is ANDed with the pre-generated track region mask to filter out the detection results outside the track region.

[0027] Perform connected component analysis on the image after AND operation, and calculate the area and aspect ratio of each connected component;

[0028] Remove connected blocks whose area is smaller than a preset area threshold or whose aspect ratio exceeds a preset ratio range, and retain the remaining connected blocks as foreign objects.

[0029] Furthermore, in S1, a feature map set containing three layers is constructed. ,in, For the highest resolution feature map, It is a medium-resolution feature map. This is the lowest resolution feature map.

[0030] Furthermore, the pitch angle between the camera's optical axis and the horizontal ground is 0.5 radians.

[0031] Furthermore, The value range is 0.001-0.05.

[0032] The beneficial effects are as follows: This invention, by introducing a perspective scale scaling factor, mathematically establishes a correlation between image depth and feature scale. At the far end of the image, the algorithm automatically assigns extremely high weights to the high-frequency detail layer, ensuring that even tiny foreign objects occupying only a few pixels are clearly preserved in the saliency map, significantly improving the sensitivity of long-distance detection. At the near end of the image, by reducing the weights of the high-frequency layer and utilizing the low-frequency structure layer, high-frequency texture noise such as oil stains and scratches on the ground is naturally filtered out, effectively reducing the false alarm rate. Furthermore, this solution involves only basic image transformations and algebraic operations, resulting in low computational complexity. It can be directly deployed on low-power industrial control computers, reducing engineering implementation costs. Attached Figure Description

[0033] Figure 1 This is a flowchart of a foreign object detection method for the operation area of ​​a large hangar door based on multi-scale feature fusion.

[0034] Figure 2 This is a schematic diagram illustrating the trend of perspective scale scaling factor as image depth changes.

[0035] Figure 3 This is a schematic diagram showing the distribution of feature fusion weights at different depths.

[0036] Figure 4 Comparison of foreign object detection response intensity at different viewing distances. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] An embodiment of the foreign object detection method for large hangar door operation areas based on multi-scale feature fusion provided by the present invention:

[0039] like Figure 1 As shown, the method for detecting foreign objects in the work area of ​​a large hangar door based on multi-scale feature fusion includes the following steps:

[0040] S1: Acquire real-time grayscale images of the hangar door operation area floor, preprocess the real-time grayscale images, and use the Laplacian pyramid algorithm to perform multi-scale decomposition on the preprocessed images to construct a feature pyramid containing multi-layer feature maps.

[0041] In this embodiment, an industrial-grade wide-angle camera is first installed on the top or side bracket of the hangar door to acquire real-time grayscale images of the work area floor. Due to vibrations and changes in lighting in the hangar environment, it is necessary to... Preprocessing: First, adaptive histogram equalization is performed to enhance image contrast, followed by... A Gaussian filter is used for smoothing to remove random noise generated by the sensor, resulting in a preprocessed image. .

[0042] Furthermore, in order to separate the features of foreign objects of different physical sizes, the Laplace pyramid algorithm is used. Perform multi-scale decomposition. Construct a system containing... Layer (in this embodiment) Feature map set .in, The (bottom layer) is the highest resolution feature map, which mainly preserves the high-frequency components of the image, corresponding to tiny details in the image, such as small screws at the far end and edge contours; (Middle layer) is a medium-resolution feature map, preserving the mid-frequency components; The top layer is the lowest resolution feature map, which mainly preserves the low-frequency components of the image, corresponding to large-scale structural information, such as near-end ground color patches and shadow areas.

[0043] By constructing a feature pyramid, image information is decomposed into sub-images of different frequency bands, enabling subsequent steps to select the most suitable feature frequency bands for processing targets at different distances, thus laying the foundation for solving the scale distortion problem.

[0044] S2 uses the camera's mounting geometry parameters to establish a mapping relationship between the row coordinates of the preprocessed image and the physical scale, calculates the perspective scale scaling factor at different row coordinates of the preprocessed image, and constructs a spatial adaptive weight field corresponding to each layer of feature map.

[0045] In this embodiment, since the camera's installation position relative to the ground is fixed, the vertical coordinate of each row of pixels in the preprocessed image directly corresponds to the physical distance to the ground. Therefore, based on the principle of perspective, it is necessary to perform a feature map analysis for each layer. Construct a row coordinate system based on the preprocessed image Variable spatial weights .

[0046] First, construct the perspective scale scaling factor. This metric is used to evaluate the first element in the image. The scaling factor of the physical scale represented by a row of pixels is calculated using the following formula:

[0047]

[0048] in, These are the row coordinates of the pixels in the preprocessed image, with values ​​ranging from 1 to 2. 0 represents the first row of pixels at the top of the preprocessed image. This represents the last row of pixels at the bottom of the preprocessed image. The vertical height of the camera lens center from the ground; This refers to the equivalent focal length parameter of the camera; The pitch angle between the camera's optical axis and the horizontal ground; This is the normalized adjustment coefficient; These are preset non-zero positive numbers used to prevent the denominator from being zero.

[0049] To explain the calculation process of this formula more clearly, a specific calculation example is given below:

[0050] Assuming the camera is installed at a certain height meters, equivalent focal length meters, pitch angle radian, , Preset parameters , .

[0051] Scenario 1: Calculate the scaling factor at the far end of the image, i.e., the first row of pixels at the top of the preprocessed image. ;

[0052]

[0053] Scenario 2: Calculate the scaling factor for the near end of the image, i.e., the last row of pixels at the bottom of the preprocessed image. .

[0054]

[0055] Calculations show that the scaling factor at the near end is much larger than that at the far end. This is consistent with the physical law that things appear larger when they are closer and smaller when they are farther away. In other words, the physical size represented by pixels at the near end is smaller, or the same physical object occupies more pixels at the near end and has a larger scaling factor.

[0056] Based on the aforementioned perspective scaling factor, the feature map of each layer is calculated. At different depths Fusion weights at the point The formula is as follows:

[0057]

[0058] in, For the first The preset optimal response scale center corresponding to the layer feature map, for example, is set. , ; For the first Scale tolerance of layer weights; It is a very small positive number, for example, such as 0.001.

[0059] Based on the calculation example above, at the remote end... hour, ,near ,therefore A value close to 1 indicates that the remote end primarily relies on... The layer, i.e., the high-frequency layer; in the near end, i.e. hour, ,near ,therefore A value close to 1 indicates that the proximal end is primarily dependent on... The layer is the low-frequency layer.

[0060] By constructing a spatially adaptive weight field, dynamic allocation of feature weights for different depth regions of the image is achieved, ensuring that high-frequency features are used to capture details at the far end and low-frequency features are used to suppress texture at the near end.

[0061] S3 utilizes a spatial adaptive weight field to perform pixel-level weighted fusion of feature maps at each layer of the feature pyramid, generating a foreign object saliency map that eliminates the influence of perspective scale.

[0062] In this embodiment, the spatial variation weight field generated in step S2 is used to perform pixel-level weighted superposition of multi-scale feature maps to generate the final foreign object saliency map. The calculation formula is as follows:

[0063]

[0064] in, For the first Layer feature map in coordinates Pixel value at; This is the sum of the weights of all feature layers at the current position; It is a fixed, non-zero small constant, for example, such as 0.01.

[0065] Introduction This is to prevent, in certain extreme situations, if It falls precisely in the gap between the response intervals of the two feature layers, causing the weights of all layers to... Both are extremely small, and the denominator approaches 0, thus causing numerical instability. This ensures the stability of the normalization process.

[0066] This step enables automatic feature selection and balancing: in the upper part of the image, (Detail layer) has a high weighting, making small foreign objects at a distance visible; in the lower part of the image, The (structural layer) has a large weight ratio, and because High-frequency textures are filtered out, which automatically suppresses high-frequency noise such as scratches and oil stains on the near-side ground.

[0067] By using weighted normalization fusion, a saliency map with uniform illumination and feature response was generated, effectively eliminating the interference of perspective effect on the detection results.

[0068] S4 performs dual-threshold segmentation on the foreign object saliency map, and combines the preset orbital region mask and connected component filtering strategy to determine the location of the foreign object and output the detection results.

[0069] In this embodiment, firstly... Perform dual threshold segmentation, setting a higher threshold. and low threshold Pixel values ​​higher than The point was directly identified as a strong foreign object; due to and If a point between two points has a strong foreign object in its neighborhood, it is retained; otherwise, it is discarded. This method can effectively preserve the weak edges of foreign objects.

[0070] Then, based on the fixed position of the hangar door guide rail in the preprocessed image, a binary mask image is generated in advance. The segmentation results are compared with Perform AND operations to forcibly filter out all detection results outside the guide rail area, eliminating interference from non-operational areas such as walkways and aprons.

[0071] Finally, connected component analysis is performed on the final binary image to calculate the area and aspect ratio of each connected component. Regions with excessively small areas (noise) or extremely abnormal aspect ratios (such as thin, elongated rays) are removed, and the center coordinates of the remaining connected components are output as the location of the foreign object, triggering an alarm.

[0072] By employing a dual-threshold segmentation and post-processing strategy, the accuracy of the detection results was further improved, ensuring the reliability of the alarm signal.

[0073] like Figure 2 As shown in the figure, this diagram illustrates the perspective scale scaling factor as a function of the image's vertical coordinate. The curve showing the change, with the horizontal axis representing the vertical coordinate of the image, from... arrive The vertical axis represents the physical scale mapping magnification. The blue solid line in the figure shows that on the left side of the image, the scaling factor is extremely low and changes gradually, corresponding to the physical fact that distant objects are imaged very small; as the coordinates move to the right, the curve rises sharply and non-linearly, corresponding to the phenomenon that near objects are imaged very large. This curve demonstrates that the geometric model constructed in this invention can accurately quantify the degree of perspective distortion in a hangar door scene.

[0074] like Figure 3 As shown in the figure, this graph illustrates the distribution of weights at different feature levels with image depth. The horizontal axis represents the vertical coordinate of the image, and the vertical axis represents the normalized feature fusion weights. The red solid line (lower-level high-frequency features) reaches its peak on the left side of the image, indicating that the algorithm primarily utilizes high-frequency details to capture small foreign objects at distant locations. The orange dashed line (middle-level structural features) peaks in the middle of the image, indicating that this layer is mainly responsible for transition detection in mid-range regions, filling the feature gap between distant and near locations and ensuring detection continuity. The green dotted line (top-level low-frequency features) reaches its peak on the right side of the image, indicating that the algorithm primarily utilizes low-frequency information at near locations, thereby suppressing magnified ground texture noise at close range.

[0075] like Figure 4 As shown in the figure, this graph compares the detection performance of the present invention and existing technologies at different distances. The horizontal axis represents the physical distance from the camera, and the vertical axis represents the response intensity of foreign object features; the higher the value, the easier it is to detect. The gray dashed line represents the existing technology, whose response intensity drops significantly on the left, indicating a tendency to miss distant foreign objects. The purple solid line represents the present invention, whose response intensity remains consistently high across the entire line of sight. The light purple filled area in the figure visually demonstrates the significant improvement in long-distance detection capability of the present invention compared to existing technologies.

[0076] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting foreign objects in the operating area of ​​a large hangar door based on multi-scale feature fusion, characterized in that, Includes the following steps: S1. Acquire real-time grayscale images of the hangar door operation area floor, preprocess the real-time grayscale images, and use the Laplacian pyramid algorithm to perform multi-scale decomposition on the preprocessed images to construct a feature pyramid containing multi-layer feature maps. S2, using the camera's mounting geometry parameters, establish the mapping relationship between the row coordinates of the preprocessed image and the physical scale, and calculate the perspective scale scaling factor at different row coordinates of the preprocessed image, satisfying the following formula: in, Indicates the image after preprocessing. The perspective scale scaling factor corresponding to the row pixels; This represents the row coordinate of a pixel in the preprocessed image, with a value range of [value missing]. 0 represents the first row of pixels at the top of the preprocessed image. This represents the last row of pixels at the bottom of the preprocessed image. This indicates the vertical height of the camera lens center from the ground; This represents the equivalent focal length parameter of the camera. This indicates the pitch angle between the camera's optical axis and the horizontal ground. This represents the normalization adjustment coefficient; This represents a pre-defined non-zero positive integer, used to prevent the denominator from being zero; Based on this, a spatial adaptive weight field corresponding to each layer of feature maps is constructed, satisfying the following formula: in, Indicates the first Layer feature maps in image rows The fusion weight at the location; Indicates the first The preset optimal response scale center corresponding to the layer feature map; Indicates the first Scale tolerance of layer weights; Represents extremely small positive numbers; S3 represents an exponential function with the natural constant as the base; using a spatially adaptive weight field, pixel-level weighted fusion is performed on the feature maps of each layer in the feature pyramid to generate a foreign object saliency map that eliminates the influence of perspective scale. S4 performs dual-threshold segmentation on the foreign object saliency map, and combines the preset orbital region mask and connected component filtering strategy to determine the location of the foreign object and output the detection results.

2. The method for detecting foreign objects in the operating area of ​​a large hangar door based on multi-scale feature fusion according to claim 1, characterized in that, Pixel-level weighted fusion of feature maps from each layer of the feature pyramid is performed, satisfying the following formula: in, Coordinates in the graph representing the salience of foreign objects Pixel value at; Indicates the number of layers in the feature pyramid; Indicates the first Layer feature map in coordinates Pixel value at; Indicates the first Layer feature maps in image rows The fusion weight at the location; This represents a fixed, non-zero small constant used to ensure numerical stability. Indicates the first Layer feature maps in image rows The fusion weight at the location.

3. The method for detecting foreign objects in the operating area of ​​a large hangar door based on multi-scale feature fusion according to claim 1, characterized in that, The real-time grayscale image is preprocessed, including: adaptive histogram equalization of the acquired real-time grayscale image to enhance image contrast; and smoothing the equalized image using a Gaussian filter to remove random noise generated by the sensor.

4. The method for detecting foreign objects in the operating area of ​​a large hangar door based on multi-scale feature fusion according to claim 1, characterized in that, Perform double-threshold segmentation on the foreign object saliency map, including: Set a high threshold and low threshold ; Pixel value higher The pixels were confirmed as strong foreign objects; Set pixel value between and Pixels that are between and contain strong foreign objects in their neighborhood are identified as weak foreign objects and retained; otherwise, they are discarded.

5. The method for detecting foreign objects in the operating area of ​​a large hangar door based on multi-scale feature fusion according to claim 1, characterized in that, Combining preset orbital region masks and connected component filtering strategies, including: The binary image after double threshold segmentation is ANDed with the pre-generated track region mask to filter out the detection results outside the track region. Perform connected component analysis on the image after AND operation, and calculate the area and aspect ratio of each connected component; Remove connected blocks whose area is smaller than a preset area threshold or whose aspect ratio exceeds a preset ratio range, and retain the remaining connected blocks as foreign objects.

6. The method for detecting foreign objects in the operating area of ​​a large hangar door based on multi-scale feature fusion according to claim 1, characterized in that, In S1, a feature map set containing 3 layers is constructed. ,in, For the highest resolution feature map, It is a medium-resolution feature map. This is the lowest resolution feature map.

7. The method for detecting foreign objects in the operating area of ​​a large hangar door based on multi-scale feature fusion according to claim 1, characterized in that, The pitch angle between the camera's optical axis and the horizontal ground is 0.5 radians.

8. The method for detecting foreign objects in the operating area of ​​a large hangar door based on multi-scale feature fusion according to claim 2, characterized in that, The value range is 0.001-0.05.

Images