A runway contaminant identification method and system based on high-definition video

Abstract

This application relates to the field of image recognition technology, specifically to a method and system for runway contaminant identification based on high-definition video. The method includes: acquiring a video stream of an airport runway; determining the illumination influence value based on the difference in the number of locally bright and dark pixels in each frame and their distance from the center point, combined with the overall local brightness; determining high-illumination feature values ​​based on the texture feature values ​​of pixels and the correlation between rows and columns; determining medium- and low-illumination feature values ​​based on the gradient Shannon entropy within a neighborhood window and the structural similarity of eight neighborhoods between pixels; obtaining a comprehensive feature value by weighting the high-illumination and medium- and low-illumination feature values ​​using the illumination influence value as a weight; constructing final regularization parameters to complete image preprocessing; and identifying contaminants from the real-time video stream after training a model using a neural network. This application improves the identification effect for runway contaminants.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, specifically to a method and system for identifying runway contaminants based on high-definition video. Background Technology

[0002] With the current development of the aviation industry, runway contaminant identification based on high-definition video is one of the key technologies for ensuring aviation safety. Contaminants such as ice, snow, and frost on airport runway surfaces significantly reduce the friction coefficient, seriously threatening aircraft takeoff and landing safety. Many aviation organizations worldwide have imposed stringent requirements on runway contaminant depth measurement with millimeter-level accuracy. Traditional inspection methods are inefficient, subjective, and lack sufficient precision, making it difficult to meet the efficient and accurate operational needs of modern airports. Therefore, the current trend towards automated and intelligent high-definition video recognition technology is of paramount importance for improving the real-time performance and accuracy of runway condition assessment and ensuring flight safety.

[0003] High-definition video-based runway contaminant identification typically targets the detection of water film coverage on airport runways. During this process, it's crucial to consider that in airport runway scenarios, the acquired high-definition video often shows cyclical runway lighting, potentially leading to areas of excessive brightness or shadow, resulting in uneven illumination. Furthermore, changes in lighting conditions alter the characteristics of runway contaminants such as frost and snow, blurring edge areas. This causes feature extraction errors in subsequent segmentation and recognition models, leading to false positives and false negatives in runway contaminant detection, severely impacting the accuracy of runway contaminant identification. Summary of the Invention

[0004] To address the technical problem of low pollutant identification accuracy, this application provides a method and system for runway pollutant identification based on high-definition video. The specific technical solution adopted is as follows: Firstly, this application proposes a method for runway contaminant identification based on high-definition video, the method comprising the following steps: Capture video streams from airport runways; For each target pixel in each frame of the video stream, a neighborhood window is set; the brightness ratio is determined based on the ratio of the difference in the number of bright and dark pixels within the neighborhood window to the total number of pixels; the brightness distribution feature value is determined based on the difference in distance between the bright and dark pixels and the target pixel; the illumination influence value is determined based on the overall brightness within the neighborhood window, the brightness ratio, and the brightness distribution feature value. Texture feature values ​​are obtained based on the LBP difference between a pixel within a neighborhood window and its eight neighboring pixels; the comprehensive correlation is obtained by multiplying the correlation between the mean brightness of all rows and the brightness of each row, and the correlation between the mean brightness of all columns and the brightness of each column; and high-light feature values ​​are obtained based on the comprehensive correlation and texture feature values. Obtain the Shannon entropy of the directional gradient histogram within the neighborhood window; calculate the mean of the eight-neighbor structural similarity between the target pixel and the other pixels within the neighborhood window as the mean of structural similarity; obtain the low-to-medium illumination feature values ​​based on the Shannon entropy of the pixel's neighborhood window and the mean of structural similarity. The high illumination feature value and the medium-low illumination feature value are weighted by the illumination influence value to obtain a comprehensive feature value; the final regularization parameter is constructed based on the comprehensive feature value; and the image filtering is completed based on the final regularization parameter. The preprocessed images are trained using a neural network, and the pollutant identification is completed by inputting a real-time video stream into the model.

[0005] In the aforementioned solution, this application considers that during the airport runway contaminant identification process based on high-definition video, the airport runway environment typically experiences cyclical illumination from runway lights, which may lead to localized overexposure or shadows, resulting in uneven illumination and abnormal changes in contaminant characteristics. This, in turn, causes feature extraction errors during contaminant identification, affecting the accuracy of runway contaminant identification. This application analyzes the brightness conditions in the image and, combined with the feature differences between contaminant edges and non-edges, as well as background runway areas under different lighting conditions, obtains the optimal regularization parameters for each pixel. This ensures that the guided filtering effectively preserves the edge features of contaminants, improves the local features of runway contaminants, and further enhances the identification effect of runway contaminants.

[0006] In one embodiment, the method for determining the brightness ratio based on the ratio of the difference in the number of bright and dark pixels within a neighborhood window to the total number of pixels is as follows: Obtain the brightness value of all pixels in each frame of the image, use the Otsu thresholding method to calculate the maximum inter-class variance of the brightness value of all pixels in the image, obtain the optimal threshold T, set the pixels with brightness values ​​greater than the threshold T as high brightness, and set the pixels with brightness values ​​less than or equal to the threshold T as low brightness. For each pixel's neighborhood window, the difference between the number of high-brightness and low-brightness pixels in the neighborhood window is obtained, and the ratio of the difference to the total number of pixels in the neighborhood window is recorded as the brightness ratio.

[0007] In one embodiment, the method for determining the brightness distribution feature value based on the distance difference between bright and low-brightness pixels and the target pixel is as follows: The average Euclidean distance between the bright spot and the target pixel within the neighborhood window is calculated and recorded as the first average. The average Euclidean distance between the low spot and the target pixel within the neighborhood window is calculated and recorded as the second average. The difference between the first average and the second average is used as the brightness distribution feature value.

[0008] In one embodiment, the illumination influence value is positively correlated with the overall brightness, the brightness distribution characteristic value, and the brightness proportion; wherein the Euclidean distance between the pixel and the target pixel is calculated and recorded as the weighted distance of each pixel, and the ratio of the maximum distance to the weighted distance of each pixel is used as the weight to perform a weighted summation of the brightness values ​​of the pixels to obtain the overall brightness of the neighborhood window.

[0009] In one embodiment, the method of using the product of the correlation between the mean brightness of all rows and the brightness of each row, and the correlation between the mean brightness of all columns and the brightness of each column, as the comprehensive correlation is as follows: Calculate the mean brightness value of each column, use the mean as the feature brightness value of that column, and arrange the feature brightness values ​​of all columns in order from left to right to form the mean brightness value sequence of the row; for each row in the neighborhood window, arrange all the brightness values ​​of each row in order from left to right to form a brightness value sequence, calculate the correlation coefficient between the brightness value sequence of each row and the mean brightness value sequence of the row, and calculate the mean of the correlation coefficients of each row to obtain the row correlation degree; Using the method described above, the average brightness value of each row is calculated as the characteristic brightness value of that row, and the characteristic brightness values ​​of all rows are arranged in order from top to bottom as the average brightness value sequence of the column; the correlation coefficient between the brightness value sequence of each column and the average brightness value sequence of the column is calculated, and the average value is used to obtain the column correlation. The product of row relevance and column relevance is used as the overall relevance.

[0010] In one embodiment, the high-light feature value is positively correlated with the texture feature value and negatively correlated with the overall correlation.

[0011] In one embodiment, the low-to-medium illumination characteristic value is positively correlated with Shannon entropy and negatively correlated with the mean structural similarity value.

[0012] In one embodiment, the method for obtaining a comprehensive feature value by weighting high illumination feature values ​​and medium-low illumination feature values ​​using illumination influence values ​​as weights is as follows: , This represents the illumination effect value of the neighborhood window of the a-th pixel. This represents the highlight feature value of the neighborhood window of the a-th pixel. This represents the low-to-medium illumination feature value of the neighborhood window of the a-th pixel. This represents the comprehensive feature value of the neighborhood window of the a-th pixel.

[0013] In one embodiment, the method for constructing the final regularization parameter based on the comprehensive feature values ​​is as follows: , This indicates the preset minimum regularization parameter. This represents the difference between the preset maximum regularization parameter and the preset minimum regularization parameter. This represents the comprehensive feature value of the neighborhood window of the a-th pixel. This represents the final regularization parameter for the a-th pixel.

[0014] On the other hand, embodiments of this application also provide a runway contaminant identification system based on high-definition video, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any of the above-described methods for runway contaminant identification based on high-definition video.

[0015] The beneficial effects of this application are as follows: This application addresses the issue that during airport runway contaminant identification based on high-definition video, the runway environment often experiences cyclical lighting, potentially leading to localized overexposure or shadows, resulting in uneven illumination and abnormal changes in contaminant characteristics. This, in turn, causes feature extraction errors during contaminant identification, impacting the accuracy of runway contaminant recognition. This application analyzes the brightness of the image and, combined with the feature differences between contaminant edges and non-edge areas, as well as background runway regions under different lighting conditions, obtains the optimal regularization parameters for each pixel. This ensures that the guided filtering effectively preserves the edge features of contaminants, improves the local features of runway contaminants, and further enhances the recognition performance of runway contaminants. Attached Figure Description

[0016] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a method for identifying runway contaminants based on high-definition video, provided as an embodiment of this application. Detailed Implementation

[0018] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive objective, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a runway contaminant identification method and system based on high-definition video proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0020] An embodiment of a method and system for runway contaminant identification based on high-definition video: The following description, in conjunction with the accompanying drawings, details a specific scheme for a high-definition video-based method and system for identifying runway contaminants provided in this application.

[0021] Please see Figure 1 The illustration shows a flowchart of a method and system for identifying runway contaminants based on high-definition video, according to an embodiment of this application. The method includes the following steps: Step S001: Acquire the video stream of the airport runway.

[0022] In runway contaminant identification based on high-definition video, the first step is to collect and transmit runway contaminant data. High-precision cameras are configured; the specific camera configuration is not limited and can be selected by the implementer according to needs, ensuring high-resolution images for subsequent runway contaminant identification. High-definition video streams of the airport runway are acquired in real-time by the deployed cameras and transmitted wirelessly frame by frame using RTSP (Real-Time Streaming Protocol) to the terminal processing module for subsequent computation.

[0023] At this point, the video stream of the runway has been obtained.

[0024] Step S002: Based on the difference in the number of bright and dark pixels in each frame of the image and their distance from the center point, and combined with the overall local brightness, determine the illumination influence value.

[0025] Preprocessing is performed on the real-time frame-by-frame transmitted video data. Because the boundaries between contaminants (such as thin ice and oil) and the runway surface often have low contrast and blurred edges, traditional filtering methods may smooth out these fragile edges during actual processing, leading to feature loss. Therefore, when performing image filtering enhancement on contaminant images in airport runway scenarios, guided filtering algorithms are used to optimize filtering for situations with low visibility, such as fog or overcast skies.

[0026] For each frame in the video stream, a neighborhood window is set, and preprocessing is performed using guided filtering. In this embodiment, the neighborhood window size is 11*11 to ensure good filtering effect while avoiding blockiness or excessive blurring of details near contaminant edges. When the number of neighboring pixels cannot meet the 11×11 window size, the mean interpolation method is used to fill in the missing pixels within the neighborhood window. Guided filtering and mean interpolation are well-known techniques and will not be described in detail in this application.

[0027] Considering that airport runway scenarios typically involve cyclical runway lighting, which can lead to localized overexposure or shadows, the low brightness conditions, where the differences between contaminants and the background are often small, further blur the edges. Conversely, under strong light, contaminant areas can exhibit specular reflection, resulting in highly irregular internal textures and affecting contaminant edge features. To ensure effective preservation of contaminant edge features during denoising, the regularization parameters of the guided filtering algorithm need further optimization by combining image features.

[0028] Based on the above analysis, in the airport runway scene, the characteristics of the contaminant area will change differently under the influence of highlights and low light. For any pixel, denoted as the target pixel, when analyzing the neighborhood window centered on the target pixel, the closer the pixel is to the target pixel, the better it can show the influence of light on the area where the target pixel is located. Therefore, it is necessary to first analyze the influence of light on the image.

[0029] First, obtain the brightness value of all pixels in each frame of the image. Then, use the Otsu thresholding method to calculate the maximum inter-class variance of the brightness values ​​of all pixels in the image to obtain the optimal threshold T. Pixels with brightness values ​​greater than the threshold T are set as high brightness, and pixels with brightness values ​​less than or equal to the threshold T are set as low brightness.

[0030] For each pixel's neighborhood window, the difference between the number of high-brightness and low-brightness pixels in the neighborhood window is obtained. The ratio of this difference to the total number of pixels in the neighborhood window is used to represent the brightness ratio feature within the target pixel's neighborhood window, and this value is recorded as the brightness ratio.

[0031] Furthermore, considering that the more severe the influence of bright light, the more concentrated the high-brightness points in the neighborhood window will be towards the target pixel, a brightness distribution feature value is calculated. Specifically, the average Euclidean distance between the high-brightness points and the target pixel in the neighborhood window is calculated and recorded as the first average value, and the average Euclidean distance between the low-brightness points and the target pixel in the neighborhood window is calculated and recorded as the second average value. The difference between the first average value and the second average value is used as the brightness distribution feature value.

[0032] The closer the high-brightness distribution is to the target pixel at the center, the larger the obtained brightness distribution feature value; the farther the high-brightness distribution is from the target pixel at the center, the smaller the obtained brightness distribution feature value.

[0033] The calculated brightness distribution characteristic values ​​and brightness percentages are normalized to ensure that the values ​​are not negative, and then a lighting influence index is constructed to characterize local lighting conditions. In this embodiment, the normalization used is maximum-minimum value normalization.

[0034] Next, for each pixel within the neighborhood window, the Euclidean distance between the pixel and the target pixel is calculated and recorded as the weighted distance for each pixel. The ratio of the maximum distance to the weighted distance of each pixel is then used as a weight to calculate the overall brightness of the neighborhood window. The closer a pixel is to the target pixel, the higher its weight and the more important its brightness performance.

[0035] The illumination impact value is calculated based on the overall brightness within the domain window, the brightness distribution characteristic value, and the brightness proportion.

[0036] The illumination influence value is positively correlated with the overall brightness, the brightness distribution characteristic value, and the brightness proportion.

[0037] It should be noted that positive correlation means that when one variable increases, the other variable also increases, and the two variables change in the same direction. When one variable changes from large to small or from small to large, the other variable also changes from large to small or from small to large. The specific relationship is determined by the actual application, and this application does not impose any special restrictions.

[0038] Preferably, in this embodiment, the expression for the illumination influence value is: , This represents the brightness percentage of the neighborhood window of the a-th pixel. This represents the brightness distribution feature value of the neighborhood window of the a-th pixel. This represents the overall brightness of the neighborhood window of the a-th pixel. This represents the illumination effect value of the neighborhood window of the a-th pixel. This represents the normalization function for maximum and minimum values.

[0039] This application considers that the correlation between different pixels and the target pixel may vary due to the brightness distribution within the window, resulting in different situations such as strong edge illumination and weak center illumination, or strong center illumination and weak edge illumination, yet the same average value is obtained. This can interfere with the illumination influence analysis and lead to errors in subsequent pollutant identification. Therefore, based on the aforementioned prior knowledge, this application combines the brightness distribution and a weighted analysis of pixels at different distances to comprehensively obtain the illumination influence within the target neighborhood window, thereby reducing subsequent errors.

[0040] When the neighborhood window of the target pixel is severely affected by high brightness and the light is concentrated at the center of the target pixel, the calculated brightness ratio is... Because high-brightness pixels account for a larger proportion and thus have relatively larger values, after assigning greater weight to the brightness values ​​of pixels closer to the target pixel at the center, the resulting illumination influence value is... The larger the value, the greater the brightness ratio; when the neighborhood window of the target pixel is severely affected by low brightness and the low-brightness points are concentrated in the target pixel at the center, the higher the calculated brightness ratio. The value decreases as the number of low-brightness pixels increases. After assigning greater weight to the brightness values ​​of pixels closer to the target pixel at the center, the obtained illumination influence value... The smaller the value, the better; when the distribution of highlights and low highlights within the neighborhood window of the target pixel is relatively similar, the closer the distribution of highlights is to the target pixel at the center, the smaller the overall acquired illumination influence value. The larger the value, and the further away the high-brightness distribution is from the target pixel in the center, the greater the overall acquired illumination influence value. The smaller the relative size.

[0041] At this point, the illumination influence value of the neighborhood window of each pixel has been obtained.

[0042] Step S003: Determine the highlight feature value based on the texture feature value of the pixel and the correlation of the row and column respectively.

[0043] An analysis of the characteristics of pollutant areas under bright light conditions reveals that, considering the irregular, raised shapes of airport runway pollutants (such as snow, ice, and frost), the edges of these pollutant areas exhibit complex shapes under bright light. Furthermore, due to changes in the material properties on both sides of the edges, significant differences in brightness occur between the two sides of the pollutant edge. While the edge texture retains some texture under illumination, the brightness values ​​are discontinuous. However, the non-edge areas within the pollutant and the background environment, under strong light, suffer from reduced detail textures that are overwhelmed by the noise generated by the highlights.

[0044] Based on the above analysis, firstly, the LBP value of each pixel is calculated, and then the average LBP value of each pixel and its eight neighboring pixels is calculated. A comprehensive analysis of local features is performed. The LBP value of each pixel is calculated as the average of the LBP values ​​of its eight neighboring pixels. The absolute difference is used as the texture feature value of the pixel.

[0045] When a pixel is located at the edge of a contaminant, its LBP will differ significantly from the LBP of its eight neighboring pixels, resulting in a larger calculated texture feature value. Conversely, when a pixel is located at the edge of a non-contaminant, the LBP of its pixel will differ less significantly from the LBP of its eight neighboring pixels due to the smoothing effect of the specular highlight on the detailed texture, resulting in a smaller calculated texture feature value.

[0046] Furthermore, the brightness variations described above are analyzed. The mean brightness value of each column is calculated and used as the characteristic brightness value of that column. The characteristic brightness values ​​of all columns are then arranged from left to right to form a sequence of mean brightness values ​​for each row. For each row in the neighborhood window, all brightness values ​​in that row are arranged from left to right to form a brightness value sequence. The correlation coefficient between the brightness value sequence of each row and the mean brightness value sequence of the row is calculated. The average of the correlation coefficients for each row is then used to obtain the row correlation. In this embodiment, the correlation coefficient is calculated using the Pearson correlation coefficient.

[0047] Using the method described above, the average brightness value of each row is calculated as the characteristic brightness value of that row, and the characteristic brightness values ​​of all rows are arranged in order from top to bottom as the average brightness value sequence of the column. Then, the correlation coefficient between the brightness value sequence of each column and the average brightness value sequence of the column is calculated, and the average value is used to obtain the column correlation.

[0048] The overall relevance is obtained by multiplying the row relevance and column relevance. When the edges of the neighborhood window are more pronounced and the brightness values ​​exhibit discontinuous changes, the row and column correlation of the brightness values ​​within the neighborhood window are relatively low, resulting in a lower overall correlation score. The smaller the value, the higher the overall correlation. Conversely, when the area is a non-edge region within the pollutants under bright light or a flat background track, the overall brightness caused by the light is relatively concentrated, resulting in relatively high row and column correlations and a higher overall correlation. The larger it is.

[0049] For each pixel, the highlight feature value is obtained based on the comprehensive relevance and texture feature value within its neighborhood window.

[0050] The high-light feature value is positively correlated with the texture feature value, and negatively correlated with the overall correlation.

[0051] It should be noted that negative correlation means that when one variable increases, the other variable decreases accordingly, and the two variables change in opposite directions. When one variable changes from large to small or from small to large, the other variable also changes from small to large or from large to small. The specific relationship is determined by practical application, and this application does not impose any special restrictions.

[0052] Preferably, the expression for the high illumination characteristic value is: , This represents the texture feature value of the neighborhood window of the a-th pixel. This represents the overall relevance of the neighborhood window of the a-th pixel. This represents a very small constant, whose function is to ensure that the denominator is not zero. In this embodiment, it is taken as 0.001. This represents the highlight feature value of the neighborhood window of the a-th pixel. This represents the normalization function for maximum and minimum values.

[0053] At this point, the highlight feature values ​​of the neighborhood window for each pixel have been obtained.

[0054] Step S004: Determine the low-light feature values ​​based on the gradient Shannon entropy within the neighborhood window and the structural similarity of the eight neighborhoods between pixels.

[0055] The characteristics of pollutants under normal and low light conditions were analyzed. Under normal and low light conditions, the pollutant areas (such as snow, ice, and frost) usually have certain detailed texture features. The gradient angles of pixels in these areas are usually relatively dispersed, and there are usually some differences in local features. These details usually need to be preserved as much as possible during denoising. On the other hand, the background runway area usually has a more concentrated gradient angle due to its own road surface texture and the existence of planned runway lines. The overall gradient distribution is more concentrated, and the road surface area needs to use stronger denoising to smooth the noise.

[0056] Therefore, for a neighborhood window of a pixel, the Sobel operator is used to obtain the gradient angles of all pixels within that window, resulting in the HOG (Histogram of Oriented Gradients) for that window, where the 360° angle is divided into multiple intervals. The Shannon entropy of the HOG is then calculated. This is used to characterize the gradient angle distribution of pixels within the window. In this embodiment, the angle is divided into 18 intervals, each interval being 20 degrees.

[0057] Furthermore, for each pixel, the eight-neighbor window is denoted as a local small window. The SSIM structural similarity is calculated using the local small window corresponding to the pixel and the local small windows corresponding to the other pixels in the neighboring window. In this embodiment, the standard deviation of the Gaussian kernel is set to 1.5, and the steady-state constant is... The calculation uses brightness values ​​as input. The mean of all structural similarity results within the window is obtained. When the target pixel is located in a contaminant area, its interior or edges typically contain complex and varied detailed texture features, leading to difficulties in the computational acquisition. The value is relatively small, but when the target pixel is located in the road surface area, due to the regularity of the road surface texture features, the calculated value is... The value is usually relatively large.

[0058] Then, the mid-to-low illumination feature values ​​are obtained based on the Shannon entropy and the mean structural similarity of the pixel neighborhood window.

[0059] The low-to-medium illumination characteristic values ​​are positively correlated with the entropy of the scallops and negatively correlated with the mean structural similarity.

[0060] Preferably, in this embodiment, the expression for the low-to-medium illumination characteristic value is: , This represents the Shannon entropy of the neighborhood window of the a-th pixel. This represents the mean structural similarity of the neighborhood window of the a-th pixel. This represents a very small constant, whose function is to ensure that the denominator is not zero. In this embodiment, it is taken as 0.001. This represents the low-to-medium illumination feature value of the neighborhood window of the a-th pixel. This represents the normalization function for maximum and minimum values.

[0061] At this point, the low-to-medium illumination feature values ​​of the neighborhood window for each pixel have been obtained.

[0062] Step S005: Use the illumination influence value as a weight to weight the high illumination feature value and the medium and low illumination feature value to obtain a comprehensive feature value; then construct the final regularization parameter to complete the image preprocessing.

[0063] The above steps are used to obtain the high illumination feature value and medium-low illumination feature value of the neighborhood window of each pixel, and then the comprehensive feature value is constructed by weighting them with the illumination influence value.

[0064] Preferably, in this embodiment, the expression for the comprehensive feature value is: , This represents the illumination effect value of the neighborhood window of the a-th pixel. This represents the highlight feature value of the neighborhood window of the a-th pixel. This represents the low-to-medium illumination feature value of the neighborhood window of the a-th pixel. This represents the comprehensive feature value of the neighborhood window of the a-th pixel.

[0065] In actual processing, the local features of the target pixel within its neighborhood window vary under different lighting conditions, leading to errors in acquiring and identifying pollutant features. Therefore, based on the aforementioned prior knowledge, this application fully combines the lighting intensity within the neighborhood window of the pixel to perform weighted analysis on the local features within the neighborhood window, obtaining the comprehensive feature information within the neighborhood window of the pixel, thereby reducing the error in acquiring runway pollutant features.

[0066] The final regularization parameter is constructed based on the comprehensive feature values. The expression for the regularization parameter is as follows: , This indicates the preset minimum regularization parameter. This represents the difference between the preset maximum regularization parameter and the preset minimum regularization parameter. This represents the comprehensive feature value of the neighborhood window of the a-th pixel. This represents the final regularization parameter for the a-th pixel. The preset maximum and minimum regularization parameters are typical values ​​of the regularization parameters, which are 0.01 and 0.0001 in this example.

[0067] In actual processing, the local features of each pixel under the influence of illumination are not considered, which leads to a large error when identifying runway pollutants. Therefore, based on the above prior knowledge, this application fully combines the features of each pixel under the influence of illumination in its corresponding neighborhood window, and performs feature weighting on the regularization parameters to obtain the final regularization parameters for the target pixel, thereby reducing the error in runway pollutant identification.

[0068] Calculate the final regularization parameter for each pixel. Then, guided filtering is used to filter, denoise and enhance each acquired image, and the filtered image is output. Guided filtering is a well-known technique, and the specific operation steps will not be described in detail. This completes the preprocessing operation for the image.

[0069] This completes the preprocessing of each frame of the image.

[0070] Step S006: After training the model using a neural network, pollutant identification is performed on the real-time video stream.

[0071] After preprocessing the video stream, a contaminant dataset was prepared to train a runway contaminant detection model for real-time runway contaminant identification. 3000 images were acquired from the video stream as the training dataset. LabelMe annotation software was used to label the acquired dataset, and after labeling the contaminants in the images, the 3000 images were divided into training, testing, and validation sets in a 7:2:1 ratio, which were then used as inputs. The U-net segmentation algorithm was used to train the contaminant identification and segmentation model. In configuring the algorithm, an SE attention mechanism was added to each upsampling layer. This attention mechanism adaptively calibrates the importance weights of each feature channel through compression and activation operations. For airport runway detection, this helps the model focus on feature channels that are useful in distinguishing between "contaminants" and "background," thereby enhancing feature representation capabilities.

[0072] The algorithm uses the binary cross-entropy loss function as its core loss function. This loss function, being the most commonly used binary classification loss function, is suitable for pixel-level classification tasks and offers faster convergence and higher accuracy for pollutant detection. It also combines the Dice loss function with the binary cross-entropy loss function to form a combined loss function, further improving the accuracy of pollutant detection. The AdamW optimizer is chosen for the optimizer, incorporating momentum and adaptive gradient scaling to improve computational efficiency while preventing overfitting. The airport runway pollutant identification model is output after 100 iterations of training.

[0073] Using real-time acquired high-definition video stream data as input, and employing a pre-trained airport runway contaminant identification model, the system performs real-time detection of contaminants in the airport runway area, outputting a segmented and identified contaminant coverage patch map for each frame. This completes the runway contaminant identification operation based on high-definition video.

[0074] Based on the same inventive concept as the above method, this embodiment of the invention also provides a runway contaminant identification system based on high-definition video, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described methods for runway contaminant identification based on high-definition video.

[0075] It should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

[0076] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for identifying runway contaminants based on high-definition video, characterized in that, The method includes the following steps: Capture video streams from airport runways; For each target pixel in each frame of the video stream, a neighborhood window is set; the brightness ratio is determined based on the ratio of the difference in the number of bright and dark pixels within the neighborhood window to the total number of pixels; the brightness distribution feature value is determined based on the difference in distance between the bright and dark pixels and the target pixel; the illumination influence value is determined based on the overall brightness within the neighborhood window, the brightness ratio, and the brightness distribution feature value. Texture feature values ​​are obtained based on the LBP difference between a pixel within a neighborhood window and its eight neighboring pixels; the comprehensive correlation is obtained by multiplying the correlation between the mean brightness of all rows and the brightness of each row, and the correlation between the mean brightness of all columns and the brightness of each column; and high-light feature values ​​are obtained based on the comprehensive correlation and texture feature values. Obtain the Shannon entropy of the directional gradient histogram within the neighborhood window; calculate the mean of the eight-neighbor structural similarity between the target pixel and the other pixels within the neighborhood window as the mean of structural similarity; obtain the low-to-medium illumination feature values ​​based on the Shannon entropy of the pixel's neighborhood window and the mean of structural similarity. The high illumination feature value and the medium-low illumination feature value are weighted by the illumination influence value to obtain a comprehensive feature value; the final regularization parameter is constructed based on the comprehensive feature value; and the image filtering is completed based on the final regularization parameter. The preprocessed images are trained using a neural network, and real-time video streams are input into the model to complete pollutant identification. The method for using the product of the correlation between the mean brightness of all rows and the brightness of each row, and the correlation between the mean brightness of all columns and the brightness of each column, as the comprehensive correlation is as follows: Calculate the mean brightness value of each column, use the mean as the feature brightness value of that column, and arrange the feature brightness values ​​of all columns in order from left to right to form the mean brightness value sequence of the row; for each row in the neighborhood window, arrange all the brightness values ​​of each row in order from left to right to form a brightness value sequence, calculate the correlation coefficient between the brightness value sequence of each row and the mean brightness value sequence of the row, and calculate the mean of the correlation coefficients of each row to obtain the row correlation degree; Similarly, calculate the average brightness value of each row as the characteristic brightness value of that row, and arrange the characteristic brightness values ​​of all rows in order from top to bottom as the average brightness value sequence of the column; calculate the correlation coefficient between the brightness value sequence of each column and the average brightness value sequence of the column, and calculate the average to obtain the column correlation. The product of row relevance and column relevance is used as the overall relevance.

2. The method for identifying runway contaminants based on high-definition video as described in claim 1, characterized in that, The method for determining the brightness ratio based on the ratio of the difference in the number of bright and dark pixels within a neighborhood window to the total number of pixels is as follows: Obtain the brightness value of all pixels in each frame of the image, use the Otsu thresholding method to calculate the maximum inter-class variance of the brightness value of all pixels in the image, obtain the optimal threshold T, set the pixels with brightness values ​​greater than the threshold T as high brightness, and set the pixels with brightness values ​​less than or equal to the threshold T as low brightness. For each pixel's neighborhood window, obtain the difference between the number of high-brightness and low-brightness pixels in the neighborhood window, and record the ratio of the difference to the total number of pixels in the neighborhood window as the brightness ratio.

3. The method for identifying runway contaminants based on high-definition video as described in claim 1, characterized in that, The method for determining the brightness distribution feature value based on the distance difference between bright and low-brightness pixels and the target pixel is as follows: The average Euclidean distance between the bright spot and the target pixel within the neighborhood window is calculated and recorded as the first average. The average Euclidean distance between the low spot and the target pixel within the neighborhood window is calculated and recorded as the second average. The difference between the first average and the second average is used as the brightness distribution feature value.

4. The method for identifying runway contaminants based on high-definition video as described in claim 1, characterized in that, The illumination influence value is positively correlated with the overall brightness, the brightness distribution characteristic value, and the brightness proportion. The process involves calculating the Euclidean distance between a pixel and the target pixel, recording it as the weighted distance for each pixel, and then using the ratio of the maximum distance to the weighted distance of each pixel as a weight to perform a weighted summation of the pixel brightness values ​​to obtain the overall brightness of the neighborhood window.

5. The method for identifying runway contaminants based on high-definition video as described in claim 1, characterized in that, The high-light feature value is positively correlated with the texture feature value, and negatively correlated with the overall correlation.

6. The method for identifying runway contaminants based on high-definition video as described in claim 1, characterized in that, The low-to-medium illumination characteristic values ​​are positively correlated with the entropy of the scallops and negatively correlated with the mean structural similarity.

7. The method for identifying runway contaminants based on high-definition video as described in claim 1, characterized in that, The method for obtaining a comprehensive feature value by weighting high illumination feature values ​​and medium-low illumination feature values ​​using illumination influence values ​​as weights is as follows: , This represents the illumination influence value of the neighborhood window of the a-th pixel. This represents the highlight feature value of the neighborhood window of the a-th pixel. This represents the low-to-medium illumination feature value of the neighborhood window of the a-th pixel. This represents the combined feature value of the neighborhood window of the a-th pixel.

8. The method for identifying runway contaminants based on high-definition video as described in claim 1, characterized in that, The method for constructing the final regularization parameter based on comprehensive feature values ​​is as follows: , This indicates the preset minimum regularization parameter. This represents the difference between the preset maximum regularization parameter and the preset minimum regularization parameter. This represents the combined feature value of the neighborhood window of the a-th pixel. This represents the final regularization parameter for the a-th pixel.

9. A runway contaminant identification system based on high-definition video, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the runway pollutant identification method based on high-definition video as described in any one of claims 1-8.

Images