Airfield pavement large-view-field two-dimensional image imaging method and related device

Abstract

The invention belongs to the field of intelligent management of airfield pavements, and particularly relates to an airfield pavement large-view-field two-dimensional image imaging method and a related device. The invention discloses an airfield pavement large-view-field two-dimensional image imaging method. The method comprises the following steps: acquiring a pavement image acquired by using an unmanned aerial vehicle; preprocessing the pavement image to obtain a preprocessed image; feature point extraction is carried out on the preprocessed image, and feature points in the preprocessed image are extracted; feature point matching is carried out on feature points in the preprocessed image; filtering the matched feature point pairs to eliminate mismatching points; based on the matched feature points, splicing the matched images to obtain a spliced image; and based on the spliced image, carrying out image fusion to obtain an airport pavement panorama. And various problems of the existing image fusion defense line are solved.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent management of airport pavements, and in particular relates to a method for imaging large-field-of-view two-dimensional images of airport pavements and related devices. Background Art

[0002] Airport pavements are the primary areas for aircraft takeoff, landing, and taxiing, and their condition is directly related to flight safety. Therefore, the maintenance and management of airport pavements are of vital importance. Existing pavement management methods primarily rely on quality inspection data for performance evaluation, enabling scientific and rational evaluation, prediction, and maintenance decisions. Traditional manual inspection methods require staff to conduct section-by-section inspections of the pavement, which is inefficient and has high labor costs. For large-scale airport pavements, especially runways and taxiways at international airports, manual inspections can take hours or even days, seriously affecting the real-time nature of inspections. Furthermore, during manual inspections, staff face risks such as flight safety and inclement weather.

[0003] While on-board imaging systems can cover a certain area of ​​the road surface, their viewing angle is limited, making it difficult to ensure full coverage of the entire airport pavement. Furthermore, because vehicles must travel along fixed routes, they struggle to effectively inspect complex, marginal areas, leading to blind spots and missed detection of pavement defects.

[0004] Existing pavement defect inspection results are often recorded in text or 2D charts, lacking intuitive visuals and making it difficult for pavement managers to grasp the overall pavement condition in a timely manner. However, cracks, potholes, and other defects on airport pavements are often unevenly distributed. High-precision, large-field-of-view 2D pavement images can more intuitively display details of localized areas, helping to identify potential hazards.

[0005] Current pavement inspection technologies lack integration with airport smart management systems, making automated data collection and management difficult. This not only reduces inspection efficiency but also hinders the development of intelligent airport pavement management.

[0006] Image stitching technology is a key research area in computer vision and digital image processing. It aims to stitch multiple images of the same area from different perspectives into a single image with a larger field of view and higher resolution by geometrically transforming them. Traditional cameras have a narrow field of view and are unable to track targets over a large field of view, resulting in numerous shortcomings. Furthermore, monocular cameras are prone to loss of focus during target tracking, which can affect performance. Therefore, drone photography technology can be used to achieve high-precision, large-field-of-view image stitching of airport pavement surfaces.

[0007] A search revealed that, in recent years, the development of drones has enabled them to efficiently capture high-resolution road surface images while in flight. Drone photography technology has been used to achieve high-precision, large-field-of-view image stitching of airport pavement. However, there are still some problems and challenges in stitching and fusion, mainly in the following aspects:

[0008] 1. Differences in lighting and viewing angle: Differences in ambient brightness at different times of aerial photography, as well as slight changes in the shooting angle, can lead to inconsistent brightness and shadow changes in the image area, thus affecting the clarity and recognition accuracy of feature points.

[0009] 2. Accuracy control of image transformation model: The geometric relationship between images is complex, such as scale changes and perspective deformation. If the model estimation is inaccurate, it will directly affect the accuracy and edge continuity of the stitching results.

[0010] 3. The naturalness of stitching seams and fusion transitions: In overlapping image areas, if brightness differences are not handled properly, obvious stitching marks may appear, affecting visual continuity and even interfering with road surface structure recognition.

[0011] In summary, the algorithm needs to be further optimized to improve the accuracy of feature extraction and matching, the precision of image transformation and alignment, the naturalness of image fusion, as well as the computational complexity and real-time performance. Summary of the Invention

[0012] The purpose of the present invention is to provide a method and related device for imaging two-dimensional images of airport pavement with a large field of view, which solves various problems existing in the existing image fusion defense line.

[0013] The present invention is achieved through the following technical solutions:

[0014] The present invention discloses a method for imaging a large-field two-dimensional image of an airport pavement, comprising:

[0015] S1. Obtaining road surface images collected by a UAV;

[0016] S2. Preprocessing the road surface image to obtain a preprocessed image;

[0017] S3, extracting feature points from the preprocessed image;

[0018] S4, performing feature point matching on the feature points in the preprocessed image;

[0019] S5. Filter the matched feature point pairs to eliminate mismatched points.

[0020] S6. Based on the matched feature points, the matched images are stitched together to obtain a stitched image.

[0021] S7. Based on the stitched images, perform image fusion to obtain a panoramic view of the airport pavement.

[0022] Furthermore, in S2, the preprocessing is to perform a de-reflection operation on the image, specifically: through a three-level Gaussian kernel multi-scale decomposition and a polarization physical model, accurate separation of the reflection layer is achieved in the HSV space, Retinex theory is coupled with optical polarization characteristics, and the reflection suppression intensity is dynamically adjusted through a global adjustment factor;

[0023] The image is converted to HSV space, reflection is suppressed on the luminance channel, and chrominance is adjusted to compensate for color distortion. After suppressing highlights, a detail recovery method based on guided filtering is used to smooth the edges of the luminance image.

[0024] Furthermore, in S3, the optimized sift algorithm is used to extract feature points from the preprocessed image, which specifically includes the following steps:

[0025] (1) constructing an image pyramid containing multiple scales based on the preprocessed image, wherein each layer of the image pyramid corresponds to the flight altitude at the time of shooting;

[0026] (2) During the pyramid construction process, pyramid position coding information related to the image position and flight altitude is injected into each layer of the image to enhance the feature alignment capability of the same region in images at different flight altitudes;

[0027] (3) On the image pyramid with pyramid position coding, the improved SIFT algorithm is used to perform key point detection, main direction determination, gradient histogram calculation and feature descriptor construction in sequence to obtain the image feature point set;

[0028] (4) Screening and normalizing the extracted feature points to form a stable feature point set required for subsequent image stitching;

[0029] In step (3), the improved SIFT algorithm uses a faster gradient calculation method to calculate the gradient histogram, replacing the square root with the absolute value:

[0030] m(x,y)=|L(x+1,y)-L(x-1,y)|+|L(x,y+1)-L(x,y-1)|

[0031] Among them, m(x,y) refers to the modulus of the pixel at that position. After completing the gradient calculation of the key point, the SIFT algorithm uses a histogram to count the gradient and direction of the pixels in the neighborhood. The gradient histogram divides the direction range of 0 to 360 degrees into 36 columns, where each column is 10 degrees, and the direction angle is rigidly assigned to a fixed interval. Then, bilinear interpolation is used to reduce the edge quantization error. The formula is as follows:

[0032]

[0033] Among them, Δθ,θ′,θ min They represent the 10° interval of the direction interval, the actual calculated direction angle, and the lower limit angle of the current direction interval respectively; w is the interpolation weight, which is used to distribute the gradient amplitude to adjacent intervals;

[0034] Dynamically adjust the Gaussian kernel parameters, the formula is:

[0035] σ adapt =max(1.2σ,3)

[0036] Among them, σ, σ adapt They represent the basic standard deviation and the standard deviation after adaptive adjustment respectively;

[0037] The construction of the feature descriptor includes four parameters: two-dimensional position information x and y, scale and main direction; based on these four parameters, the generated SIFT descriptor is a 128-dimensional vector; then the 5-dimensional position code is directly concatenated with the 128-dimensional vector to form a 133-dimensional enhanced descriptor.

[0038] Furthermore, in S4, feature point matching is performed using multi-scale feature descriptors. For each feature point descriptor, the most similar one is found in the descriptor set of the other image. The specific process is as follows:

[0039] To address the feature similarity problem caused by repeated textures of airport pavement markings, a bidirectional KNN matching strategy is adopted, combined with a distance ratio threshold, and a bidirectional verification mechanism to provide a high-purity matching pair set for subsequent geometric verification.

[0040] Furthermore, in S5, the GMS method is used to distinguish correct matches from incorrect matches by analyzing the motion consistency of feature matching in the local grid. The specific steps are as follows:

[0041] A1: First, the image is divided into N×M uniform grids, and the matching points are mapped to each grid. For each matching pair (p i ,q i ), calculate the grid coordinates to which it belongs;

[0042] A2: For each matching pair (p i ,q i ) calculate its displacement vector Δx i =q i -p i , and calculate its support score S in the 3×3 neighborhood grid i ;

[0043] A3: Dynamically calculate the retention threshold T based on the density of matching points in the grid i :If Si >T i , the matching pair (p i , q i ) is considered reliable, and this matching pair is retained; otherwise, it is regarded as a false match and eliminated.

[0044] Furthermore, in S6, the images that have completed matching are stitched through the normalized direct linear transformation method to estimate the homography matrix. The steps include:

[0045] A1: Normalize the coordinates of the matched feature points in the matching images so that their centers are aligned with the origin of the coordinates and perform uniform scaling to normalize the average distance from the points to the origin to a constant;

[0046] A2: Based on the normalized point coordinates, construct the linear constraint relationships between each group of matching points for establishing the projection transformation model between images;

[0047] A3: Integrate the linear relationships of all point pairs into a set of linear equations and use the singular value decomposition method to solve the optimal homography matrix parameters;

[0048] A4: Perform denormalization processing on the obtained homography matrix parameters to restore them to the original image coordinate system, and obtain the final geometric transformation model for image stitching.

[0049] Furthermore, in S7, the image fusion is specifically as follows: perform weighted processing on the gray values of all pixel points in the overlapping area of the images;

[0050]

[0051] Among them, I1∩I2 represents the overlapping area of the images, w1(x, y) represents the weighted value of the image to be stitched 1 at the pixel point (x, y), w2(x, y) represents the weighted average value of the image to be stitched 2 at the pixel point (x, y), 0 < w1, w2 < 1, w1 + w2 = 1; I1(x, y) represents the pixel gray value of the image 1 at the position (x, y); I2(x, y) represents the pixel gray value of the image 2 at the position (x, y);

[0052] Based on the weight assignment of the Euclidean distance, the weight of the area closer to the image to be stitched 1 is greater. The formula is as follows:

[0053]

[0054] Among them, x start 、x end are respectively the starting coordinate and the ending coordinate of the overlapping area in the horizontal direction, and y start 、y end are respectively the starting coordinate and the ending coordinate of the overlapping area in the vertical direction.

[0055] The present invention discloses a large-field-of-view two-dimensional imaging system for an airport pavement, comprising:

[0056] An image acquisition module is used to acquire road surface images collected by a drone;

[0057] A preprocessing module, used for preprocessing the road surface image to obtain a preprocessed image;

[0058] A feature extraction module is used to extract feature points from the preprocessed image;

[0059] A feature matching module is used to perform feature point matching on feature points in the preprocessed image;

[0060] Elimination module, used to filter the matched feature point pairs and eliminate false matching points;

[0061] A stitching module is used to establish an image transformation model based on the matched feature points, and stitch the matched images to obtain a stitched image;

[0062] The image fusion module is used to perform image fusion based on the stitched images to obtain a panoramic view of the airport pavement.

[0063] The present invention discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the method for imaging a large-field-of-view two-dimensional image of an airport pavement is implemented.

[0064] The present invention discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for imaging a large-field-of-view two-dimensional image of an airport pavement is realized.

[0065] Compared with the prior art, the present invention has the following beneficial technical effects:

[0066] The present invention provides a high-precision, large-field-of-view, two-dimensional imaging method for airport pavements. This method uses drones for image acquisition, which execute image acquisition tasks along preset routes and waypoints to ensure uniform image coverage and sufficient overlap. Furthermore, the method combines high-resolution industrial-grade cameras to achieve stable acquisition of pavement images at multiple scales and in strong light scenarios.

[0067] In the preprocessing stage, a multimodal fusion architecture combining three-level Gaussian kernel multi-scale decomposition with polarized light physical modeling is introduced to separate the specular reflection layer from the illumination layer in HSV space. Retinex theory and guided filtering are combined to achieve adaptive reflection suppression in highlight areas such as water and oil stains, enhancing the robustness of subsequent feature point extraction.

[0068] During the feature extraction phase, a pyramid position encoding mechanism (PPE) was introduced to establish spatial consistency encoding based on image hierarchical position and flight altitude. This addressed the issue of feature point offsets for the same structure in images at different altitudes. The SIFT algorithm was improved, with bilinear interpolation and dynamic Gaussian kernel modulation employed to enhance feature distribution uniformity and scale stability.

[0069] During the feature matching phase, a bidirectional KNN matching strategy combined with a ratio test mechanism was used to ensure that matching points in both images hold in the bidirectional mapping, eliminating potential mismatches and improving matching accuracy. In step S5, the GMS algorithm was used to analyze the motion consistency of feature points within the grid area. Combined with a dynamic density threshold screening strategy, this significantly reduced mismatch interference caused by repeated markings on the airport pavement, reducing the average mismatch rate to below 8%.

[0070] During the calculation phase of the multi-scale transformation optimization stitching model, a three-level pyramid image structure is constructed, rapidly converging the global homography matrix from the low-resolution layer, and then refining and optimizing the local geometric consistency layer by layer. The APAP algorithm is introduced to independently model each local mesh based on the global homography, balancing overall rigid transformation with local elastic correction, effectively alleviating edge misalignment and local stretching issues.

[0071] During the image fusion stage, the Euclidean distance based on pixel position is used to calculate the weighting coefficient and dynamically adjust the grayscale fusion ratio of the overlapping area of ​​the images. Compared with traditional average fusion, the fusion result has smooth edges without mutations, and the image brightness gradient is natural, ensuring the visual consistency of the splicing area and significantly improving the continuous perception and readability of the road surface structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is an overall flow chart of a method for high-precision two-dimensional imaging of a large-view of an airport pavement according to the present invention;

[0073] Figure 2 Waypoint map generated for the airport pavement detection area;

[0074] Figure 3 It is the process of matching feature points of road surface images;

[0075] Figure 4 Solve the process for the homography matrix;

[0076] Figure 5 To stitch the matched road surface images. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solutions and advantages of the present invention more clear, the following is a further detailed description with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. That is, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments.

[0078] The components described and illustrated in the drawings and embodiments of the present invention may be arranged and designed in a variety of different configurations. Therefore, the detailed description of the embodiments of the present invention provided in the following drawings is not intended to limit the scope of the claimed invention, but merely represents a selected embodiment of the present invention. All other embodiments derived by those skilled in the art based on the drawings and embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0079] It should be noted that the terms "comprises", "includes" or any other variations are intended to cover non-exclusive inclusion, so that a process, element, method, article or apparatus that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to the process, element, method, article or apparatus.

[0080] The features and performance of the present invention are further described in detail below with reference to the embodiments.

[0081] like Figure 1 As shown, an embodiment of the present invention provides a method for imaging a large-field-of-view two-dimensional image of an airport pavement, which specifically includes the following steps:

[0082] Step 1: Use drones to collect high-resolution road surface images.

[0083] Considering that airports are usually in a busy traffic state, drone image collection should be carried out during non-peak hours to avoid interference with flight operations, such as in the morning or evening, to avoid strong light reflections and shadows caused by direct sunlight, while ensuring airport safety and normal operations.

[0084] The quality of two-dimensional images captured by drones is affected by a variety of factors, including flight altitude, image overlap, and ambient noise. Therefore, appropriate photography parameters must be set based on the specific scenario. Considering the actual needs of airport pavement, the image overlap was set to 50% to ensure a seamless stitching effect. The drone's flight altitude along the runway was set to 15 meters, and the flight speed was set to 2 meters per second. A high-resolution, industrial-grade camera with a CMOS sensor and high dynamic range (HDR) capabilities was used to cope with the airport's complex lighting conditions. The minimum configuration was 20 megapixels to ensure that the captured images could distinguish cracks and small defects on the pavement. The image size was typically configured at 5472 x 3648 pixels, providing sufficient spatial resolution to capture pavement details.

[0085] During the acquisition process, the waypoint control software DJIPILOT2 is used to plan the UAV mapping mission. According to the specified flight and photography parameters, different routes and waypoints are automatically generated within the detection area. Figure 2 After the route planning is completed, the drone takes off from both sides of the runway, with the camera angle perpendicular to the road surface, and flies according to the predetermined flight route and waypoints while automatically collecting road images.

[0086] Step 2: Image preprocessing to improve imaging quality.

[0087] During the image acquisition process, drones are subject to unavoidable subjective and objective factors, which can result in image quality failing to meet registration requirements. If image stitching is performed in this situation, processing speed will be slowed down and the stitching results will be affected.

[0088] The main interference factors include: first, brightness differences caused by changes in environmental conditions; second, the camera or body of the drone may shake during flight, resulting in blurred and unclear images. Image preprocessing operations can reduce or even eliminate the impact of shooting on subsequent processing.

[0089] Therefore, the image needs to be de-reflected before stitching:

[0090] Specifically, the brightness channel I of the input image is treated with Gaussian kernels of different scales, G σ Perform blur processing and calculate the average of the three blurred images as the illumination estimation image B. The expression is as follows:

[0091]

[0092] Where B represents the smoothed estimated value of the image brightness, which is used to approximate the basic lighting components of the environment; I represents the brightness information of the image, which comes from the V channel in the HSV space; represents the convolution operation; G σ3 ,G σ15 ,G σ30 represents Gaussian filters with standard deviations σ of 3, 15, and 30, respectively, used to simulate different light diffusion scales.

[0093] Calculate the reflection layer using Retinex theory:

[0094] R=log(I+∈)-log(B+∈)

[0095] ∈ is 0.01 to prevent zero value.

[0096] The polarization coefficient K is introduced to suppress the reflected light intensity. The formula is:

[0097]

[0098] Where θ is the polarization angle. Due to the different lighting conditions caused by cloudy days and noon, the fixed polarization angle is changed to adaptive calculation:

[0099]

[0100] Among them, avg(L highliaght )、avg(L shadow ) represent the average brightness of the highlight and shadow areas respectively. The final reflection layer correction formula is:

[0101] R new =R(1-K.λ)

[0102] Where R represents the original Retinex difference image, which represents the estimated reflection component of the highlight area in the image; λ is a global adjustment factor used to control the overall strength of reflection suppression, which is usually an empirical value ranging from 0.9 to 1.2; R new This is the final reflection correction image, which is used to suppress the brightness contribution of high reflection areas.

[0103] Convert the image to HSV space, suppress the reflection of the luminance channel (V), and adjust the chrominance (H, S) to compensate for color distortion:

[0104] H new =H.(1-0.3R new )

[0105] S new =S.(1-0.2R new )

[0106] Among them, H is the hue channel value of the original image in the HSV space, S is the saturation channel of the original image in the HSV space, and H new is the adjusted hue channel value, S new is the adjusted saturation channel value.

[0107] To further enhance the structure preservation capability of the reflection-suppressed image, after suppressing highlights, this paper uses a detail restoration method based on guided filtering to smooth the edges of the luminance image while preserving texture features. The optimization objective function is as follows:

[0108]

[0109] Among them, a and b are chroma channels, ∈ is 0.01 to prevent zero value; I j represents the guidance map intensity of pixel j in the input image (usually selected from the grayscale or brightness channel of the original image), p j Represents the intermediate image after the reflection suppression process (such as R new) is the value of pixel j in .

[0110] Image processing is achieved through a multimodal fusion architecture, which combines three-level Gaussian kernel multiscale decomposition, Retinex theory, a polarization physical model, and a λ adaptive adjustment mechanism. The three-level Gaussian kernel multiscale decomposition, combined with the polarization physical model, achieves precise separation of reflection layers in HSV space. Retinex theory is innovatively coupled with optical polarization characteristics, dynamically adjusting the intensity of reflection suppression through an adaptive λ coefficient. Matrix optimization and memory pre-allocation strategies improve processing speed by 53%. Combined with guided filtering edge-preserving technology (SSIM 0.91) and chromaticity compensation, this approach enables real-time processing, improving specular reflection suppression by 35% and reducing memory consumption by 29% compared to traditional methods. This approach balances algorithm robustness with engineering practicality.

[0111] Step 3: Feature extraction of preprocessed images

[0112] To ensure the robustness of the subsequent matching stage to illumination changes and scale differences, after completing image reflection suppression and chromaticity compensation, an improved SIFT algorithm is used to extract rotation and scale invariant features based on the characteristics of airport pavement such as weak texture, repeated markings, and multi-scale changes, thereby improving the feature extraction speed while maintaining the uniformity of feature point distribution.

[0113] The SIFT algorithm's scale-space detection and keypoint localization achieve multi-scale feature extraction by constructing a Gaussian scale space. First, the preprocessed image undergoes multi-scale blurring: Gaussian kernels with increasing standard deviations are used to smooth the image layer by layer, forming a scale-space pyramid consisting of multiple octaves. Within each octave, scale expansion is achieved by gradually increasing the Gaussian kernel parameters. Adjacent octaves are connected by downsampling (half the resolution), covering the scale range from local details to global structures.

[0114] In order to improve the alignment accuracy of feature points in the same structural area in multi-scale images, the present invention adds a pyramid position encoding processing step before performing SIFT feature extraction, which specifically includes:

[0115] (1) Construct the preprocessed image into a multi-layer scale pyramid {P0,P1,…,P i ,…P L}, where each layer image P i Corresponding to the flight altitude h during aerial photography i ;

[0116] (2) During the pyramid construction process, pyramid position coding information related to the image position and flight altitude is injected into each layer of the image to enhance the feature alignment capability of the same region in images at different flight altitudes; specifically:

[0117] For each layer image P i The pixel point (x, y) in the image is introduced as follows: position encoding vector e(x, y, h i ):

[0118]

[0119] Among them, W i , H i The width and height of the image layer respectively; h max is the set maximum flight altitude; to ensure the consistency of the feature point positions extracted by SIFT in images at multiple altitudes, after this encoding processing, the overlap of feature point positions in the pyramid scale of the same area photographed at different flight altitudes is improved, effectively reducing the subsequent matching drift problem.

[0120] (3) On the image pyramid with pyramid position coding, the improved SIFT algorithm is used to perform key point detection, main direction determination, gradient histogram calculation and feature descriptor construction in sequence to obtain the image feature point set;

[0121] (4) The extracted feature points are screened and normalized to form the stable feature data required for subsequent image stitching.

[0122] Based on the Gaussian scale space, the Difference of Gaussians (DoG) operator is used to approximate the Laplace extreme value detection. The Gaussian blurred images of adjacent scales are subtracted to generate a DoG pyramid, and extreme points are searched in the three-dimensional space: each pixel needs to be compared with the 8 neighboring points at the same level and the 18 neighboring points at the corresponding positions of the adjacent scales to screen out candidate key points with significant scale response.

[0123] Candidate keypoints undergo sub-pixel refinement and filtering: Taylor expansion is used to fit the DoG function, iteratively calculating the precise location and scale of extreme points, and eliminating low-contrast responses (threshold control). The principal curvature ratio is further analyzed using the Hessian matrix to eliminate unstable points in edge regions (curvature ratio threshold constraint). The remaining keypoints exhibit spatial position and scale parameter stability and high discrimination, providing a robust foundation for subsequent feature description. Cross-scale extreme value detection and geometric verification are used throughout the entire process to ensure the strong invariance of feature points to scale and rotation changes.

[0124] In the original SIFT algorithm, direction assignment is performed by calculating the gradient direction histogram of the key point neighborhood and selecting the main direction. This time, a faster gradient calculation method is used, replacing the square root with the absolute value. After improvement, the speed is increased by 200%, but the accuracy loss is less than 5%.

[0125] m(x,y)=|L(x+1,y)-L(x-1,y)|+|L(x,y+1)-L(x,y-1)|

[0126] Among them, m(x,y) refers to the modulus of the pixel at that position. After completing the gradient calculation of the key point, the original SIFT algorithm uses a histogram to count the gradient and direction of the pixels in the neighborhood. The gradient histogram divides the direction range of 0 to 360 degrees into 36 columns, with each column of 10 degrees. This method rigidly assigns the direction angle to a fixed interval, resulting in quantization error. Therefore, bilinear interpolation is used to reduce the edge quantization error. The formula is as follows:

[0127]

[0128] Among them, Δθ,θ′,θ min They represent the 10° interval of the direction interval, the actual calculated direction angle, and the lower limit angle of the current direction interval respectively; w is the interpolation weight, which is used to distribute the gradient amplitude to adjacent intervals. After comparison, the direction allocation accuracy is improved by about 30%. The original method of Gaussian weighting adopts a fixed standard deviation and cannot adapt to multi-scale changes. The weighting range is expanded at fine scales to enhance the stability of the road surface marking edge; the coarse scale suppresses over-smoothing. Therefore, the Gaussian kernel parameters are dynamically adjusted, and the formula is:

[0129] σ adapt =max(1.2σ,3)

[0130] Among them, σ, σ adapt They represent the basic standard deviation and the adaptively adjusted standard deviation, respectively, ensuring that the weighted range takes into account both local details and global stability. The weighted range is finer and the stability of feature size is improved.

[0131] Finally, a scale-invariant SIFT descriptor is constructed with four parameters: the two-dimensional position information (x and y), the scale, and the main direction. The resulting SIFT descriptor is a 128-dimensional vector. The 5-dimensional position encoding is then directly concatenated with the 128-dimensional vector to form a 133-dimensional enhanced descriptor, which provides a more reliable matching foundation for solving the subsequent geometric transformation model.

[0132] Step 4: Feature point matching, constructing the road surface image splicing relationship, such as Figure 3 As shown, the specific steps include the following steps.

[0133] Use descriptors to match feature points. For each feature point descriptor, find the most similar one in the descriptor set of another image.

[0134] The KNN matching method is used to find the two nearest matching points, and the distance ratio between the nearest neighbor and the next nearest neighbor is calculated. Only matching pairs with a ratio lower than the threshold are retained.

[0135] The feature descriptors of the two images are constructed into sets A and B respectively, where set A represents all descriptors in image A and set B represents all descriptors in image B.

[0136] Traditional one-way matching methods (e.g., matching feature points from image A to image B) may have asymmetry and easily lead to mismatching.

[0137] Therefore, a two-way matching verification is performed between images A and B. That is, a feature point in image A is found to correspond to the most similar feature point in image B, and then the feature point in image B is verified to be the same feature point in image A. Only feature points that pass the two-way verification are considered valid matching points.

[0138] For set D A Each descriptor d in a , in set D B Find the descriptor d with the smallest distance b , and verify d b In D A Is the nearest neighbor in d a Only feature point matching pairs that pass bidirectional verification are retained to reduce mismatched points.

[0139] Step 5: Eliminate mismatched points.

[0140] Although the two-way matching has preliminarily screened out high-confidence feature pairs, since the airport concrete pavement is composed of the same pavement panels and similar markings may exist on each pavement image, eliminating the mismatched point pairs in the repeated texture area is a relatively important step. In order to improve the accuracy of matching point screening, the present invention introduces a mismatch elimination strategy (GMS) based on grid statistics. First, the image is divided into a uniform grid, and the displacement vector of each pair of matching points is calculated, and its support in the local 3×3 grid neighborhood is counted. If the displacement direction of a matching point deviates significantly from that of most points in the neighborhood (the support is lower than the set threshold), it is determined to be a mismatch and eliminated, thereby improving the consistency and reliability of the matching results.

[0141] First, divide the image into N×M uniform grids (default 30×30), and the size of each grid is:

[0142]

[0143] Among them, W and H are the width and height of the image, grid width ,grid height is the grid width and height.

[0144] Map the matching points to each grid, for each matching pair (p i ,q i), calculate the grid coordinates to which it belongs:

[0145]

[0146] Among them, grid x , grid y is the matching point p i The horizontal and vertical numbers of the grid, To round down, p i .x,p i .y represents point p i Position on the x-axis and y-axis.

[0147] For each matching pair (p i ,q i ) calculate its displacement vector Δx i =q i -p i , and calculate its support score in the 3×3 neighborhood grid:

[0148]

[0149] Among them, N(i) is the matching point p i The 3×3 neighborhood grid set, grid j are all matching points in the jth grid, Δx i , Δx k is the displacement vector of matching pair i and k, τ is the displacement difference threshold, usually set to 3 pixels, Ⅱ(.) represents the indicator function, which is 1 when the condition is met and 0 otherwise, S i Represents the support score obtained by matching point i.

[0150] To adapt to different texture density scenarios, this embodiment defines the average support of the full-image grid as:

[0151]

[0152] in, represents the average support of the entire grid, S j represents the average support score of the matching points in the jth grid, D is the total number of grids obtained by image division, and As a benchmark, set a unified threshold:

[0153]

[0154] Among them, a is the strictness coefficient, which is 3.5 after cross-validation; taking the square root can suppress the false rejection caused by excessive threshold in high-density scenarios. If S i >T i, it is determined to be consistent with local motion consistency and retained, otherwise it is considered as a false match and removed.

[0155] In the area of ​​repeated texture on the airport pavement, the displacement direction of the correct match is consistent, while the mismatch produces random offsets due to the repeated pattern, which can effectively resist the interference of repeated textures. It can also adapt to low-light scenes, and the dynamic threshold T i The accuracy decreases as the matching density decreases, avoiding over-selection. GMS significantly improves the robustness of feature matching by statistically analyzing the motion consistency of matching points using gridding, combined with dynamic thresholding. Compared to traditional methods (such as RANSAC), it is computationally more efficient and is particularly suitable for low-texture, high-dynamic interference scenarios such as airport pavement splicing.

[0156] Step 6: Calculate the homography matrix to ensure the stitching accuracy, such as Figure 4 As shown, the specific steps include the following steps.

[0157] After obtaining a high-purity matching point set, in order to achieve accurate and efficient registration between different images, it is necessary to refer to their various morphological changes and thus construct a real image transformation model that can be as close to these transformations as possible.

[0158] In order to improve numerical stability, the input point coordinates are normalized (translated and scaled) to avoid numerical instability. First, the mean u of the feature point coordinates needs to be calculated. x ,u y and standard deviation σ x ,σ y , normalize the coordinates and construct normalized matrices T and T′. The normalized coordinates are Generate two rows of constraint equations for each pair of points:

[0159]

[0160] Among them, x i ,y i represents the coordinates of the i-th matching point in the reference image, x i ′,y i ′ represents the coordinates of the i-th matching point in the target image, A i Represents the linear constraint term of the 2×9 homography matrix generated by the i-th pair of points.

[0161] Stack all Aces i Get the matrix A, which is a 2n*9 matrix. Perform singular value decomposition on A and get V T The last line of is the solution of h, which is then denormalized to restore the actual homography matrix and reshaped to get H:

[0162] H=T′ -1 .reshape(h).T

[0163] Where h is the last column of V and represents the vectorized form of the homography matrix; H represents the homography transformation matrix, which is used to describe the projective transformation relationship between images; reshape(h) means converting the 9-dimensional vector h into a 3×3 matrix.

[0164] Although the DLT algorithm provides a preliminary homography estimate at a global scale, it still has deficiencies in modeling large perspective changes or local deformations. To this end, a three-level Gaussian pyramid is constructed to quickly converge the global transformation parameters at the low-resolution level. Subsequently, the re-projection error (RMSE) is reduced from 2.8 pixels to 1.2 pixels. Input the original images I1 and I2 and construct the Gaussian pyramid:

[0165]

[0166] in, is the pyramid image of the kth layer; I k-1 is the pyramid image of the k-1th layer; GaussianBlur represents the Gaussian filtering operation, blurring details and retaining the main structure; ↓2 represents downsampling by two times, followed by layered optimization, with H1 used as the initial value for the top layer and LM algorithm optimization:

[0167]

[0168] H coarse Upsample and use it as the initial value of the next layer, refine it layer by layer, optimize it repeatedly, and finally output the optimized homography matrix H2.

[0169] To further address the problem of stitching boundary misalignment, the APAP algorithm is introduced to perform elastic optimization of local grids under the global homography constraint. First, the image is divided into a 20×20 pixel grid, which is suitable for most scenarios. By alternating fixed variables, global and local parameters are optimized in stages:

[0170] Fixed global homography matrix H global , optimize the local homography matrix H j , so that under the global constraint, each mesh is allowed to be optimized independently to minimize the local deformation. The formula is:

[0171]

[0172] where p i 、p i ′ is the homogeneous coordinate form of the matching point pair in the original image and the target image, H global is the global homography matrix, which is used to describe the overall projection transformation between images. jis the independent homography matrix of the jth local grid, and λ is a balancing weight, usually set to 0.1, which controls the weight between global consistency and local deformation. Each grid optimization is an independent least squares problem that can be solved using the LM algorithm.

[0173] Fixed local homography matrix H j , optimize the global homography matrix H global , so that it adjusts the global transformation according to the local optimization results to ensure overall consistency. The formula is:

[0174]

[0175] Update H by the mean or weighted average global , repeat the above steps until convergence. After alternating optimization, the global homography matrix H is obtained global and locally adjusted {H j}, and finally apply the corresponding H to each grid during splicing j .

[0176] Through coordinate normalization, multi-scale optimization, and APAP global-local joint optimization, the homography matrix solution significantly improves accuracy, efficiency, and adaptability. This approach is particularly suitable for complex scenarios such as airport pavement stitching (repeated textures, dynamic interference, and large viewing angle changes). While ensuring global consistency, this improved method allows for local elastic deformation, effectively resolving stitching boundary misalignment and laying the foundation for high-precision image fusion.

[0177] Step 7: Image fusion to enhance the visual effect of the road surface panorama.

[0178] Image fusion refers to processing the matched stitched images to remove the stitching traces. The quality of the processed images will directly affect the final display effect.

[0179] Preferably, the quality of drone aerial images is relatively close, and the weighted average method is not only relatively simple and computationally efficient, but also the spliced ​​road surface images can obtain clearer road surface conditions.

[0180] The essence of this method is to weight the grayscale values ​​of all pixels in the overlapping area of ​​the image rather than simply averaging them. The formula is as follows:

[0181]

[0182] Among them, I1∩I2 represents the image overlapping area, w1(x,y) represents the weighted average value of the pixel of the to-be-stitched image 1 at this pixel point, w2(x,y) represents the weighted average value of the pixel of the to-be-stitched image 2 at this pixel point, 0 < w1(x,y), w2(x,y) < 1, w1(x,y) + w2(x,y) = 1, I1(x,y) represents the pixel gray value of image 1 at the position (x,y); I2(x,y) represents the pixel gray value of image 2 at the position (x,y). Based on the weight assignment of the Euclidean distance, the closer to the area of the to-be-stitched image 1, the greater the weight. The formula is as follows:

[0183]

[0184] Where x start 、x end are respectively the starting and ending coordinates of the overlapping area in the horizontal direction, y start 、y end are respectively the starting and ending coordinates of the overlapping area in the vertical direction. After the above steps of stitching and fusion, the effect of the airport pavement is as Figure 5 shown.

[0185] The present invention provides a high-precision large-field two-dimensional image imaging method for airport pavements. This method collects pavement images by an unmanned aerial vehicle according to preset parameters on the designated routes and waypoints at the airport; secondly, preprocesses the collected images. In the preprocessing stage, combining the polarization optical model and the Retinex theory, through the dynamic reflection suppression coefficient and the edge-preserving technology of guided filtering, the specular reflection of water stains and oil stains on the pavement is eliminated, and at the same time, the crack edge details are retained. In the feature extraction stage, the SIFT algorithm is improved, and bilinear interpolation and a dynamic Gaussian kernel are used to provide high-density and high-precision features for subsequent registration. In the feature point matching and false matching point elimination stage, a two-way KNN-GMS joint strategy is proposed. Through grid motion statistics and dynamic density thresholds, the false matching rate in the repeated marking scenario is ≤8%. A multi-scale pyramid-APAP elastic model is constructed, and global homography optimization is combined with local grid deformation compensation to support seamless stitching of pavements of hundreds of meters. In the image fusion stage, the gray values of all pixel points in the overlapping area of the images are weighted, rather than simply averaging directly. The weights are assigned based on the Euclidean distance, and the closer to image 1, the greater the weight.

[0186] The present invention also discloses an airport pavement large-field two-dimensional image imaging system, including:

[0187] An image acquisition module for acquiring pavement images collected by an unmanned aerial vehicle;

[0188] A preprocessing module for preprocessing the pavement images to obtain preprocessed images;

[0189] A feature extraction module is used to extract feature points from the preprocessed image;

[0190] A feature matching module is used to perform feature point matching on feature points in the preprocessed image;

[0191] Elimination module, used to filter the matched feature point pairs and eliminate false matching points;

[0192] A stitching module is used to establish an image transformation model based on the matched feature points, and stitch the matched images to obtain a stitched image;

[0193] The image fusion module is used to perform image fusion based on the stitched images to obtain a panoramic view of the airport pavement.

[0194] The present invention also discloses a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for capturing a large-field-of-view two-dimensional image of an airport pavement are implemented. The memory may include internal memory, such as a high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device. The processor, network interface, and memory are interconnected via an internal bus. The internal bus may be an industrial standard architecture bus, a peripheral component interconnect standard bus, an extended industrial standard architecture bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory is used to store programs. Specifically, the programs may include program code, and the program code includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0195] The present invention also discloses a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the method for capturing a large-field-of-view two-dimensional image of an airport pavement. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory and / or cache memory, etc. The non-volatile memory may include read-only memory, a hard disk, a flash memory, an optical disk, a magnetic disk, etc.

[0196] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, optical storage, etc.) containing computer-usable program code.

[0197] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0198] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0199] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A method for imaging large-field two-dimensional images of airport pavement, characterized in that: include: S1. Obtaining road surface images collected by a UAV; S2. Preprocessing the road surface image to obtain a preprocessed image; S3, extracting feature points from the preprocessed image; S4, performing feature point matching on the feature points in the preprocessed image; S5. Filter the matched feature point pairs to eliminate mismatched points. S6. Based on the matched feature points, the matched images are stitched together to obtain a stitched image. S7. Based on the stitched images, perform image fusion to obtain a panoramic view of the airport pavement.

2. The method for imaging a large-field-of-view two-dimensional image of an airport pavement according to claim 1, characterized in that: In S2, the preprocessing is to remove the reflection of the image, specifically: through the three-level Gaussian kernel multi-scale decomposition and polarization physical model, the reflection layer is accurately separated in the HSV space, the Retinex theory is coupled with the optical polarization characteristics, and the reflection suppression intensity is dynamically adjusted through the global adjustment factor; The image is converted to HSV space, reflection is suppressed on the luminance channel, and chrominance is adjusted to compensate for color distortion. After suppressing highlights, a detail recovery method based on guided filtering is used to smooth the edges of the luminance image.

3. The method for imaging a large-field two-dimensional image of an airport pavement according to claim 1, characterized in that: In S3, the optimized sift algorithm is used to extract feature points from the preprocessed image, which specifically includes the following steps: (1) constructing an image pyramid containing multiple scales based on the preprocessed image, wherein each layer of the image pyramid corresponds to the flight altitude at the time of shooting; (2) During the pyramid construction process, pyramid position coding information related to the image position and flight altitude is injected into each layer of the image to enhance the feature alignment capability of the same region in images at different flight altitudes; (3) On the image pyramid with pyramid position coding, the improved SIFT algorithm is used to perform key point detection, main direction determination, gradient histogram calculation and feature descriptor construction in sequence to obtain the image feature point set; (4) Screening and normalizing the extracted feature points to form a stable feature point set required for subsequent image stitching; In step (3), the improved SIFT algorithm uses a faster gradient calculation method to calculate the gradient histogram, replacing the square root with the absolute value: m(x,y)=|L(x+1,y)-L(x-1,y)|+|K(x,y+1)-L(x,y-1)| Among them, m(x,y) refers to the modulus of the pixel at that position. After completing the gradient calculation of the key point, the SIFT algorithm uses a histogram to count the gradient and direction of the pixels in the neighborhood. The gradient histogram divides the direction range of 0 to 360 degrees into 36 columns, where each column is 10 degrees, and the direction angle is rigidly assigned to a fixed interval. Then, bilinear interpolation is used to reduce the edge quantization error. The formula is as follows: Among them, Δθ,θ ′ ,θ min They represent the 10° interval of the direction interval, the actual calculated direction angle, and the lower limit angle of the current direction interval respectively; w is the interpolation weight, which is used to distribute the gradient amplitude to adjacent intervals; Dynamically adjust the Gaussian kernel parameters, the formula is: s adapt =max(1.2σ,3) Among them, σ, σ adapt They represent the basic standard deviation and the standard deviation after adaptive adjustment respectively; The construction of the feature descriptor includes four parameters: two-dimensional position information x and y, scale and main direction; based on these four parameters, the generated SIFT descriptor is a 128-dimensional vector; then the 5-dimensional position code is directly concatenated with the 128-dimensional vector to form a 133-dimensional enhanced descriptor.

4. The method for imaging a large-field two-dimensional image of an airport pavement according to claim 1, characterized in that: In S4, multi-scale feature descriptors are used for feature point matching. For each feature point descriptor, the most similar one is found in the descriptor set of the other image. The specific process is as follows: To address the problem of feature similarity caused by repetitive textures of airport pavement markings, a bidirectional KNN matching strategy is adopted. Combining with a distance ratio threshold, a high-purity set of matching pairs is provided for subsequent geometric verification through a bidirectional verification mechanism.

5. The method for imaging a large-field-of-view two-dimensional image of an airport pavement according to claim 1, characterized in that: In S5, the GMS method is used to distinguish correct matches from incorrect matches by analyzing the motion consistency of feature matches in a local grid. The specific steps are as follows: A1: First, the image is divided into N×M uniform grids, and the matching points are mapped to each grid. For each matching pair (p i ,q i ), calculate the grid coordinates to which it belongs; A2: For each matching pair (p i ,q i ) calculate its displacement vector Δx i =q i -p i , and calculate its support score S in the 3×3 neighborhood grid i ; A3: Dynamically calculate the retention threshold T based on the density of matching points in the grid i :If S i >T i , then it is considered that the matching pair (p i ,q i ) is reliable, the matching pair is retained; otherwise, it is considered a false match and removed.

6. The method and related device for imaging large-field two-dimensional images of airport pavement according to claim 1, characterized in that: In S6, the normalized direct linear transformation method is used to stitch the images that have been matched to estimate the homography matrix. The steps include: A1: Normalize the coordinates of the matched feature points in the matched images so that their centers are aligned with the origin of the coordinates and uniformly scaled so that the average distance from the points to the origin is normalized to a constant; A2: Based on the normalized point coordinates, construct the linear constraint relationships between each group of matched points to establish the projective transformation model between images; A3: Integrate the linear relationships of all point pairs into a set of linear equations and use the singular value decomposition method to solve for the optimal homography matrix parameters; A4: Denormalize the obtained homography matrix parameters and restore them to the original image coordinate system to obtain the final geometric transformation model for image stitching.

7. The method for imaging a large-field-of-view two-dimensional image of an airport pavement according to claim 1, characterized in that: In S7, image fusion specifically means: performing weighted processing on the gray values of all pixel points in the overlapping area of the images; Among them, I1∩I2 represents the overlapping area of the images, w1(x,y) represents the weighting value of the image to be stitched 1 at the pixel point (x,y), w2(x,y) represents the weighted average value of the image to be stitched 2 at the pixel point (x,y), 0 < w1, w2 < 1, w1 + w2 = 1; I1(x,y) represents the pixel gray value of image 1 at the position (x,y); I2(x,y) represents the pixel gray value of image 2 at the position (x,y); Based on the weight assignment using the Euclidean distance, the closer to the area of the image to be stitched 1, the greater the weight. The formula is as follows: Among them, x start 、x end are the starting and ending coordinates of the overlapping area in the horizontal direction, y start 、y end They are the starting and ending coordinates in the vertical direction respectively.

8. A large-field-of-view two-dimensional imaging system for airport pavement, characterized in that: including: An image acquisition module for acquiring pavement images collected by an unmanned aerial vehicle; A preprocessing module for preprocessing the pavement images to obtain preprocessed images; A feature extraction module for extracting feature points from the preprocessed images; A feature matching module for performing feature point matching on the feature points in the preprocessed images; An elimination module for filtering the matched feature point pairs to eliminate incorrect matching points; A stitching module for establishing an image transformation model based on the matched feature points and stitching the images that have been matched to obtain a stitched image; An image fusion module for performing image fusion based on the stitched image to obtain a panoramic image of the airport pavement.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, it implements the airport pavement large field-of-view two-dimensional image imaging method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the airport pavement large field-of-view two-dimensional image imaging method according to any one of claims 1 to 7.

Images