Intelligent Assessment Method for Airport Runway Pavement Health Based on Computer Vision
By combining computer vision-based methods with partitioning and multi-scale feature pyramids, we have achieved accurate identification and three-dimensional quantification of runway pavement defects. This solves the problems of defect feature loss and dynamic monitoring in existing technologies, and improves the comprehensiveness and accuracy of the assessment.
Patent Information
- Application Number
- CN202611556550.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-09
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies for airport runway pavement health assessment suffer from problems such as easy loss of disease characteristics, insufficient accuracy in multi-scale disease identification, inability to quantify three-dimensional damage, and difficulty in dynamically tracking disease evolution, thus failing to meet the requirements of high-reliability and high-safety routine health monitoring for airport runways.
Using a computer vision-based approach, runway images are collected through inspection equipment. Interference areas and suspected defect areas are identified by region. A multi-scale feature pyramid is constructed, and combined with edge constraints, the two-dimensional contours and three-dimensional morphologies of defects are output. Temporal image registration and differential analysis are performed, and a comprehensive judgment is made in conjunction with airport pavement health evaluation standards.
It achieves pixel-level precise segmentation of runway pavement defects, improving the comprehensiveness and accuracy of defect identification, quantifying three-dimensional damage, dynamically monitoring defect evolution trends, and generating health assessment reports.
Smart Images

Figure CN123071331A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and airport runway health monitoring technology, specifically to a computer vision-based intelligent assessment method for the health status of airport runway pavement. Background Technology
[0002] Airport runway pavement refers to the surface structure of an airport runway used for aircraft takeoff, landing, and taxiing. It is the part of the runway that directly contacts the ground and is one of the most critical infrastructures of an airport.
[0003] Civil aviation airport runway pavements are prone to various defects. Current mainstream assessment methods mostly rely on monocular two-dimensional vision inspection schemes or directly transplanted general road image processing algorithms. Traditional monocular schemes use globally uniform image preprocessing, failing to address specific interference zones such as runway markings, strong glare, and oil stains. This easily leads to the loss of subtle defect features. Furthermore, relying solely on single-scale feature segmentation can result in missed detection of small defects and fragmented segmentation of large defect areas. They can only output two-dimensional planar data of defects, unable to quantify three-dimensional damage indicators such as pothole depth and settlement. Additionally, they only support single static inspections, lacking time-series image registration and differential analysis processes, and are therefore inadequate for... Filtering out invalid pixel interference from lighting and temporary equipment makes it difficult to track the long-term evolution trend of pavement defects and predict potential risks. Furthermore, general road algorithms have poor adaptability, lack specific identification constraints for pavement defects, and do not match civil aviation runway operation and maintenance evaluation standards. As a result, the evaluation results lack industry-specific quantitative basis. In summary, existing methods generally suffer from defects such as easy loss of defect features under complex working conditions, insufficient accuracy in multi-scale defect identification, inability to quantify three-dimensional damage, and difficulty in dynamically tracking defect evolution. They also have shortcomings such as inaccurate detection results, single evaluation dimensions, and inability to support preventive operation and maintenance decisions, making it difficult to meet the high-reliability and high-safety routine health monitoring needs of airport runways. Summary of the Invention
[0004] This invention addresses the technical problems existing in the prior art by providing a computer vision-based intelligent assessment method for the health status of airport runway pavement.
[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A computer vision-based intelligent assessment method for airport runway pavement health status, comprising the following steps: S1: The inspection equipment collects visible light and binocular stereo images of the runway, removes invalid images, and obtains valid raw image data; S2: Identify interference areas and suspected disease areas in the original image by partitioning them, perform image transformation processing on each region to remove interference and retain disease features, and obtain an optimized image; S3: Construct a multi-scale feature pyramid for the optimized image, fuse deep and shallow image features and combine edge constraints to output the two-dimensional contour, location and range information of the disease; S4: Based on the acquired binocular stereo images, combined with the output location information of suspected disease areas, calculate the parallax and reconstruct the three-dimensional shape of the pavement, map pixel and depth parameters, and measure the three-dimensional deformation index of the disease. S5: Based on the retained historical time-series images of the same area, the historical time-series images are registered with the suspected disease area as the benchmark. Environmental interference is removed by pixel difference, and the disease changes are statistically analyzed by combining data from multiple periods to analyze the evolution trend. S6: Integrate two-dimensional segmentation results, three-dimensional deformation indicators, and time-series evolution data, and combine them with airport pavement health evaluation standards to make a comprehensive judgment, classify health and risk levels, and generate a pavement health assessment report.
[0006] In a preferred embodiment, in step S1, the inspection equipment equipped with an image acquisition module travels along the entire area of the airport runway, simultaneously capturing visible light sequence images and binocular stereo images. The acquired batch of images are screened one by one, and images with no analytical value, such as those that are severely blurred, have a view that deviates from the runway, have severe obstruction, or have abnormal exposure, are removed. Images with complete view and qualified image quality are retained and uniformly collected to form an effective original image dataset.
[0007] In a preferred embodiment, in step S2, for the input original runway image, the region is divided according to the visual features of the image, and the gray value range, color distribution, texture gradient and edge contour information of the image are statistically analyzed by pixel to obtain the basic visual feature data corresponding to each pixel. Based on gray value and color features, a preliminary classification is made: if the pixel gray value is consistently in the extremely high range and there is no obvious texture detail in the local brightness change, it is determined to be a strong light reflection area; if the pixel gray value is lower than the average level of normal pavement and the color shows the dark characteristics of oil stains and dirt, it is determined to be a surface oil stain area; if the pixel shows the high contrast color unique to runway markings and the local pixels form regular linear contours, it is determined to be a runway marking area. Further differentiation is based on texture and edge features: For areas not marked as reflective, oily, or marked, the uniformity of local texture is statistically analyzed: areas with uniform texture and no obvious abrupt changes are judged as normal and intact pavement areas; if local pixels show abrupt changes in grayscale, texture breaks, or irregular edges, and do not meet the characteristics of markings, reflective, or oily, they are judged as suspected defective areas with abnormal texture. It distinguishes between runway marking areas, surface oil and stain areas, areas with strong light reflection, normal and intact pavement areas, and suspected defect areas with abnormal texture, thus obtaining a precise distinction between interference areas and defect areas to be detected. For different regions, corresponding pure image transformation operations are applied: reflection suppression processing is performed on reflective areas to eliminate overexposed pixels in strong light, texture smoothing filtering is performed on interference areas such as markings and debris, contrast correction is performed on uneven brightness areas, the overall image quality is unified, and visual interference caused by lighting, markings, stains and debris is specifically eliminated. While optimizing the interference area, the edge contours, texture details and pixel features of the suspected disease area are preserved. After completing the partitioned image transformation and interference removal, an optimized standard image with uniform image quality, interference information removal and complete and prominent disease features is obtained.
[0008] In a preferred embodiment, in step S3, a single optimized image is scaled and decomposed into layers step by step to obtain image layers of different resolutions. The high-resolution layer retains the fine texture and local details of the image, while the low-resolution layer presents the overall shape and large-scale regional morphology, forming an image layer system covering different observation scales. Extract visual features corresponding to each level of the image. Extract basic visual features of the image according to level: For high-resolution levels, extract detailed features of local pixel gray-level gradient, texture direction, and edge breakpoints; for low-resolution levels, extract global features of overall gray-level distribution, contour shape, and connected region morphology to form a set of visual features corresponding to each level. Combine the local detailed features of high-resolution levels with the global morphological features of low-resolution levels.
[0009] In a preferred embodiment, the edge boundary is defined based on pixel grayscale changes and contour transition rules to lock the boundary between the pavement area and the disease area. Using a single pixel as the smallest unit, combined with the fused features and edge constraints, normal pavement pixels and diseased pixels are distinguished one by one to completely divide the areas where various diseases are located. Statistical segmentation results are used to extract two-dimensional data of the disease's outline, pixel coordinates, and area coverage. Contour line extraction: Edge detection is performed on the set of defective pixels: Each defective pixel is checked one by one. If its neighborhood contains both defective pixels and normal pavement pixels, then the pixel is determined to be a defective edge point. All edge points are connected sequentially according to spatial connection relationship to form a closed defective contour line. Pixel coordinate location extraction: For all pixels in the diseased area, read their horizontal and vertical coordinate values in the image coordinate system, count the bounding rectangle of the diseased area, and record the coordinates of the upper left and lower right corners of the rectangle. Region coverage extraction: Count the total number of all diseased pixels within the diseased area, calculate the pixel area of the diseased area, combine it with the actual physical resolution of the image, convert the pixel area into the actual coverage of the diseased area, and record the length and width of the bounding rectangle to fully represent the regional scale information of the disease.
[0010] In a preferred embodiment, in step S4, based on the acquired binocular stereo image and combined with the output location information of the disease area coordinates and range, the corresponding disease area is accurately located in the binocular stereo image, the calculation target range is defined, the pixels at the same spatial position in the binocular stereo image are compared, the position offset of the same pixel in the left and right views is counted, the disparity data is calculated, and the disparity distribution map of the whole area and the disease area is obtained. Based on disparity data and the geometric relationship of binocular imaging, 2D planar image information is converted into spatial stereo information. Using the calculated disparity data, the positional offsets of each corresponding pixel in the left and right views are matched to form corresponding pixel relationships, constructing a set of pixel correspondences within the image plane. According to the fixed shooting position and imaging angle of the binocular camera, the correspondence logic between pixel offset and spatial depth is determined: a larger pixel offset indicates a closer spatial position to the camera, and a smaller offset indicates a farther distance. For each matched pixel, its 2D coordinates in the image plane are combined with the corresponding rules... Using parallax data, the position of the pixel in three-dimensional space is calculated. Points on the image plane are mapped to a spatial coordinate system. All mapped pixels are traversed and arranged according to their spatial positions to form a continuous three-dimensional point cloud. The point cloud data is constructed into surfaces according to adjacency relationships to restore the true three-dimensional surface morphology of the entire runway surface and the damaged areas. This completes the conversion from two-dimensional planar information to spatial three-dimensional information, restores the true three-dimensional surface morphology of the entire runway surface and the damaged areas, and establishes the corresponding conversion relationship between image pixel coordinates and actual spatial depth, converting each pixel into a corresponding spatial depth value.
[0011] In a preferred embodiment, based on depth data, deformation parameters such as pothole depth, pavement settlement, and surface protrusion height are calculated for the identified diseased areas. Pothole depth calculation: In the 3D data of the diseased area, the location with the largest depth value is selected, the corresponding depth data is read, and the depth value is compared with the depth of the reference surface. The difference between the two is the pothole depth. Calculation of pavement settlement: Statistically analyze the average depth data of the entire affected area and compare it with the depth of the reference surface. The difference between the two is the pavement settlement of that area. Surface protrusion height calculation: In the three-dimensional data of the diseased area, the position with the smallest depth value is selected, the corresponding depth data is read, and the depth of the reference surface is compared with the depth value. The difference between the two is the surface protrusion height.
[0012] In a preferred embodiment, in step S5, based on time-series images of the same runway section at different times, and using the determined contours and coordinates of the defect area as a reference, the position, angle and scale of each time-series image are adjusted to achieve precise spatial alignment of the pavement and defect area in multiple images, and pixel-by-pixel difference calculation is performed on the registered multi-period images. By comparing pixel differences between adjacent time periods, and based on the output disease area outline and coordinates, the position, angle, and scale of images of the same section collected at different time periods are adjusted to ensure that pixels at the same physical location in multiple images correspond, forming an aligned image sequence. After alignment, adjacent images are compared by pixel position, and the grayscale and color values of pixels at the same position are read to determine whether the value of the pixel has changed in the two images, thus obtaining the pixel difference result. Pixels with changed values are marked as difference pixels, and they are merged according to spatial adjacency to form difference regions. The set of regions where images have changed in adjacent time periods is initially identified. Combining the position, shape, and distribution characteristics of the difference regions, pixel changes caused by changes in lighting, temporary debris, and short-term interference from mobile devices are distinguished. Invalid differences of this type are removed, and pixel regions where the pavement itself has changed are retained.
[0013] In a preferred embodiment, by combining the two-dimensional range of the disease and the three-dimensional deformation index corresponding to multiple images, the changes in the area, boundary range, depth and settlement of the disease area are statistically analyzed period by period. Based on multiple sets of statistical data, the changing patterns of disease morphological expansion and deformation intensification are sorted out, and the development speed and evolution trend of the disease are judged.
[0014] In a preferred embodiment, in step S6, the output two-dimensional location, outline, and range data of the disease are uniformly summarized, and a complete disease dataset covering the morphology, scale, three-dimensional damage, and dynamic changes is formed based on the calculated three-dimensional deformation indicators of disease depth, settlement, and protrusion, as well as the analyzed disease temporal changes and evolution rate data. Based on the airport runway pavement operation and maintenance health evaluation standard, the integrated parameters of various defects are compared with the standard thresholds one by one. Taking into account multiple factors such as defect size, degree of deformation, and development speed, a comprehensive assessment of pavement damage is completed. Based on the comprehensive assessment results, the pavement health status of each section of the runway is classified into levels. At the same time, based on the defect evolution trend, the potential spread of defects and structural damage risks are determined, and the corresponding risk levels are marked.
[0015] The beneficial effects of this invention are: by using adaptive layered image enhancement for complex runway interference, it can specifically eliminate visual interference caused by outdoor environments such as strong light reflection, surface oil stains, runway markings, rain and snow stains, while completely preserving the edge contours and texture details of defects such as cracks and potholes, effectively solving the problem of loss of defect features due to interference in traditional solutions. By constructing a multi-scale feature pyramid and fusing deep and shallow image features, combined with edge constraints, this invention can simultaneously adapt to different scales and forms of defects such as microcracks, network cracks and large-area potholes, avoiding the defects of missing small defects and fragmented segmentation of large defects, achieving pixel-level accurate and complete segmentation of pavement defects, and improving the comprehensiveness and accuracy of defect identification. Based on binocular stereo image parallax calculation and three-dimensional shape reconstruction, a mapping relationship between pixel coordinates and actual depth is established. It can automatically calculate three-dimensional deformation indicators such as pothole depth, pavement settlement, and surface protrusion height in the diseased area, upgrading traditional two-dimensional visual inspection to quantifiable three-dimensional health assessment, and solving the limitation that two-dimensional images cannot characterize three-dimensional damage. By performing precise registration and pixel difference calculation on runway images of different periods, this invention can filter out short-term interference such as changes in illumination, temporary debris, and mobile devices, extract time-series change data on the area, boundary range, depth, and settlement of the disease, analyze the expansion rate and evolution trend of the disease, and achieve an upgrade from static single detection to long-term dynamic monitoring. Attached Figure Description
[0016] Figure 1 This is a flowchart of the present invention. Detailed Implementation
[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0018] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0019] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.
[0020] like Figure 1 This embodiment provides a computer vision-based intelligent assessment method for the health status of airport runway pavement, comprising the following steps: S1: The inspection equipment collects visible light and binocular stereo images of the runway, removes invalid images, and obtains valid raw image data; S2: Identify interference areas and suspected disease areas in the original image by partitioning them, perform image transformation processing on each region to remove interference and retain disease features, and obtain an optimized image; S3: Construct a multi-scale feature pyramid for the optimized image, fuse deep and shallow image features and combine edge constraints to output the two-dimensional contour, location and range information of the disease; S4: Based on the acquired binocular stereo images, combined with the output location information of suspected disease areas, calculate the parallax and reconstruct the three-dimensional shape of the pavement, map pixel and depth parameters, and measure the three-dimensional deformation index of the disease. S5: Based on the retained historical time-series images of the same area, the historical time-series images are registered with the suspected disease area as the benchmark. Environmental interference is removed by pixel difference, and the disease changes are statistically analyzed by combining data from multiple periods to analyze the evolution trend. S6: Integrate two-dimensional segmentation results, three-dimensional deformation indicators, and time-series evolution data, and combine them with airport pavement health evaluation standards to make a comprehensive judgment, classify health and risk levels, and generate a pavement health assessment report.
[0021] In step S1, the inspection equipment equipped with the image acquisition module travels along the entire area of the airport runway, simultaneously capturing visible light sequence images and binocular stereo images. The batch of images acquired are screened one by one, and images with no analytical value, such as those that are severely blurred, have a view that deviates from the runway, have severe obstruction, or have abnormal exposure, are removed. Images with complete view and qualified image quality are retained and uniformly collected to form an effective original image dataset.
[0022] In step S2, for the input original runway image, the region is divided according to the visual features of the image. The gray value range, color distribution, texture gradient and edge contour information of the image are statistically analyzed by pixel to obtain the basic visual feature data corresponding to each pixel. Based on gray value and color features, a preliminary classification is made: if the pixel gray value is consistently in the extremely high range and there is no obvious texture detail in the local brightness change, it is determined to be a strong light reflection area; if the pixel gray value is lower than the average level of normal pavement and the color shows the dark characteristics of oil stains and dirt, it is determined to be a surface oil stain area; if the pixel shows the high contrast color unique to runway markings and the local pixels form regular linear contours, it is determined to be a runway marking area. Further differentiation is based on texture and edge features: For areas not marked as reflective, oily, or marked, the uniformity of local texture is statistically analyzed: areas with uniform texture and no obvious abrupt changes are judged as normal and intact pavement areas; if local pixels show abrupt changes in grayscale, texture breaks, or irregular edges, and do not meet the characteristics of markings, reflective, or oily, they are judged as suspected defective areas with abnormal texture. It distinguishes between runway marking areas, surface oil and stain areas, areas with strong light reflection, normal and intact pavement areas, and suspected defect areas with abnormal texture, thus obtaining a precise distinction between interference areas and defect areas to be detected. It should be noted that the process of distinguishing between track marking areas, surface oil and stain areas, areas with strong glare, normal and intact pavement areas, and areas suspected of being damaged due to abnormal texture involves: performing color and brightness statistics on image pixels. If a pixel displays the high-contrast color characteristic of runway markings, and local pixels form regular linear or blocky outlines, it is determined to be a runway marking area. If the pixel brightness is significantly higher than that of the normal surface, and there is no texture detail in some areas, and the area is overexposed, it is determined to be a strong light reflection area. If the pixel brightness is lower than the average level of a normal surface and the color appears dull or off-color, it is determined to be an area of surface oil stains. For regions not labeled as the three types of interference mentioned above, the uniformity of local texture in the image is statistically analyzed: Areas with uniform texture and no obvious abrupt changes are identified as normal and intact pavement areas. If local pixels exhibit sudden grayscale changes, texture breaks, irregular edges, and do not meet the characteristics of marking lines, reflections, or oil stains, they are identified as suspected disease areas with texture abnormalities. For different regions, corresponding pure image transformation operations are applied: reflection suppression processing is performed on reflective areas to eliminate overexposed pixels in strong light, texture smoothing filtering is performed on interference areas such as markings and debris, contrast correction is performed on uneven brightness areas, the overall image quality is unified, and visual interference caused by lighting, markings, stains and debris is specifically eliminated. While optimizing the interference area, the edge contours, texture details and pixel features of the suspected disease area are preserved. After completing the partitioned image transformation and interference removal, an optimized standard image with uniform image quality, interference information removal and complete and prominent disease features is obtained. It should be noted that when optimizing the interference area, the optimization process for the strong light reflection area is as follows: after identifying the strong light reflection area, the brightness of the overexposed pixels in the area is adjusted to reduce the excessively high brightness value, eliminate the local overexposure state, and make the brightness of the area consistent with the surrounding normal pavement, thereby eliminating the visual interference caused by strong light reflection. Optimization of runway markings and debris interference areas: Texture smoothing is applied to interference areas such as markings and debris. By weakening the edge contours and color contrast of these areas, the visual difference between them and the pavement itself is eliminated, making their texture appearance closer to the surrounding pavement, and reducing the impact of markings and debris on subsequent defect identification. Optimization for uneven brightness areas: Contrast correction is performed on uneven brightness in the whole or part of the image, adjusting the brightness and color distribution of each area, unifying the overall image quality, eliminating visual interference caused by uneven lighting, and making the overall brightness and color performance of the image more uniform and stable.
[0023] In S3, a single optimized image is scaled and decomposed into layers to obtain image layers of different resolutions. The high-resolution layer retains the fine texture and local details of the image, while the low-resolution layer presents the overall shape and large-scale regional morphology, forming an image layer system covering different observation scales. Extract visual features corresponding to each level of the image. Extract basic visual features of the image according to level: For high-resolution levels, extract detailed features of local pixel gray-level gradient, texture direction, and edge breakpoints; for low-resolution levels, extract global features of overall gray-level distribution, contour shape, and connected region morphology to form a set of visual features corresponding to each level. Combine the local detailed features of high-resolution levels with the global morphological features of low-resolution levels.
[0024] Based on pixel grayscale changes and contour transition rules, the edge boundaries are defined, and the boundary between the pavement area and the disease area is locked. Using a single pixel as the smallest unit, combined with the fused features and edge constraints, normal pavement pixels and diseased pixels are distinguished one by one, and the areas where various diseases are located are completely divided. It should be noted that the specific process for determining the boundary between the pavement area and the diseased area is as follows: Traverse all pixels in the image and statistically analyze the grayscale difference between each pixel and its neighboring pixels. If there is a significant and continuous abrupt change in grayscale between adjacent pixels, it is determined that there is a potential boundary in this area. These pixels are marked as boundary candidate points. The continuity and morphology of the boundary candidate points are then verified: if the candidate points can form a continuous, closed, or semi-closed contour line, and the difference in grayscale and texture features between the inside and outside of the contour line is stable, then it is determined to be a valid boundary point. Invalid candidate points that are isolated, scattered, or do not conform to the contour features of the disease are eliminated. All valid boundary points are connected according to their spatial connection relationship to form a complete closed contour. This contour line is the boundary line between the pavement area and the diseased area, thus determining the boundary range between the two. Statistical segmentation results are used to extract two-dimensional data of the disease's outline, pixel coordinates, and area coverage. Contour line extraction: Edge detection is performed on the set of defective pixels: Each defective pixel is checked one by one. If its neighborhood contains both defective pixels and normal pavement pixels, then the pixel is determined to be a defective edge point. All edge points are connected sequentially according to spatial connection relationship to form a closed defective contour line. Pixel coordinate location extraction: For all pixels in the diseased area, read their horizontal and vertical coordinate values in the image coordinate system, count the bounding rectangle of the diseased area, and record the coordinates of the upper left and lower right corners of the rectangle. Region coverage extraction: Count the total number of all diseased pixels within the diseased area, calculate the pixel area of the diseased area, combine it with the actual physical resolution of the image, convert the pixel area into the actual coverage of the diseased area, and record the length and width of the bounding rectangle to fully represent the regional scale information of the disease.
[0025] In step S4, based on the acquired binocular stereo images, combined with the output location information of the disease area coordinates and range, the corresponding disease area is accurately located in the binocular stereo images, the calculation target range is defined, the pixels at the same spatial position in the binocular stereo images are compared, the position offset of the same pixel in the left and right views is counted, the disparity data is calculated, and the disparity distribution map of the whole area and the disease area is obtained. It should be noted that the specific process for comparing pixels at the same spatial location in binocular stereo images is as follows: Combining the output coordinates and range information of the diseased area, the target comparison range corresponding to the diseased area is delineated in the binocular stereo images (left and right views). The calculation is focused on the target area to reduce invalid calculations. Based on the imaging angle and shooting position of the binocular camera, the set of pixels corresponding to the same physical spatial location in the left and right views is determined to form corresponding pixel pairs as comparison objects. For each pair of pixels with the same name, their image coordinate positions are read in the left and right views. The positional differences between the two in the horizontal direction are compared, and the pixel offset is counted. All the pixel offsets obtained from the comparison are organized into disparity data in the target area to form a disparity distribution map of the diseased area, thus completing the comparison of pixels at the same spatial location in the binocular stereo images. Based on disparity data and the geometric relationship of binocular imaging, 2D planar image information is converted into spatial stereo information. Using the calculated disparity data, the positional offsets of each corresponding pixel in the left and right views are matched to form corresponding pixel relationships, constructing a set of pixel correspondences within the image plane. According to the fixed shooting position and imaging angle of the binocular camera, the correspondence logic between pixel offset and spatial depth is determined: a larger pixel offset indicates a closer spatial position to the camera, and a smaller offset indicates a farther distance. For each matched pixel, its 2D coordinates in the image plane are combined with the corresponding rules... Using parallax data, the position of the pixel in three-dimensional space is calculated. The points on the image plane are mapped to the spatial coordinate system. All mapped pixels are traversed and arranged according to their spatial positions to form a continuous three-dimensional point cloud. The point cloud data is constructed into surfaces according to adjacency relationships to restore the true three-dimensional surface morphology of the entire runway surface and the damaged areas. The conversion from two-dimensional planar information to spatial three-dimensional information is completed, and the true three-dimensional surface morphology of the entire runway surface and the damaged areas is restored. The corresponding conversion relationship between image pixel coordinates and actual spatial depth is established, and each pixel is converted into the corresponding spatial depth value. It should be noted that the specific process for establishing the correspondence between image pixel coordinates and actual spatial depth is as follows: Based on the fixed shooting position and viewing angle of the binocular camera, a basic correspondence rule between pixel offset (parallax) and spatial depth is first established: the larger the pixel offset, the closer the corresponding spatial position is to the camera; the smaller the offset, the farther the corresponding distance. For each matched pixel, combined with its two-dimensional coordinates in the image and the corresponding parallax data, a corresponding spatial depth value is assigned to the pixel according to the above correspondence rule, forming a set of mapping relationships from pixel coordinates to depth values. The above mapping process is repeated for all pixels in the image to establish a unified correspondence between pixel coordinates and actual spatial depth covering the entire image, so that every pixel in the image can be converted into a corresponding spatial depth value.
[0026] Based on depth data, for the identified diseased areas, the deformation parameters of pothole depth, pavement settlement, and surface protrusion height are calculated respectively. Pothole depth calculation: In the 3D data of the diseased area, the location with the largest depth value is selected, the corresponding depth data is read, and the depth value is compared with the depth of the reference surface. The difference between the two is the pothole depth. Calculation of pavement settlement: Statistically analyze the average depth data of the entire affected area and compare it with the depth of the reference surface. The difference between the two is the pavement settlement of that area. Surface protrusion height calculation: In the three-dimensional data of the diseased area, the position with the smallest depth value is selected, the corresponding depth data is read, and the depth of the reference surface is compared with the depth value. The difference between the two is the surface protrusion height.
[0027] In S5, based on the time-series images of the same runway section at different times, and using the determined outline and coordinates of the defect area as a reference, the position, angle and scale of each time-series image are adjusted so that the pavement and defect area in multiple images are accurately spatially aligned, and the difference operation is performed pixel by pixel on the multi-period images that have been registered. It should be noted that the pixel-by-pixel difference operation is performed on the registered multi-phase images. The specific process is as follows: Based on the multi-phase image sequence that has completed spatial registration, ensure that pixels representing the same physical location in two adjacent phase images correspond in the image coordinate system. For adjacent phase images, traverse in pixel position order, read the grayscale or color value of pixels at the same location in turn, compare the pixel values at the same location, and determine whether they have changed. If the values in the two periods are the same, they are determined to be pixels with no difference. If the values in the two periods are different, they are identified as differing pixels, and the location is marked. All pixels marked as differing are merged according to their spatial adjacency to form a continuous set of differing regions, thus completing the pixel-by-pixel difference calculation. By comparing pixel differences between adjacent time periods, and based on the output contour and coordinates of the diseased area, the position, angle, and scale of the same segment images collected at different time periods are adjusted so that pixels at the same physical location in multiple images correspond to each other, forming an aligned image sequence. After alignment, adjacent images are compared by pixel position, and the grayscale and color values of pixels at the same position are read to determine whether the value of the pixel has changed in the two images, thus obtaining the pixel difference result. Pixels with changed values are marked as difference pixels, and they are merged according to spatial adjacency to form difference regions. The set of regions where the images have changed in adjacent time periods is initially identified. Combining the position, shape, and distribution characteristics of the difference regions, pixel changes caused by changes in lighting, temporary debris, and short-term interference from mobile devices are distinguished. Invalid differences of this type are removed, and pixel regions where the pavement itself has changed are retained. It should be noted that the initial identification process for regions where images changed within adjacent time periods involves the following steps: For two adjacent images that have been registered, each pixel is compared: The grayscale and color values of pixels at the same location are read. If the values change between the two periods, the pixel is marked as a differing pixel; if the values do not change, it is marked as a non-dissimilar pixel. All pixels marked as differing are iterated over, and their neighboring pixels are checked to see if they are also differing pixels. Adjacent differing pixels are merged to form one or more continuous pixel sets, each corresponding to an independent region of change. The location, range, and outline of the merged continuous pixel sets are recorded, ultimately forming the set of regions where images changed within adjacent time periods, thus completing the initial identification.
[0028] By combining the two-dimensional range and three-dimensional deformation index of the disease corresponding to multiple images, the changes in the area, boundary range, depth and settlement of the disease area are statistically analyzed period by period. Based on multiple sets of statistical data, the changing patterns of disease morphological expansion and deformation intensification are sorted out to determine the disease development speed and evolution trend.
[0029] In S6, the output two-dimensional location, outline, and range data of the disease are uniformly summarized, and a complete disease dataset covering morphology, scale, three-dimensional damage, and dynamic changes is formed based on the calculated three-dimensional deformation indicators of disease depth, settlement, and protrusion, as well as the analyzed disease time-series changes and evolution rate data. Based on the airport runway pavement operation and maintenance health evaluation standard, the integrated parameters of various defects are compared with the standard thresholds one by one. Taking into account multiple factors such as defect size, degree of deformation, and development speed, a comprehensive assessment of pavement damage is completed. Based on the comprehensive assessment results, the pavement health status of each section of the runway is classified into levels. At the same time, based on the defect evolution trend, the potential spread of defects and structural damage risks are determined, and the corresponding risk levels are marked. It should be noted that the specific process for classifying the pavement health status of each section of the runway is as follows: First, the two-dimensional dimensions, three-dimensional deformation, and evolution rate of the defects in each section are compared one by one with the standard thresholds in the airport runway pavement operation and maintenance health evaluation specifications to determine the compliance status of each parameter. Then, the severity of the pavement damage in that section is comprehensively assessed by taking into account factors such as the size of the defects, the degree of deformation, and the speed of development, avoiding one-sided judgment based on a single indicator. Based on the comprehensive assessment results, the pavement health status is divided into different levels (such as intact, minor damage, moderate damage, and severe damage). In addition, the potential risk level is determined simultaneously according to the defect evolution trend, thus completing the classification of the pavement health status of each section.
[0030] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0031] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0032] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0033] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0034] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0035] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0036] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A computer vision-based intelligent assessment method for airport runway pavement health status, characterized in that, Includes the following steps: S1: The inspection equipment collects visible light and binocular stereo images of the runway, removes invalid images, and obtains valid raw image data; S2: Identify interference areas and suspected disease areas in the original image by partitioning them, perform image transformation processing on each region to remove interference and retain disease features, and obtain an optimized image; S3: Construct a multi-scale feature pyramid for the optimized image, fuse deep and shallow image features and combine edge constraints to output the two-dimensional contour, location and range information of the disease; S4: Based on the acquired binocular stereo images, combined with the output location information of suspected disease areas, calculate the parallax and reconstruct the three-dimensional shape of the pavement, map pixel and depth parameters, and measure the three-dimensional deformation index of the disease. S5: Based on the retained historical time-series images of the same area, the historical time-series images are registered with the suspected disease area as the benchmark. Environmental interference is removed by pixel difference, and the disease changes are statistically analyzed by combining data from multiple periods to analyze the evolution trend. S6: Integrate two-dimensional segmentation results, three-dimensional deformation indicators, and time-series evolution data, and combine them with airport pavement health evaluation standards to make a comprehensive judgment, classify health and risk levels, and generate a pavement health assessment report.
2. The intelligent assessment method for airport runway pavement health status based on computer vision according to claim 1, characterized in that, In step S1, the inspection equipment equipped with the image acquisition module travels along the entire area of the airport runway, simultaneously capturing visible light sequence images and binocular stereo images. The batch of images acquired are screened one by one, and images with no analytical value, such as those that are severely blurred, have a view that deviates from the runway, have severe obstruction, or have abnormal exposure, are removed. Images with complete view and qualified image quality are retained and uniformly collected to form an effective original image dataset.
3. The intelligent assessment method for airport runway pavement health status based on computer vision according to claim 1, characterized in that, In step S2, for the input original runway image, the region is divided according to the visual features of the image. The gray value range, color distribution, texture gradient and edge contour information of the image are statistically analyzed by pixel to obtain the basic visual feature data corresponding to each pixel. Based on gray value and color features, a preliminary classification is made: if the pixel gray value is consistently in the extremely high range and there is no obvious texture detail in the local brightness change, it is determined to be a strong light reflection area; if the pixel gray value is lower than the average level of normal pavement and the color shows the dark characteristics of oil stains and dirt, it is determined to be a surface oil stain area; if the pixel shows the high contrast color unique to runway markings and the local pixels form regular linear contours, it is determined to be a runway marking area. Further differentiation is based on texture and edge features: For areas not marked as reflective, oily, or marked, the uniformity of local texture is statistically analyzed: areas with uniform texture and no obvious abrupt changes are judged as normal and intact pavement areas; if local pixels show abrupt changes in grayscale, texture breaks, or irregular edges, and do not meet the characteristics of markings, reflective, or oily, they are judged as suspected defective areas with abnormal texture. It distinguishes between runway marking areas, surface oil and stain areas, areas with strong light reflection, normal and intact pavement areas, and suspected defect areas with abnormal texture, thus obtaining a precise distinction between interference areas and defect areas to be detected. For different regions, corresponding pure image transformation operations are applied: reflection suppression processing is performed on reflective areas to eliminate overexposed pixels in strong light, texture smoothing filtering is performed on interference areas such as markings and debris, contrast correction is performed on uneven brightness areas, the overall image quality is unified, and visual interference caused by lighting, markings, stains and debris is specifically eliminated. While optimizing the interference area, the edge contours, texture details and pixel features of the suspected disease area are preserved. After completing the partitioned image transformation and interference removal, an optimized standard image with uniform image quality, interference information removal and complete and prominent disease features is obtained.
4. The intelligent assessment method for airport runway pavement health status based on computer vision according to claim 1, characterized in that, In S3, a single optimized image is scaled and decomposed into layers to obtain image layers of different resolutions. The high-resolution layer retains the fine texture and local details of the image, while the low-resolution layer presents the overall shape and large-scale regional morphology, forming an image layer system covering different observation scales. Extract visual features corresponding to each level of the image. Extract basic visual features of the image according to level: For high-resolution levels, extract detailed features of local pixel gray-level gradient, texture direction, and edge breakpoints; for low-resolution levels, extract global features of overall gray-level distribution, contour shape, and connected region morphology to form a set of visual features corresponding to each level. Combine the local detailed features of high-resolution levels with the global morphological features of low-resolution levels.
5. The intelligent assessment method for airport runway pavement health status based on computer vision according to claim 4, characterized in that, Based on pixel grayscale changes and contour transition rules, the edge boundaries are defined, and the boundary between the pavement area and the disease area is locked. Using a single pixel as the smallest unit, combined with the fused features and edge constraints, normal pavement pixels and diseased pixels are distinguished one by one, and the areas where various diseases are located are completely divided. Statistical segmentation results are used to extract two-dimensional data of the disease's outline, pixel coordinates, and area coverage. Contour line extraction: Edge detection is performed on the set of defective pixels: Each defective pixel is checked one by one. If its neighborhood contains both defective pixels and normal pavement pixels, then the pixel is determined to be a defective edge point. All edge points are connected sequentially according to spatial connection relationship to form a closed defective contour line. Pixel coordinate location extraction: For all pixels in the diseased area, read their horizontal and vertical coordinate values in the image coordinate system, count the bounding rectangle of the diseased area, and record the coordinates of the upper left and lower right corners of the rectangle. Region coverage extraction: Count the total number of all diseased pixels within the diseased area, calculate the pixel area of the diseased area, combine it with the actual physical resolution of the image, convert the pixel area into the actual coverage of the diseased area, and record the length and width of the bounding rectangle to fully represent the regional scale information of the disease.
6. The intelligent assessment method for airport runway pavement health status based on computer vision according to claim 1, characterized in that, In step S4, based on the acquired binocular stereo images, combined with the output location information of the disease area coordinates and range, the corresponding disease area is accurately located in the binocular stereo images, the calculation target range is defined, the pixels at the same spatial position in the binocular stereo images are compared, the position offset of the same pixel in the left and right views is counted, the disparity data is calculated, and the disparity distribution map of the whole area and the disease area is obtained. Based on disparity data and the geometric relationship of binocular imaging, 2D planar image information is converted into spatial stereo information. Using the calculated disparity data, the positional offsets of each corresponding pixel in the left and right views are matched to form corresponding pixel relationships, constructing a set of pixel correspondences within the image plane. According to the fixed shooting position and imaging angle of the binocular camera, the correspondence logic between pixel offset and spatial depth is determined: a larger pixel offset indicates a closer spatial position to the camera, and a smaller offset indicates a farther distance. For each matched pixel, its 2D coordinates in the image plane are combined with the corresponding rules... Using parallax data, the position of the pixel in three-dimensional space is calculated. Points on the image plane are mapped to a spatial coordinate system. All mapped pixels are traversed and arranged according to their spatial positions to form a continuous three-dimensional point cloud. The point cloud data is constructed into surfaces according to adjacency relationships to restore the true three-dimensional surface morphology of the entire runway surface and the damaged areas. This completes the conversion from two-dimensional planar information to spatial three-dimensional information, restores the true three-dimensional surface morphology of the entire runway surface and the damaged areas, and establishes the corresponding conversion relationship between image pixel coordinates and actual spatial depth, converting each pixel into a corresponding spatial depth value.
7. The intelligent assessment method for airport runway pavement health status based on computer vision according to claim 6, characterized in that, Based on depth data, for the identified diseased areas, the deformation parameters of pothole depth, pavement settlement, and surface protrusion height are calculated respectively. Pothole depth calculation: In the 3D data of the diseased area, the location with the largest depth value is selected, the corresponding depth data is read, and the depth value is compared with the depth of the reference surface. The difference between the two is the pothole depth. Calculation of pavement settlement: Statistically analyze the average depth data of the entire affected area and compare it with the depth of the reference surface. The difference between the two is the pavement settlement of that area. Surface protrusion height calculation: In the three-dimensional data of the diseased area, the position with the smallest depth value is selected, the corresponding depth data is read, and the depth of the reference surface is compared with the depth value. The difference between the two is the surface protrusion height.
8. The intelligent assessment method for airport runway pavement health status based on computer vision according to claim 1, characterized in that, In S5, based on the time-series images of the same runway section at different times, and using the determined outline and coordinates of the defect area as a reference, the position, angle and scale of each time-series image are adjusted so that the pavement and defect area in multiple images are accurately spatially aligned, and the difference operation is performed pixel by pixel on the multi-period images that have been registered. By comparing pixel differences between adjacent time periods, and based on the output disease area outline and coordinates, the position, angle, and scale of images of the same section collected at different time periods are adjusted to ensure that pixels at the same physical location in multiple images correspond, forming an aligned image sequence. After alignment, adjacent images are compared by pixel position, and the grayscale and color values of pixels at the same position are read to determine whether the value of the pixel has changed in the two images, thus obtaining the pixel difference result. Pixels with changed values are marked as difference pixels, and they are merged according to spatial adjacency to form difference regions. The set of regions where images have changed in adjacent time periods is initially identified. Combining the position, shape, and distribution characteristics of the difference regions, pixel changes caused by changes in lighting, temporary debris, and short-term interference from mobile devices are distinguished. Invalid differences of this type are removed, and pixel regions where the pavement itself has changed are retained.
9. The intelligent assessment method for airport runway pavement health status based on computer vision according to claim 8, characterized in that, By combining the two-dimensional range and three-dimensional deformation index of the disease corresponding to multiple images, the changes in the area, boundary range, depth and settlement of the disease area are statistically analyzed period by period. Based on multiple sets of statistical data, the changing patterns of disease morphological expansion and deformation intensification are sorted out to determine the disease development speed and evolution trend.
10. The intelligent assessment method for airport runway pavement health status based on computer vision according to claim 1, characterized in that, In S6, the output two-dimensional location, outline, and range data of the disease are uniformly summarized, and a complete disease dataset covering morphology, scale, three-dimensional damage, and dynamic changes is formed based on the calculated three-dimensional deformation indicators of disease depth, settlement, and protrusion, as well as the analyzed disease time-series changes and evolution rate data. Based on the airport runway pavement operation and maintenance health evaluation standard, the integrated parameters of various defects are compared with the standard thresholds one by one. Taking into account multiple factors such as defect size, degree of deformation, and development speed, a comprehensive assessment of pavement damage is completed. Based on the comprehensive assessment results, the pavement health status of each section of the runway is classified into levels. At the same time, based on the defect evolution trend, the potential spread of defects and structural damage risks are determined, and the corresponding risk levels are marked.
