Intelligent detection method for airport pavement state based on image recognition

Abstract

The application discloses an airport pavement state intelligent detection method based on image recognition, first, photoelectric equipment is laid out and airport pavement full-range images are acquired, and pretreatment is carried out, wherein airport pavement diseases acquire pavement images under sunny weather, FOD adopts real-time video images, snow and ice conditions adopt pavement pictures under snow and ice weather, and the friction coefficient adopts pavement pictures under sunny weather without diseases and FOD. Then, an airport pavement state recognition model is designed, and specific attributes of the airport pavement state are obtained. Among them, the airport pavement diseases, FOD and snow and ice conditions adopt a YOLO-Airport deep neural network recognition algorithm, and the friction coefficient adopts point cloud reconstruction of the three-dimensional texture of the airport pavement. According to the requirements of the airport specification, the application combines the specific attributes of the recognized airport pavement state, proposes a quantitative rating of the airport pavement state, provides detection technology and evaluation support for the daily maintenance of the airport pavement, and helps the construction of a smart airport.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of airport detection, in particular to an airport pavement state intelligent detection method based on image recognition. BACKGROUND

[0002] The daily inspection content of the airport pavement includes the pavement damage condition, the pavement cleaning condition and the pavement texture state of the airport pavement, in addition to the snow accumulation under complex winter climate conditions of the runway daily inspection content. The pavement damage condition mainly refers to the disease type, position and damage amount, the disease type mainly includes the crack type and the pit or loose type; the pavement cleaning condition is mainly the detection of the foreign object debris (FOD) of the airport; the snow accumulation mainly detects the snow thickness of the airport pavement; and the pavement texture state mainly detects the maximum texture depth (MTD) and the friction coefficient of the airport pavement. However, the current detection mainly has problems of high dependence on manual work, low detection efficiency, low precision of detection equipment and single detection index.

[0003] The traditional airport pavement disease and FOD detection method mainly relies on manual inspection, and such manual detection has low detection efficiency. The manual detection has problems of poor precision, strong subjectivity and low efficiency in disease detection, and it is difficult to meet the actual requirements of high precision, narrow time limit and large range of the airport; the manual FOD detection is not conducive to the prevention and analysis of the FOD work, and greatly occupies the valuable runway time, thereby reducing the number of flights. In the detection of the pavement snow and ice state, the current ice and snow state recognition equipment in China has problems of low accuracy and high environmental sensitivity. In the detection of the pavement texture condition, the current airport mainly uses the fixed-point and continuous anti-skid performance detection equipment. However, the fixed-point detection equipment has small detection range, low data precision and single detection index, and cannot comprehensively evaluate the pavement texture state; the continuous detection equipment can only judge the adhesion condition of the detection vehicle and the pavement, and cannot truly reflect the texture data of the pavement.

[0004] In summary, the four kinds of airport pavement states have a crucial influence on the operation of the airport. However, due to the limitation of the detection method and the detection equipment, the current detection technology is difficult to realize the real-time, efficient and high-precision detection of the airport pavement state. And the detection of the four states is separately carried out, which is time-consuming, low in efficiency and high in consumption. Therefore, it is an important problem to be solved in the industry to develop an automatic disease, FOD, friction coefficient and snow and ice condition detection technology with low cost, intelligence and high detection accuracy, and to establish an intelligent recognition and automatic grading evaluation of the four kinds of airport pavement states. SUMMARY

[0005] The purpose of the present application is to establish an airport pavement state intelligent detection method based on image recognition, and to solve the problem that the prior art cannot detect the airport pavement in real time.

[0006] To achieve the above object, the present application provides the following technical solutions: an airport pavement state intelligent detection method based on image recognition, comprising the following steps:

[0007] S1, based on the airport pavement state type, optical equipment is arranged on the airport pavement, and images of each type of airport pavement state are collected; and the images of each type of airport pavement state are preprocessed;

[0008] S2, taking the images of each type of airport pavement state as input and the corresponding airport pavement properties as output, an airport pavement state recognition model is respectively constructed and trained;

[0009] S3, taking the properties of each type of airport pavement as input and the corresponding evaluation index as output, an airport pavement state evaluation model is respectively constructed and trained;

[0010] S4, taking the images of each type of airport pavement state as input and the corresponding evaluation index as output, and combining the airport pavement state recognition model and the airport pavement state evaluation model, an airport pavement state detection model is constructed;

[0011] S5, inputting real-time airport pavement state image data into the airport pavement state detection model to obtain the detection evaluation result of the airport pavement.

[0012] Further, in the step S1, the airport pavement state image data is preprocessed, specifically:

[0013] The airport pavement disease state image is extracted and identified under sunny weather;

[0014] The airport pavement FOD state image is identified by using real-time all-weather video data;

[0015] The airport pavement rain and snow state image is identified by using rain and snow weather condition screening;

[0016] The airport pavement friction coefficient state image is identified by using the airport pavement picture screening under sunny weather without FOD and disease.

[0017] Further, the step S2 is specifically: the YOLO-Airport algorithm is used to construct the airport pavement FOD recognition model, the airport pavement disease recognition model, and the airport pavement rain and snow recognition model, and the point cloud reconstruction method is used to construct the airport pavement three-dimensional texture recognition model.

[0018] Further, the airport pavement state evaluation model in the step S3 includes the airport pavement FOD evaluation model, the airport pavement disease evaluation model, the airport pavement rain and snow evaluation model, and the airport pavement friction coefficient evaluation model.

[0019] Further, in the aforementioned step S2, when constructing the airport pavement state recognition model, the airport FOD state image data, the disease state image data, the rain and snow state image data, and the image data of no FOD and no disease in sunny weather are all subjected to state image data labeling by using the LabelImg tool package, rectangular frames are used for positioning, and finally the rectangular frame coordinates are saved in an xml file to complete manual labeling.

[0020] Further, in the aforementioned step S2, the airport pavement FOD attributes include the category, size, and position of the FOD; the airport pavement disease attributes include the category, size, and position of the disease; and the airport pavement rain and snow state attributes include the category, depth, and position of the rain and snow.

[0021] Further, the aforementioned construction of the airport pavement three-dimensional texture recognition model specifically includes: importing the image data of no FOD and no disease in sunny weather into Photoscan, performing image feature matching, creating a dense point cloud of the airport pavement friction coefficient, then importing the created dense point cloud into Geomagic studio to perform surface flatness and size calibration; and finally packaging the dense point cloud to generate a texture model, importing the three-dimensional coordinates of the pavement texture model into matlab to obtain a three-dimensional coordinate matrix of the pavement point cloud, performing pavement texture 3D reconstruction, and obtaining an airport pavement three-dimensional texture map.

[0022] Further, the aforementioned airport pavement friction coefficient evaluation model includes the following sub-steps:

[0023] S3.1, the texture average structure depth index MTD index is calculated according to the following formula:

[0024]

[0025] wherein z(x, y) represents the three-dimensional point cloud coordinate height at point (x, y), L is the linear dimension of the rubber block used in theoretical calculation; A represents the grid area of m x n pixels; and Meshgap represents the precision after gridding;

[0026] S3.2, the pavement autocorrelation function power spectral density C(q) is calculated according to the following formula:

[0027]

[0028] In the x-y coordinate system, there are N data points, the texture elevation at the origin is h(0, 0), the area A = L 2 The texture elevation coordinates at any position in the area are h(x, y), and the coordinates at n measurement points can be represented as (n xa ,n ya ); wherein n x = 1, 2,..., N; n y= 1, 2,..., N, the Fourier transform is used to obtain the power spectral density of the pavement autocorrelation function;

[0029] S3.3, the friction coefficient of the airport runway is calculated according to the following formula:

[0030]

[0031]

[0032] wherein P(q) represents the contact ratio coefficient;

[0033] λ represents the wavelength of the road surface texture;

[0034] σ f represents the shear stress;

[0035] σ0 represents the tire pressure stress;

[0036] γ represents the density factor of the microconvex body; q represents the wave vector, wherein q L = 2π / L is the lower limit of integration, q1 = 2π / λ1 is the upper limit of integration, λ1 is the minimum wavelength of the road surface; q0 = 2π / λ0 represents the effective lower limit of integration, λ0 is the maximum wavelength of the road surface; E(q0ζvcosφ) represents the complex modulus of the rubber.

[0037] Further, the aforementioned airport runway state intelligent detection method based on image recognition obtains the size attribute according to the following formula:

[0038]

[0039] wherein, represents the conversion coefficient when the pixel behavior y2;

[0040] represents the conversion coefficient when the pixel column is

[0041] The position attribute is obtained according to the following formula:

[0042]

[0043] wherein the upper left corner of the airport runway state image is set as the coordinate origin, the horizontal direction is the x-axis, and the vertical direction is the y-axis; the original left upper corner coordinates of the bounding box are set as (x1, y1), and the original right lower corner coordinates are set as (x2, y2); the image resolution is 3840*2160.

[0044] Further, the aforementioned airport runway state intelligent detection method based on image recognition, when constructing and training the airport runway disease evaluation model, the pavement damage rate is calculated according to the size of the disease according to the following formula:

[0045] ​

[0046] wherein w i represents the weight coefficient of the disease i;

[0047] A i represents the damage area of the disease i;

[0048] A represents the road surface area.

[0049] Then, the airport pavement disease classification evaluation is obtained according to the damage rate and a preset value.

[0050] Compared with the prior art, the beneficial effects of the present application are as follows: the present application develops a pavement surface disease identification model based on image recognition technology, classifies, locates, segments, measures and counts multi-class pavement surface diseases, and evaluates the pavement health condition; develops an airport runway foreign object intelligent detection model, studies real-time dynamic identification technology of airport runway foreign objects, establishes fine foreign object indexes and danger level evaluation; develops an airport pavement rain, snow and ice state identification model, and performs rain, snow and ice state level evaluation; establishes an airport pavement texture state identification model, evaluates the pavement anti-skid level by the friction coefficient, and realizes intelligentization of the whole process of airport runway detection. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 is a method flowchart of the present application.

[0052] Figure 2 is an airport optical equipment field layout provided by the embodiment of the present application; (a) is an optical equipment situation schematic diagram, (b) is a field layout situation diagram, (c) is a field data acquisition diagram, and (d) is an optical equipment diagram used for layout.

[0053] Figure 3 is an airport runway monitoring full-range coverage schematic diagram provided by the embodiment of the present application.

[0054] Figure 4 is a YOLO-Airport network model structure provided by the embodiment of the present application.

[0055] Fig. 5 is a YOLO-Airport identification result diagram provided by the embodiment of the present application; (a) is an airport pavement disease identification effect diagram; (b) is an airport runway FOD identification effect diagram; (c) is an airport apron FOD identification effect diagram; and (d) is a different thickness rain, snow and ice state identification effect diagram.

[0056] Figure 6 is a friction coefficient identification result provided by the embodiment of the present application; (a) is a picture collection schematic diagram, (b) is a matching repair diagram, (c) is a coordinate system adjustment diagram, and (d) is a model visualization output diagram. DETAILED DESCRIPTION

[0057] For a more complete understanding of the technical content of the present application, specific embodiments are described below with reference to the accompanying drawings.

[0058] Aspects of the present application are described in the detailed description with reference to the accompanying drawings. Embodiments of the present application are not limited to the drawings described. It should be understood that the present application is implemented by any one of the above-described various concepts and embodiments, and the concepts and embodiments disclosed in the present application are not limited to any embodiment. In addition, some aspects disclosed in the present application can be used alone or in any appropriate combination with other aspects disclosed in the present application.

[0059] As Figure 1 The image recognition-based airport pavement state intelligent detection method is shown in the following steps S1 to S5.

[0060] S1, based on the type of airport pavement state, optical equipment is laid on the airport pavement, and images of each type of airport pavement state are collected; and the images of each type of airport pavement state are preprocessed;

[0061] S2, taking the image data of each type of airport pavement state as input and the corresponding airport pavement attribute as output, respectively constructing and training an airport pavement state recognition model;

[0062] S3, taking the attribute of each type of airport pavement as input and the corresponding evaluation index as output, respectively constructing and training an airport pavement state evaluation model;

[0063] S4, taking the image data of each type of airport pavement state as input and the corresponding evaluation index as output, and combining the airport pavement state recognition model and the airport pavement state evaluation model, an airport pavement state detection model is constructed;

[0064] S5, applying the airport pavement state detection model of step S4, inputting the image data of the airport pavement state into the model, and obtaining the detection evaluation result of the airport pavement.

[0065] In step S1, first determine the optical equipment parameters and the laying method. As Figure 2As shown, Fig. (a) is a schematic diagram of the optical equipment, (b) is a field layout diagram, (c) is a field data acquisition diagram, and (d) is a diagram of the optical equipment used for layout. According to the required airport pavement monitoring range and the required image resolution of the four airport states: airport pavement disease, FOD, rain and snow, and friction coefficient, the distribution distance and shooting angle of the optical equipment are determined to complete the setting of the optical equipment for real-time monitoring of the to-be-inspected area. The visible light images and infrared images taken are transmitted to the server in the form of wired transmission. In the images, FOD needs to occupy at least 4x4 pixels to meet the basic identification requirements. To meet the minimum resolution FOD identification accuracy requirements, the pixels of the equipment taken are specified, and the monitoring range of one camera is approximately triangular with an area of If a 5cm FOD in this area is to be identified, the pixel of the camera should at least reach million. Based on this, the optical equipment parameters selected by the patent are shown in Table 1:

[0066] Table 1

[0067] Pixel size (u-pixel) Resolution Frame rate Maximum detection range 2.5×2.5 3840×2160 25 Hz 80m

[0068] As Figure 3 shown, optoelectronic equipment is selected as the collection equipment, and optoelectronic units are placed on both sides of the airport runway for monitoring of the pavement

[0069] The optoelectronic unit is composed of 3 cameras, and each camera has infrared auxiliary function, which can ensure the continuous use of the optoelectronic unit at night. The optoelectronic unit monitors a 180-degree range on one side of the runway, and each camera covers a 60-degree range. Adjacent optoelectronic units are set to have an overlapping monitoring area, which can improve the monitoring accuracy. Each monitoring unit and optical detection equipment is numbered to determine its specific monitoring area on the airport runway, so that the captured images can be based on this information and optical equipment parameters to realize the monitoring of the pavement state. By using frame extraction technology, image data of the airport pavement state is obtained from the airport pavement monitoring video.

[0070] Optoelectronic equipment parameters

[0071]

[0072]

[0073] IGR represents the minimum number of pixels that meet the identification,

[0074] L pixel represents the number of pixels of the optical equipment length,

[0075] L FODmin represents the minimum length of the FOD to be detected,

[0076] L0 represents a monitoring area length,

[0077] W pixel represents the number of pixels of the optical equipment width,

[0078] W FODmin represents the minimum width of the FOD to be detected,

[0079] W0 represents a monitoring area width.

[0080] Camera shooting angle range

[0081]

[0082] h limit represents the height limit of the airport runway ancillary facilities,

[0083] W r represents the width of the airport runway,

[0084] W s represents the width of the airport runway shoulder,

[0085] i h represents the cross slope of the airport runway,

[0086] θ represents the camera shooting angle range.

[0087] The airport pavement state image data is preprocessed, specifically: the airport pavement disease state image is extracted to screen and identify the picture under sunny weather;

[0088] The airport pavement FOD state image is identified by using real-time all-weather video data;

[0089] The airport pavement rain and snow state image is screened and identified by rain and snow weather conditions;

[0090] The airport pavement friction coefficient state image is screened and identified by the airport pavement picture without FOD and disease under sunny weather. In step S2, each type of airport pavement state image data is input, and the corresponding airport pavement attribute is output, and an airport pavement state recognition model is constructed and trained; respectively including using YOLO-Airport algorithm, taking airport FOD state image data as input, and taking airport pavement FOD attribute as output, constructing and training an airport pavement FOD recognition model;

[0091] The airport pavement disease recognition model is constructed and trained by using YOLO-Airport algorithm, taking the disease state image data as input, and taking the airport pavement disease attribute as output;

[0092] The YOLO-Airport algorithm is adopted, the image data of snow and ice state is taken as input, and the runway pavement snow and ice state attribute is taken as output, and a runway pavement snow and ice recognition model is constructed and trained.

[0093] The point cloud reconstruction method is adopted, the image data of sunny weather without FOD and disease state is taken as input, and the runway pavement three-dimensional texture map is taken as output, and a runway pavement three-dimensional texture recognition model is constructed and trained.

[0094] The runway pavement FOD recognition model, the runway pavement disease recognition model, and the runway pavement snow and ice recognition model adopt the YOLO-Airport deep neural network recognition algorithm. The YOLO algorithm is a one-stage target detection algorithm, which realizes target frame prediction and category prediction in a single neural network model. While ensuring high recognition accuracy, it can meet the requirements of real-time detection of the airport. YOLO-Airport optimizes and adjusts the YOLOv5 model, and proposes to add a feature expansion module combining BiFPN and FTT. A three-layer bidirectional weighted feature pyramid is added to the original three different scale features of YOLOv5 for further feature extraction and fusion. Then, the FTT module is added to the medium and small scale features for fusion. After fusion, the three new large, medium and small scale features are formed. The specific network structure diagram is shown in Figure 4 The purpose of this is to strengthen the extraction of small-scale bottom features, and to combine the position-sensitive information provided by low-level detailed features with the context information provided by high-level semantic features through a bottom-up feature weight distribution fusion method, without changing the spatial resolution of the prediction layer, to strengthen the semantics of the target features, thereby improving the recognition accuracy of small targets by the detection algorithm. When constructing and training the runway pavement FOD detection model, the runway pavement disease detection model, the runway pavement snow and ice detection model, and the runway pavement three-dimensional texture detection model, the runway FOD state image data, the disease state image data, the snow and ice state image data, and the sunny weather without FOD and disease state image data are all labeled using the LabelImg toolkit. Rectangular frames are used for positioning, and the rectangular frame coordinates are saved in xml files for manual labeling. The types of diseases include transverse joints, longitudinal joints, cracks, pits, and repairs. The area of the disease is used to calculate the pavement damage rate, and the calculation formula is as follows. The length of the transverse joint and the longitudinal joint is multiplied by the influence width of 0.2m for calculation; the crack and the pit are calculated by the circumscribed rectangle; the block repair should be calculated by area, and the strip repair should be calculated by length(m) multiplied by the influence width of 0.2m. After calculating the pavement damage rate, the pavement condition is evaluated according to the pavement damage rate. The pavement damage rate is greater than 15% for grade I, greater than 10% and less than 15% for grade II, greater than 5% and less than 10% for grade III, and less than 5% for grade IV.

[0095]

[0096] wherein w i represents the weight coefficient of disease i;

[0097] A i represents the damaged area of disease i;

[0098] A represents the road surface area.

[0099] The results output by the model include the categories of diseases, FOD, and snow and ice, confidence probability scores, and coordinate information of the prediction box. As shown in FIG. 5, (a) is an effect diagram of airport runway disease identification; (b) is an effect diagram of airport runway FOD identification; (c) is an effect diagram of airport apron FOD identification; and (d) is an effect diagram of identification of snow and ice in different thicknesses.

[0100] The size and position information of diseases, FOD, and snow and ice are calculated according to the coordinates of the prediction box. In the process of determining the geometric characteristics of the foreign matter, the upper left corner of the original image is set as the coordinate origin, the horizontal direction is the x-axis, and the vertical direction is the y-axis. The original upper left corner coordinates of the bounding box are set as (x1, y1), and the original right lower corner coordinates are set as (x2, y2). The image resolution is 3840*2160. According to the imaging principle of the camera, the horizontal width corresponding to each row y on the picture and the distance from the camera are different. The larger the y is, the smaller the horizontal width is and the higher the accuracy is. Therefore, y2 is used to calculate the size of the foreign matter; the vertical length corresponding to each column x on the picture is different. The closer to the midpoint, the smaller the vertical width is and the higher the accuracy is. Therefore, x2 is used to calculate the size of the foreign matter. The midpoint of the y2 row is selected as the position calculation point, and the size and position calculation formula of the target to be detected in the image is as follows.

[0101]

[0102]

[0103] wherein, represents the conversion coefficient when the pixel behaves y2;

[0104] represents the conversion coefficient when the pixel column is

[0105] ​The three-dimensional texture recognition model of airport pavement adopts point cloud to reconstruct the three-dimensional texture of airport pavement. Three-dimensional regional texture analysis can provide data that can better represent the parameters of a specific region of the pavement. In order to realize the three-dimensional reconstruction of the regional texture, the captured image is imported into Photoscan, image feature matching is performed, a dense point cloud of the friction coefficient of the airport pavement is created, and then the created dense point cloud is imported into Geomagic studio for surface flatness and size calibration. Then the point cloud is encapsulated to generate a texture model, the three-dimensional coordinates of the pavement texture model are imported into matlab to obtain a three-dimensional coordinate matrix of the pavement point cloud, the pavement texture 3D reconstruction is performed, and the three-dimensional texture map of the airport pavement is obtained, such as Figure 6 wherein (a) is a picture collection schematic diagram, (b) is a matching repair map, (c) is a coordinate system adjustment map, and (d) is a model visualization output map.

[0106] In step S3, the airport pavement friction coefficient evaluation model; comprising the following sub-steps:

[0107] S3.1, calculate the texture average structure depth index MTD index according to the following formula:

[0108]

[0109] wherein z(x,y) represents the three-dimensional point cloud coordinate height at point (x,y), L is the linear dimension of the rubber block used in the theoretical calculation,

[0110] A represents the grid area of m*n pixels,

[0111] Meshgap represents the precision after meshing

[0112] The pavement autocorrelation function power spectral density (q) is calculated according to the following formula:

[0113]

[0114] According to Parseval's theorem, the square of the modulus of the Fourier transform of a signal is defined as the energy spectrum, and the time average of the energy spectrum density is the power spectrum. Assuming that there are N data points in the x-y coordinate system, the texture elevation at the origin is h(0,0), the area A=L 2 , and the texture elevation coordinates at any position in the interior are h(x,y), the coordinates at n measurement points can be represented as (n xa ,n ya ) (wherein n x =1,2,...,N; n y =1,2,...,N), and the Fourier transform is used to obtain the pavement autocorrelation function power spectral density.

[0115] The friction coefficient μ of the airport pavement is calculated according to the following formula:

[0116]

[0117]

[0118] σ f denotes the shear stress;

[0119] σ0denotes the tread compressive stress;

[0120] C(q) denotes the power spectral density,

[0121] P(q) denotes the contact fraction coefficient,

[0122] λ denotes the road texture wavelength,

[0123] γ denotes the density factor of micro-convexes;

[0124] q denotes the wave vector, which can be obtained by wavelength conversion. Wherein q L = 2π / L is the lower limit of integration, L is the linear dimension of the rubber block used in theoretical calculation. q1=2π / λ1is the upper limit of integration, λ1is the minimum wavelength of the road surface. q0=2π / λ0denotes the effective lower limit of integration, λ0is the maximum wavelength of the road surface,

[0125] E(q0ζvcosφ) denotes the complex modulus of rubber.

[0126] The FOD properties of the airport pavement include the category, size and position of the FOD; plastic, metal, golf ball, glass, tire fragment, bird, asphalt block, cement concrete block, personnel, vehicle, animal, ice block; classified according to the category, size and position of the foreign matter, and the classification is shown in Table 2.

[0127] Table 2

[0128]

[0129] The airport pavement disease properties include the category, size and position of the disease; transverse joint, longitudinal joint, alligator joint, block crack, pit and groove, repair; the snow and ice state properties of the airport pavement include the category, depth and position of the snow and ice. The preset depth of wetness, water accumulation, snow accumulation and icing, and the snow and ice state is mainly divided into 0mm, 3mm, 5mm, 7mm, 10mm and above wetness, water accumulation, snow accumulation and icing categories. Among them, 10mm and above is 0 level, 7-10mm is 1 level, 5-7mm is 2 level, 3-5mm is 3 level, 0-3mm is 4 level, and 0mm is 5 level.

[0130] The evaluation of the friction coefficient includes the pavement structure depth and the dynamic friction coefficient. Among them, when the structure depth is greater than 1.0 mm, it is good, when it is greater than 0.6 mm and less than 1.0 mm, it is medium, and when it is less than 0.6 mm, it is poor; when the dynamic friction coefficient is greater than 0.53, it is good, when it is greater than 0.43 and less than 0.53, it is medium, and when it is less than 0.43, it is poor. Then, according to the slope and offset suppression standard specified in the ISO standard (ISO 2002), the average value is moved to zero. Then, the point cloud is packaged to generate a texture model, and the generated point cloud texture model (including the x, y, z coordinates of all vertices of the grid) is imported into Matlab 2021a to obtain a three-dimensional coordinate matrix of the pavement point cloud. Taking the average value of the z value of the pavement point cloud as the reference surface, the structure depth of the pavement texture is calculated according to the distance of the z coordinate of the point cloud from the reference surface. The generated three-dimensional texture model is imported into Matlab, and the calculation of MTD is realized by programming.

[0131] A finite element model is established to model and analyze the contact between the aircraft tire and the airport pavement, to obtain the distribution information of the tire-pavement contact stress under different loads and the effective tire-pavement contact texture at different speeds. On this basis, the friction coefficient value is calculated and verified in combination with the field test data of the dynamic friction coefficient instrument.

[0132] In step S5, the airport pavement state image data of each type is taken as input, and the corresponding evaluation index is taken as output. An airport pavement state detection model is constructed in combination with an airport pavement state recognition model and an airport pavement state evaluation model.

[0133] Then, the airport pavement state detection model is applied to obtain the detection and evaluation results of the airport pavement. In the airport pavement FOD grading evaluation model, the airport pavement disease evaluation model and the airport pavement snow and ice detection model, the airport pavement state is graded, and the grading results are shown in Table 3:

[0134] Table 3

[0135] Number Class Size Position (m) Danger level FOD-1 Plastic 21.02 cm (-2.28，10.17) Class II FOD-2 Cement block 6.57 cm (-2.22，7.78) Class II FOD-3 Paper product 7.75 cm (-1.26，10.68) Class III FOD-4 Plastic 13.04 cm (-1.22，9.22) Class IV FOD-5 Metal 11.88 cm (-1.03，7.97) Class I FOD-6 Cement block 12.39 cm (1.21，9.39) Class I FOD-7 Asphalt block 25.42 cm (1.50,7.82) Class II FOD-8 Plastic 18.59 cm (2.02，9.39) Class II FOD-9 Golf ball 3.34 cm (2.28，11.38) Class III Distress-1 Vertical crack 3840 cm 2 ]]> (-0.20，0.98) Class III Distress-2 Horizontal crack 2880 cm 2 ]] (0.38，0.85) Class III Distress-3 Diagonal crack 768 cm 2 ]] (-0.53，1.56) Class IV Distress-4 Vertical crack 9360 cm 2 ]]> (0.08，1.32) Class I Distress 5 Vertical crack 5488 cm 2 ]] (0.14，1.00) Class II Distress-6 Diagonal crack 1280 cm 2 ]] (-0.80，1.05) Class IV Distress-7 Crack 19040 cm 2 ]] (0.10，1.08) Class I Distress-8 Horizontal crack 480 cm 2 ]] (0.90，1.90) Class IV Distress-9 Vertical crack 768 cm 2 ]] (0.19，0.15) Class IV RainSnowIce-1 Snow 3 mm (0.00，0.94) Class 3 RainSnowIce-2 Snow 5 mm (0.00，1.74) Class 2 RainSnowIce-3 Snow 3 mm (0.00，0.46) Class 3 RainSnowIce-4 Snow 3 mm (-0.45，1.38) Class 3 RainSnowIce-5 Snow 5 mm (0.55，1.42) Class 2

[0136] Although the present application has been described as above with reference to the preferred embodiments, it is not intended to limit the present application. Those skilled in the art can make various modifications and improvements without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the claims.

Claims

1. An intelligent detection method for airport pavement condition based on image recognition, characterized in that, Includes the following steps: S1. Based on the airport pavement condition type, optical equipment is deployed on the airport pavement to collect airport pavement condition image data of each type; and the airport pavement condition image data of each type is preprocessed. S2. Using various types of airport pavement condition image data as input and their corresponding airport pavement attributes as output, construct and train airport pavement condition recognition models respectively. Specifically, the YOLO-Airport algorithm is used to construct airport pavement FOD recognition models, airport pavement defect recognition models, and airport pavement rain, snow and ice recognition models respectively. Point cloud reconstruction method is used to construct airport pavement 3D texture recognition models. YOLO-Airport is optimized and adjusted based on the YOLOv5 model. It proposes to add a feature extension module that combines BiFPN and FTT. A three-layer bidirectional weighted feature pyramid is added to the three different scale features of the original YOLOv5 for further feature extraction and fusion. Then, the FTT module is added to the medium-scale and small-scale features for fusion. After fusion, it is fused with the large-scale features to form three new large, medium and small-scale features. S3. Using various types of airport pavement attributes as input and their corresponding evaluation indicators as output, construct and train airport pavement condition evaluation models respectively. Among them, the airport pavement condition evaluation models include airport pavement FOD evaluation models, airport pavement defect evaluation models, airport pavement rain, snow and ice evaluation models, and airport pavement friction coefficient evaluation models. When constructing and training airport pavement FOD detection models, airport pavement defect detection models, airport pavement rain, snow and ice detection models, and airport pavement 3D texture detection models, the labelImg toolkit is used to label the status image data of airport FOD status image data, defect status image data, rain, snow and ice status image data, and clear weather without FOD and defect status image data. Rectangular boxes are used for positioning, and finally the coordinates of the rectangular boxes are saved in XML files to complete manual labeling. S4. Using various types of airport pavement condition image data as input and corresponding evaluation indicators as output, and combining the airport pavement condition recognition model and the airport pavement condition evaluation model, construct an airport pavement condition detection model. S5. Input the real-time airport pavement condition image data into the airport pavement condition detection model to obtain the detection and evaluation results of the airport pavement.

2. The intelligent detection method for airport pavement condition based on image recognition according to claim 1, characterized in that, In step S1, the airport pavement condition image data is preprocessed, specifically as follows: Images of airport pavement defects under clear weather conditions are extracted and filtered for identification. The FOD (Foreign Object Demand) status images of airport pavements are identified using real-time, all-weather video data; Screening and identification of rain, snow and ice weather conditions from images of airport pavement surfaces; The system screens and identifies airport pavement images under clear weather conditions that are free of FOD (Follicular Unit Depletion) and defects.

3. The intelligent airport pavement condition detection method based on image recognition according to claim 2, characterized in that, In step S2, the airport pavement FOD attributes include the type, size, and location of the FOD; the airport pavement damage attributes include the type, size, and location of the damage; and the airport pavement rain, snow, and ice status attributes include the type, depth, and location of the rain, snow, and ice.

4. The intelligent airport pavement condition detection method based on image recognition according to claim 3, characterized in that, The specific steps for constructing a 3D texture recognition model for airport pavement are as follows: Image data of the pavement under clear weather conditions with no FOD and no defects are imported into Photoscan for image feature matching to create a dense point cloud of the airport pavement friction coefficient. This dense point cloud is then imported into Geomagic Studio for surface smoothness and size calibration. Finally, the dense point cloud is encapsulated to generate a texture model. The 3D coordinates of the pavement texture model are imported into MATLAB to obtain the 3D coordinate matrix of the pavement point cloud. 3D reconstruction of the pavement texture is then performed to obtain a 3D texture map of the airport pavement.

5. The intelligent airport pavement condition detection method based on image recognition according to claim 4, characterized in that, Airport pavement friction coefficient evaluation model; including the following sub-steps: S3.1 Calculate the texture mean construction depth (MTD) index using the following formula: Where z(x,y) represents the height of the three-dimensional point cloud coordinates at point (x,y), and L is the linear scale of the rubber block used in the theoretical calculation; A represents the grid area of ​​m×n pixels; Meshgap represents the precision after meshing; S3.2 Calculate the power spectral density C(q) of the pavement autocorrelation function using the following formula: The xy coordinate system contains N data points, with the texture elevation at the origin being h(0,0) and the area being A = L. 2 The texture elevation coordinates at any location within the area are h(x, y), and their coordinates at n measurement points can be expressed as (n xa ,n ya ); where n x =1, 2, ..., N; n y =1, 2, ..., N, and the power spectral density of the pavement autocorrelation function is obtained by Fourier transform; S3.3 Calculate the friction coefficient μ of the race track surface using the following formula: Where P(q) represents the contact ratio coefficient; λ represents the wavelength of the road surface texture; σ f Represents shear stress; σ0 represents the compressive stress on the tire tread; γ represents the density factor of the micro-convexity; q represents the wave vector, where q L =2π / L is the lower limit of integration, q1 =2π / λ1 is the upper limit of integration, λ1 is the minimum wavelength of the road surface; q0 =2π / λ0 represents the effective lower limit of integration, λ0 is the maximum wavelength of the road surface; E(q0ζvcosφ) represents the complex modulus of the rubber.

6. The intelligent airport pavement condition detection method based on image recognition according to claim 5, characterized in that, Obtain the dimension attributes using the following formula: in, The conversion factor represents the pixel behavior y2; Indicates the pixel column as Conversion factor at time; The position attribute is obtained using the following formula: In this image, the top left corner of the airport pavement status image is set as the origin, the horizontal direction is the x-axis, the vertical direction is the y-axis, the original top left corner coordinates of the bounding box are set as (x1, y1), the original bottom right corner coordinates are set as (x2, y2), and the image resolution is 3840*2160.

7. The intelligent airport pavement condition detection method based on image recognition according to claim 6, characterized in that, When constructing and training the airport pavement distress evaluation model, the pavement damage rate is calculated based on the distress size using the following formula: Among them, w i Represents the weighting coefficient of disease i; A i Indicates the damaged area of ​​disease i; A represents the road surface area; Then, based on the damage rate and preset values, the airport pavement defects classification evaluation is obtained.

Images