Automatic synthesis method for airport pavement plate

Abstract

The invention discloses an airport pavement plate automatic synthesis method comprising the following steps: S1, using a deep learning model to identify an airport pavement image, and segmenting a pavement slab seam area; s2, extracting pavement slab seams through expansion, refinement and Hough transform modes, fitting corresponding line segments, and obtaining intersection point coordinates according to the fitted line segments; s3, after coordinates of all the intersection points are obtained, the intersection points are supplemented and corrected according to the distance relation; and S4, filtering the intersection points, and fitting the airport plate according to the filtered intersection points. The method comprises the following steps: firstly, extracting a mode of extracting intersection points of transverse and longitudinal slab joints according to a slab joint recognition result, extracting plate corner points, then performing completion according to an extracted corner point result, performing intersection point filtering after completion, and fitting an airport plate according to the filtered intersection points, so that the airport pavement plate fitted by adopting intersection point information is more accurate than a method of searching the intersection points through straight line fitting, and the accuracy of the airport pavement plate fitting is improved. The edge of the plate is better attached, and an airport management unit can conveniently and efficiently manage the airport pavement.

Description

Technical Field

[0001] The invention relates to the technical field of pavement synthesis, in particular to an automatic synthesis method for airport pavement panels. Background Art

[0002] At present, airport cement pavement management and maintenance usually uses panels as units for daily pavement maintenance and health management. In addition, when the pavement robot collects high-precision data on the pavement surface, the coverage of the camera image is difficult to cover the entire panel. Therefore, after identifying pavement diseases and features in a single image, it is necessary to synthesize the pavement panel by panel synthesis to achieve airport pavement panel management.

[0003] Existing methods mainly use semantic segmentation methods to identify plate seams in airport pavement images. After plate seam identification is completed, the plate seams are sampled, and then straight lines are fitted based on the sampling points. Then, intersections are found based on all the straight lines. The coordinates of the intersections found are the corner coordinates of the plate, and finally the airport pavement plate blocks are fitted based on the corner coordinates. However, the straight line fitting method has high requirements for the continuity of plate identification. If a large number of discontinuous plate seams are identified, the number of sampling points will be too large when downsampling, making subsequent plate synthesis difficult. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an automatic synthesis method for airport pavement panels.

[0005] The object of the present invention is achieved by the following technical solution: A method for automatically synthesizing airport road surface panels, comprising the following steps:

[0006] S1: Use deep learning model to identify the airport pavement image and segment the pavement panel seam area;

[0007] S2: Extract the slab cracks by dilation, thinning and Hough transform, fit the corresponding line segments, and then obtain the intersection coordinates based on the fitted line segments;

[0008] S3: After obtaining the coordinates of all intersection points, the intersection points are filled and corrected according to the distance relationship;

[0009] S4: Filter the intersection points, and fit the airport plate according to the filtered intersection points.

[0010] Preferably, in step S1, the deep learning model is a Segformer deep learning semantic segmentation model.

[0011] Preferably, step S2 further includes the following steps:

[0012] S21: converting the polygon pixel coordinates in the recognition result into mercator position coordinates according to the recognized image position information;

[0013] S22: Generate a corresponding binary image according to the converted mercator position coordinates, and obtain the corresponding skeleton according to the binary image.

[0014] I s =Skeletonize(I0);

[0015] Among them, I0 is the image generated by the original mercator coordinate point, Skeletonize is to obtain the image skeleton calculation, I s is the generated skeleton image;

[0016] S23: Expand the acquired skeleton.

[0017] I d =Dilate(I s );

[0018] Among them, Dilate is the expansion operation, I d is the image after dilation;

[0019] S24: After obtaining the expanded skeleton image, Hough transform is used to extract line segments in the image. The calculation formula is:

[0020] Lines = HoughLinesP(I d );

[0021] Among them, HoughLinesP is Hough transform, and Lines is the straight line segment in the image extracted by Hough transform;

[0022] S25: Calculate the length and angle of the straight line according to the extracted straight line segment endpoints.

[0023]

[0024] Among them, x1, x2, y1 and y2 are the coordinates of the endpoints of the straight line, θ is the angle between the straight line segment and the coordinate axis, and r is the length corresponding to the coordinate straight line segment;

[0025] S26: extract straight line segments in each plate seam recognition result image using Hough transform, cluster the straight lines according to the angle, and filter the line segments in the image;

[0026] S27: After extracting the straight line segments in each image, when there are two straight line segments, the intersection of the two straight lines is determined according to the line segment slopes. If they intersect, the line slopes and the coordinates of the intersection are calculated;

[0027] S28: Through step S27, the detection results of each plate seam are traversed to obtain the coordinate information of all intersection points, and the detected intersection coordinates and two slope information of the intersection points are saved.

[0028] Preferably, step S27 further includes the following steps:

[0029] S27.1: Let the coordinates of the endpoints of two straight line segments be (x1, y1), (x2, y2), (x3, y3), and (x4, y4).

[0030]

[0031] Among them, k1 and k2 are the slopes of the two straight lines, b1 and b2 are the intercepts of the two straight lines, and x0 and y0 are the coordinates of the intersection point;

[0032] S27.2: Calculate the polygon boundary position,

[0033]

[0034] Among them, polygon.x is the x-coordinate point set of the polygon bounding box, min x is the x coordinate value of the current polygon, polygon.y is the y coordinate point set of the polygon bounding box, min y To obtain the y coordinate value of the current polygon;

[0035] S27.3: Based on the minimum coordinate value obtained, the Mercator coordinates of the intersection point are obtained.

[0036]

[0037] Preferably, step S3 further includes the following steps:

[0038] S31: Calculate the Euclidean distance from the point to other points based on the obtained point information.

[0039]

[0040] Among them, i and j are point numbers, where i≠j,

[0041] After obtaining the point distance, save the distance value information of each point to the four nearest points, form a distance array, and save the array information;

[0042] S32: Divide the data into multiple equal intervals, count the frequency of the data appearing in each interval, find the interval with the highest frequency, and the corresponding value of the interval is the plate length, recorded as plate_length;

[0043] S33: judging whether the Euclidean distance calculated in step S31 and the distance length of the point closest to the point meet the set conditions, if not, deleting the point;

[0044] S34: Calculate the four neighboring points of the point through the slope of the point and the length of the plate.

[0045]

[0046] Among them, x0 and y0 are the coordinates of the selected points, x1, x2, x3, x4, y1, y2, y3 and y4 are the coordinates of the added points, k1 and k2 are the two main slopes of the selected points, L palte is the plate length,

[0047] According to the calculation result of step S33, the four points closest to the selected point are found, the Euclidean distances between the four neighborhood points and the four actual neighboring points are calculated, and the missing points are supplemented according to the Euclidean distances between the actual point and the calculated neighboring points.

[0048] Preferably, in step S33, the distance between the two closest points i and j is set to d ij , the length of the obtained plate is recorded as L plate , then d ij With L plate The following conditions must be met:

[0049] 0.75×L plate ≤d ij ≤1.25×L plate .

[0050] Preferably, in step S34, when the calculated distance difference between the neighboring point and the actual point is greater than a set value, the point is added as an additional point and supplemented to the original point sequence. The set value is between 0.7 and 1.3 times the length of the plate.

[0051] Preferably, step S4 further includes the following steps:

[0052] S41: Calculate the center point according to the plate length and point information;

[0053] S42: Delete the duplicate center points, and calculate the Euclidean distance between each center point and other points through the step S31. If the distance between a certain point and the selected point is less than 1 / 4 of the plate length, delete the point;

[0054] S43: Synthesize the plate according to the final point position and center point information.

[0055] Preferably, step S41 further includes the following steps:

[0056] S41.1: Calculate the four adjacent points in the neighborhood according to step S34, and calculate the distance between the center points as follows:

[0057]

[0058] Among them, L halfPlate is the straight-line distance between the center and edge of the plate;

[0059] S41.2: Calculate the plate point of corresponding length based on the distance from the center point.

[0060]

[0061] S41.3: Rotate the four points 45° around (x0, y0) to get the final center point.

[0062]

[0063] Among them, (x c_i ,y c_i ) is the corresponding center point, θ is the rotation angle, θ = 45°.

[0064] Preferably, in step S43, the center point is traversed, the Euclidean distance between the center point and the edge point of the plate is calculated, and the four plate corner points closest to the center point are obtained. If the Euclidean distance between the four corner points and the center point is less than the plate length, the four corner points are plate contour points, and the four corner points are saved to form a closed area, which is the final synthesized plate.

[0065] The present invention has the following advantages: the present invention first extracts the intersection points of the horizontal and vertical plate seams according to the plate seam recognition results, extracts the plate corner points, and then completes them according to the extracted corner point results. After the completion, the intersection points are filtered, and the airport plate is fitted according to the filtered intersection points. Compared with the method of finding the intersection points by straight line fitting, the airport pavement panel block fitted by using the intersection information is more accurate and fits the plate edge better, which is convenient for the airport management unit to efficiently manage the airport pavement. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a structural schematic diagram of the process of the automatic synthesis method of airport road panel blocks. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0068] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0070] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0071] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use, or the positions or positional relationships commonly understood by those skilled in the art, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0072] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0073] In this embodiment, if Figure 1 As shown, a method for automatically synthesizing airport road surface panels includes the following steps:

[0074] S1: Use deep learning model to identify the airport pavement image and segment the pavement panel seam area;

[0075] S2: Extract the slab cracks by dilation, thinning and Hough transform, fit the corresponding line segments, and then obtain the intersection coordinates based on the fitted line segments;

[0076] S3: After obtaining the coordinates of all intersection points, the intersection points are filled and corrected according to the distance relationship;

[0077] S4: Filter the intersections and fit the airport plate according to the filtered intersections. First, extract the intersections of the horizontal and vertical plate seams according to the plate seam identification results, extract the plate corners, and then complete them according to the extracted corner point results. After completion, filter the intersections and fit the airport plate according to the filtered intersections. Compared with the method of finding intersections by linear fitting, the airport pavement plate block fitted by using the intersection information is more accurate and fits the plate edge better, which is convenient for the airport management unit to efficiently manage the airport pavement.

[0078] Furthermore, in step S1, the deep learning model is a Segformer deep learning semantic segmentation model.

[0079] Furthermore, step S2 further includes the following steps:

[0080] S21: converting the polygon pixel coordinates in the recognition result into mercator position coordinates according to the recognized image position information;

[0081] S22: Generate a corresponding binary image according to the converted mercator position coordinates, and obtain the corresponding skeleton according to the binary image.

[0082] I s =Skeletonize(I0);

[0083] Among them, I0 is the image generated by the original mercator coordinate point, Skeletonize is to obtain the image skeleton calculation, I s is the generated skeleton image;

[0084] S23: Expand the acquired skeleton.

[0085] I d =Dilate(I s );

[0086] Among them, Dilate is the expansion operation, I d is the image after dilation;

[0087] S24: After obtaining the expanded skeleton image, Hough transform is used to extract line segments in the image. The calculation formula is:

[0088] Lines = HoughLinesP(I d );

[0089] Among them, HoughLinesP is Hough transform, and Lines is the straight line segment in the image extracted by Hough transform;

[0090] S25: Calculate the length and angle of the straight line according to the extracted straight line segment endpoints.

[0091]

[0092] Among them, x1, x2, y1 and y2 are the coordinates of the endpoints of the straight line, θ is the angle between the straight line segment and the coordinate axis, and r is the length corresponding to the coordinate straight line segment;

[0093] S26: extracting straight line segments in each plate seam recognition result image using Hough transform, clustering the straight lines according to angles, and filtering the line segments in the image; preferably, retaining one or two straight line segments according to actual distribution conditions.

[0094] S27: After extracting the straight line segments in each image, when there are two straight line segments, the intersection of the two straight lines is determined according to the line segment slopes. If they intersect, the line slopes and the coordinates of the intersection are calculated;

[0095] S28: Through the step S27, the detection results of each plate seam are traversed to obtain the coordinate information of all intersection points, and the detected intersection coordinates and two slope information of the intersection points are saved.

[0096] In this embodiment, step S27 further includes the following steps:

[0097] S27.1: Let the coordinates of the endpoints of two straight line segments be (x1, y1), (x2, y2), (x3, y3), and (x4, y4).

[0098]

[0099] Among them, k1 and k2 are the slopes of the two straight lines, b1 and b2 are the intercepts of the two straight lines, and x0 and y0 are the coordinates of the intersection point;

[0100] S27.2: Calculate the polygon boundary position,

[0101]

[0102] Among them, polygon.x is the x-coordinate point set of the polygon bounding box, min x is the x coordinate value of the current polygon, polygon.y is the y coordinate point set of the polygon bounding box, min y To obtain the y coordinate value of the current polygon;

[0103] S27.3: Based on the minimum coordinate value obtained, the Mercator coordinates of the intersection point are obtained.

[0104]

[0105] In this embodiment, step S3 further includes the following steps:

[0106] S31: Calculate the Euclidean distance from the point to other points based on the obtained point information.

[0107]

[0108] Among them, i and j are point numbers, where i≠j,

[0109] After obtaining the point distance, save the distance value information of each point to the four nearest points, form a distance array, and save the array information;

[0110] S32: Divide the data into multiple equal intervals, count the frequency of the data appearing in each interval, find the interval with the highest frequency, and the corresponding value of the interval is the plate length, recorded as plate_length;

[0111] S33: Determine whether the Euclidean distance calculated in step S31 and the distance length of the point closest to the point meet the set conditions. If not, delete the point; preferably, set the distance between the two closest points i and j to be d ij , the length of the obtained plate is recorded as L plate , then d ij With L plate The following conditions must be met:

[0112] 0.75×L plate ≤d ij ≤1.25×L plate .

[0113] S34: Calculate the four neighboring points of the point through the slope of the point and the length of the plate.

[0114]

[0115] Among them, x0 and y0 are the coordinates of the selected points, x1, x2, x3, x4, y1, y2, y3 and y4 are the coordinates of the added points, k1 and k2 are the two main slopes of the selected points, L palte is the plate length,

[0116] According to the calculation result of step S33, the four points closest to the selected point are found, the Euclidean distances between the four neighboring points and the four actual neighboring points are calculated, and the missing points are supplemented according to the Euclidean distances between the actual point and the calculated neighboring point. Further, in step S34, when the difference between the calculated neighboring point and the actual point is greater than the set value, the point is used as an additional point and supplemented to the original point sequence. Preferably, in step S34, the set value is between 0.7 and 1.3 times the length of the plate.

[0117] In this embodiment, step S4 further includes the following steps:

[0118] S41: Calculate the center point according to the plate length and point information; further, step S41 also includes the following steps:

[0119] S41.1: Calculate the four adjacent points in the neighborhood according to step S34, and calculate the distance between the center points as follows:

[0120]

[0121] Among them, L halfPlate is the straight-line distance between the center and edge of the plate;

[0122] S41.2: Calculate the plate point of corresponding length based on the distance from the center point.

[0123]

[0124] S41.3: Rotate the four points 45° around (x0, y0) to get the final center point.

[0125]

[0126] Among them, (x c_i ,y c_i ) is the corresponding center point, θ is the rotation angle, θ = 45°;

[0127] S42: Delete the duplicate center points, and calculate the Euclidean distance between each center point and other points through the step S31. If the distance between a certain point and the selected point is less than 1 / 4 of the plate length, delete the point;

[0128] S43: Synthesize the plate according to the final point position and center point information. Further, in step S43, the center point is traversed, the Euclidean distance between the center point and the plate edge point is calculated, and the four plate corner points closest to the center point are obtained. If the Euclidean distance between the four corner points and the center point is less than the plate length, the four corner points are the plate contour points, and the four corner points are saved to form a closed area, which is the final synthesized plate.

[0129] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically synthesizing airport road surface panels, characterized in that: The following steps are involved: S1: Use deep learning model to identify the airport pavement image and segment the pavement panel seam area; S2: Extract the slab cracks by dilation, thinning and Hough transform, fit the corresponding line segments, and then obtain the intersection coordinates based on the fitted line segments; S3: After obtaining the coordinates of all intersection points, the intersection points are filled and corrected according to the distance relationship; S4: Filter the intersection points, and fit the airport plate according to the filtered intersection points.

2. The method for automatically synthesizing airport pavement panels according to claim 1, characterized in that: In step S1, the deep learning model is a Segformer deep learning semantic segmentation model.

3. The method for automatically synthesizing airport pavement panels according to claim 2, characterized in that: The step S2 further includes the following steps: S21: converting the polygon pixel coordinates in the recognition result into mercator position coordinates according to the recognized image position information; S22: Generate a corresponding binary image according to the converted mercator position coordinates, and obtain the corresponding skeleton according to the binary image. I s =Skeletonize(I0); Among them, I0 is the image generated by the original mercator coordinate point, Skeletonize is to obtain the image skeleton calculation, I s is the generated skeleton image; S23: Expand the acquired skeleton. I d =Dilate(I s ); Among them, Dilate is the expansion operation, I d is the image after dilation; S24: After obtaining the expanded skeleton image, Hough transform is used to extract line segments in the image. The calculation formula is: Lines=HoughLinesP(I d ); Among them, HoughLinesP is Hough transform, and Lines is the straight line segment in the image extracted by Hough transform; S25: Calculate the length and angle of the straight line according to the extracted straight line segment endpoints. Among them, x1, x2, y1 and y2 are the coordinates of the endpoints of the straight line, θ is the angle between the straight line segment and the coordinate axis, and r is the length corresponding to the coordinate straight line segment; S26: extract straight line segments in each plate seam recognition result image using Hough transform, cluster the straight lines according to the angle, and filter the line segments in the image; S27: After extracting the straight line segments in each image, when there are two straight line segments, the intersection of the two straight lines is determined according to the line segment slopes. If they intersect, the line slopes and the coordinates of the intersection are calculated; S28: Through the step S27, the detection results of each plate seam are traversed to obtain the coordinate information of all intersection points, and the detected intersection coordinates and two slope information of the intersection points are saved.

4. The method for automatically synthesizing airport pavement panels according to claim 3, characterized in that: The step S27 further includes the following steps: S27.1: Let the coordinates of the endpoints of two straight line segments be (x1, y1), (x2, y2), (x3, y3), and (x4, y4). Among them, k1 and k2 are the slopes of the two straight lines, b1 and b2 are the intercepts of the two straight lines, and x0 and y0 are the coordinates of the intersection point; S27.2: Calculate the polygon boundary position, Among them, polygon.x is the x-coordinate point set of the polygon bounding box, min x is the x coordinate value of the current polygon, polygon.y is the y coordinate point set of the polygon bounding box, min y To obtain the y coordinate value of the current polygon; S27.3: Based on the minimum coordinate value obtained, the Mercator coordinates of the intersection point are obtained.

5. The method for automatically synthesizing airport pavement panels according to claim 4, characterized in that: The step S3 further includes the following steps: S31: Calculate the Euclidean distance from the point to other points based on the obtained point information. Among them, i and j are point numbers, where i≠j, After obtaining the point distance, save the distance value information of each point to the four nearest points, form a distance array, and save the array information; S32: Divide the data into multiple equal intervals, count the frequency of the data appearing in each interval, find the interval with the highest frequency, and the corresponding value of the interval is the plate length, recorded as plate_length; S33: judging whether the Euclidean distance calculated in step S31 and the distance length of the point closest to the point meet the set conditions, if not, deleting the point; S34: Calculate the four neighboring points of the point through the slope of the point and the length of the plate. Among them, x0 and y0 are the coordinates of the selected points, x1, x2, x3, x4, y1, y2, y3 and y4 are the coordinates of the added points, k1 and k2 are the two main slopes of the selected points, L palte is the plate length, According to the calculation result of step S33, the four points closest to the selected point are found, the Euclidean distances between the four neighborhood points and the four actual neighboring points are calculated, and the missing points are supplemented according to the Euclidean distances between the actual point and the calculated neighboring points.

6. The method for automatically synthesizing airport pavement panels according to claim 5, characterized in that: In step S33, the distance between the two closest points i and j is assumed to be d ij , the length of the obtained plate is recorded as L plate , then d ij With L plate The following conditions must be met: 0.75×L plate ≤d ij ≤1.25×L plate 。 7. The method for automatically synthesizing airport pavement panels according to claim 6, characterized in that: In step S34, when the calculated distance difference between the neighboring point and the actual point is greater than a set value, the point is added as an additional point and added to the original point sequence. The set value is between 0.7 and 1.3 times the length of the plate.

8. The method for automatically synthesizing airport pavement panels according to claim 7, characterized in that: The step S4 further includes the following steps: S41: Calculate the center point according to the plate length and point information; S42: Delete the duplicate center points, and calculate the Euclidean distance between each center point and other points through the step S31. If the distance between a certain point and the selected point is less than 1 / 4 of the plate length, delete the point; S43: Synthesize the plate according to the final point position and center point information.

9. The method for automatically synthesizing airport pavement panels according to claim 8, characterized in that: The step S41 further includes the following steps: S41.1: Calculate the four adjacent points in the neighborhood according to step S34, and calculate the distance between the center points as follows: Among them, L halfPlate is the straight-line distance between the center and edge of the plate; S41.2: Calculate the plate point of corresponding length based on the distance from the center point. S41.3: Rotate the four points 45° around (x0, y0) to get the final center point. Among them, (x c_i ,y c_i ) is the corresponding center point, θ is the rotation angle, θ = 45°.

10. The method for automatically synthesizing airport pavement panels according to claim 9, characterized in that: In step S43, the center point is traversed, and the Euclidean distance between the center point and the edge point of the plate is calculated to obtain the four plate corner points closest to the center point. If the Euclidean distance between the four corner points and the center point is less than the plate length, the four corner points are the plate contour points. The four corner points are saved to form a closed area, which is the final synthetic plate.

Images