Method and system for identifying diseases of a three-dimensional intersection road based on unmanned aerial vehicle inspection

Abstract

The present application relates to the technical field of image processing, and proposes a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection, which comprises: collecting three-dimensional intersection road images at multiple times; obtaining a plurality of road edge segments, screening a plurality of suspected ring-shaped road segments; obtaining a plurality of continuous road edges, and obtaining a plurality of suspected ring-shaped road edges; analyzing the change of the longitudinal coordinate of the adjacent two road edge segments in the suspected ring-shaped road edge in the three-dimensional intersection road image, obtaining the initial three-dimensional degree of each suspected ring-shaped road edge in the three-dimensional intersection road image; correcting the modified three-dimensional degree of each suspected ring-shaped road edge; constructing a disease identification model based on the three-dimensional intersection road image, and constructing an attention mechanism combined with the modified three-dimensional degree of the suspected ring-shaped road edge, learning the three-dimensional information of the model, and realizing the disease identification of the three-dimensional intersection road. The present application aims to solve the problem of three-dimensional information loss of intersection roads under unmanned aerial vehicle aerial photography.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection. BACKGROUND

[0002] The present application relates to the technical field of image processing, in particular to a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection. SUMMARY

[0003] The present application provides a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection to solve the problem of loss of three-dimensional information of intersection roads under existing unmanned aerial vehicle aerial photography. The technical solution adopted is as follows:

[0004] The present application provides a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection to solve the problem of loss of three-dimensional information of intersection roads under existing unmanned aerial vehicle aerial photography. The technical solution adopted is as follows:

[0005] The present application provides a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection to solve the problem of loss of three-dimensional information of intersection roads under existing unmanned aerial vehicle aerial photography. The technical solution adopted is as follows:

[0006] The present application provides a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection to solve the problem of loss of three-dimensional information of intersection roads under existing unmanned aerial vehicle aerial photography. The technical solution adopted is as follows:

[0007] The present application provides a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection to solve the problem of loss of three-dimensional information of intersection roads under existing unmanned aerial vehicle aerial photography. The technical solution adopted is as follows:

[0008] The present application provides a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection to solve the problem of loss of three-dimensional information of intersection roads under existing unmanned aerial vehicle aerial photography. The technical solution adopted is as follows: The present application provides a three-dimensional intersection road disease identification method and system based on unmanned aerial vehicle inspection to solve the problem of loss of three-dimensional information of intersection roads under existing unmanned aerial vehicle aerial photography. The technical solution adopted is as follows:

[0009] Optionally, the road region in each of the stereoscopic intersection road images is obtained by the following method:

[0010] The road region in each of the stereoscopic intersection road images is subjected to semantic segmentation to extract the road region in the stereoscopic intersection road image.

[0011] Optionally, the plurality of road edge segments are obtained by the following method:

[0012] The road region in each of the stereoscopic intersection road images is subjected to edge detection to obtain a plurality of road edge segments of the road region in the stereoscopic intersection road image.

[0013] For any one road edge, chain codes of each pixel point on the road edge are obtained, and the pixel points with the same continuous chain code on the road edge are combined to form a road edge segment, thereby obtaining a plurality of road edge segments on the road edge.

[0014] Optionally, the plurality of suspected ring-shaped road segments are obtained by the following method:

[0015] A local length is preset, for any one road edge, if the number of pixel points in any one road edge segment of the road edge is greater than or equal to the local length, the road edge segment is taken as a local road segment of the road edge; after all the road edge segments taken as the local road segments are screened, the remaining road edge segments are traversed in a clockwise direction, the remaining road edge segments are combined, when the total number of pixel points in the combined road edge segments is greater than or equal to the local length, the combination is stopped, the combined road edge segments are taken as a local road segment, and the remaining road edge segments are combined again, and the process is repeated until the total number of pixel points in the combined road edge segments is greater than or equal to the local length; if the total number of pixel points in the remaining road edge segments between two local road segments is less than the local length, the next local road segment is obtained by traversing in the clockwise direction, if the last local road segment is obtained by combination, the remaining road edge segments are combined with the last local road segment to obtain a new local road segment; if the last local road segment corresponds to one road edge segment, the remaining road edge segments are combined to obtain a new local road segment.

[0016] For any one local road segment, the absolute value of the difference between the chain codes of any two adjacent road edge segments in the local road segment is obtained, and the average of the absolute values of the differences between the chain codes of all the adjacent road edge segments is taken as the overall bending degree of the local road segment.

[0017] Arranging the overall bending degrees of all local road sections in the road edge in ascending order, and calculating the difference between the overall bending degrees of sequentially adjacent local road sections, taking the maximum value of the overall bending degrees corresponding to the maximum value of all obtained differences as the bending threshold of the road edge, and taking the local road section with the overall bending degree greater than or equal to the bending threshold in the road edge as a suspected circular road section.

[0018] Optionally, the specific method for obtaining the plurality of continuous road edges comprises the following steps.

[0019] For any one road edge, the continuous suspected circular road sections in the road edge form a continuous road edge, and the continuous non-suspected circular road sections in the road edge form a continuous road edge.

[0020] Optionally, the specific method for obtaining the plurality of suspected circular road edges comprises the following steps.

[0021] The continuous road edge with the suspected circular road section is taken as a suspected circular road edge.

[0022] Optionally, the specific method for obtaining the initial stereoscopic degree of each suspected circular road edge in the stereoscopic intersection road image comprises the following steps.

[0023] A coordinate system is constructed with the lower left corner of any one stereoscopic intersection road image as the coordinate origin, the horizontal right as the horizontal axis, and the vertical up as the vertical axis, and the vertical coordinates of all pixel points in the stereoscopic intersection road image are obtained; for any one road edge section, the average value of the vertical coordinates of all pixel points in the road edge section is taken as the vertical coordinate of the road edge section.

[0024] For any one suspected circular road edge, the absolute value of the difference between the vertical coordinates of any two adjacent road edge sections in the suspected circular road edge is obtained, and the product of the average value and the standard deviation of the absolute values of the vertical coordinate differences of all adjacent road edge sections in the suspected circular road edge is taken as the initial stereoscopic degree of the suspected circular road edge.

[0025] Optionally, the specific method for obtaining the corrected stereoscopic degree of each suspected circular road edge comprises the following steps.

[0026] For any one road area, a two-dimensional sample space is constructed according to the pixel point quantity and the overall bending degree of the local road section in the road area, all local road sections in the road area are converted into sample points, density clustering is performed on all sample points, the Euclidean distance between sample points is used as the distance measure, and a plurality of clusters are obtained.

[0027] For any local road section, the mean of the longitudinal coordinates of all road edge sections in the local road section is taken as the longitudinal coordinate of the local road section, the minimum of the absolute values of the differences between the longitudinal coordinates of the local road section and those of the local road sections in the cluster to which the local road section belongs is taken as the road longitudinal coordinate difference of the local road section;

[0028] The ratio of the initial stereoscopic degree of any suspected ring-shaped road edge to the minimum of the road longitudinal coordinate differences of all local road sections in the suspected ring-shaped road edge is taken as the adjusted stereoscopic degree of the suspected ring-shaped road edge.

[0029] The product of the range of the longitudinal coordinates of all road edge sections in the suspected ring-shaped road edge and the adjusted stereoscopic degree of the suspected ring-shaped road edge is taken as the corrected stereoscopic degree of the suspected ring-shaped road edge.

[0030] Optionally, the method for constructing the attention mechanism based on the corrected stereoscopic degree of the suspected ring-shaped road edge comprises the following specific steps:

[0031] For any stereoscopic intersection road image, the corrected stereoscopic degrees of all suspected ring-shaped road edges in the stereoscopic intersection road image are taken as the stereoscopic degrees of the pixel points on the suspected ring-shaped road edges, the stereoscopic degrees of the pixel points other than the pixel points on the suspected ring-shaped road edges in the stereoscopic intersection road image are marked as 0, the corrected stereoscopic degrees of all pixel points in the stereoscopic intersection road image are linearly normalized, the product of the normalized result and the upper limit of the value range of the gray value is taken as the attention value of each pixel point in the stereoscopic intersection road image, and the attention value is obtained by rounding down.

[0032] The attention value of each pixel point is replaced by the gray value, and the obtained image is taken as the attention map of the stereoscopic intersection road image.

[0033] The application further provides a stereoscopic intersection road disease identification system based on unmanned aerial vehicle inspection, which comprises a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the above method when executing the computer program.

[0034] The beneficial effects of this invention are as follows: This invention uses drones to periodically inspect grade-separated highways and collect images. By analyzing the local changes and differences in the direction of road edges in the images, road edge segments are combined and suspected loop road segments are screened, thereby initially analyzing the temporal changes and differences in the direction between straight and curved roads. Furthermore, by extracting continuous road edges and suspected loop road edges from continuous suspected loop road segments, a foundation is provided for subsequent three-dimensional information analysis of suspected loop road edges. By analyzing the changes in the ordinate of road edge segments within suspected loop road edges, the greater the amplitude and fluctuation of the ordinate change, the more likely the suspected loop road edge is to be located within the area of ​​the loop road. In reality, there are significant height variations, resulting in a higher initial three-dimensionality. Simultaneously, considering the distribution of suspected loop road segments with similar directions and lengths within the entire image of the grade-separated highway, i.e., the distribution along the edges of the same road, the smaller the distance, the stronger the three-dimensionality of the corresponding road segment. This is combined with the maximum difference in the ordinates of the suspected loop road edges to correct the three-dimensionality, thereby fully quantifying the three-dimensional information representation of the loop ramps. Finally, by correcting the three-dimensionality, an attention mechanism is constructed for the grade-separated highway image to train the defect identification model, allowing it to learn three-dimensional information, reducing the impact of the grade-separated highway's three-dimensional structure on defect identification, and improving the efficiency and accuracy of defect identification for grade-separated highways. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a schematic diagram of a method for identifying defects in grade-separated highways based on unmanned aerial vehicle (UAV) inspection, provided in one embodiment of the present invention.

[0037] Figure 2 This is a schematic diagram of the drone inspection area for grade-separated highways. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Please see Figure 1The diagram illustrates a flowchart of a method for identifying defects in grade-separated highways based on unmanned aerial vehicle (UAV) inspection, according to an embodiment of the present invention. The method includes the following steps:

[0040] Step S001: Take aerial photos of the three-dimensional intersection highways at multiple times by flying a drone along a fixed route in the inspection area, and obtain the road area in each three-dimensional intersection highway image.

[0041] The purpose of this embodiment is to inspect grade-separated highways (interchange highways) along a fixed route using drones. By extracting multi-view images of grade-separated highways, three-dimensional information is learned for the defect identification model. This avoids interference and misunderstanding of defect identification by grade-separated highways, thereby improving the accuracy and intelligence of defect identification for grade-separated highways, effectively improving the precision and efficiency of highway defect identification, and ultimately achieving rapid response and scientific maintenance.

[0042] Specifically, such as Figure 2 As shown, it illustrates the drone inspection area of ​​an grade-separated highway. Figure 2 The black straight line in the middle represents the fixed flight path of the drone in the inspection area. The drone takes aerial photos of the grade-separated highway along the fixed flight path to collect images of the grade-separated highway from different perspectives. The camera lens is vertically downward, and the shooting time is selected during a time of soft light. In this embodiment, the inspection is carried out from 9:00 to 10:00 in the morning to avoid strong shadows caused by direct sunlight, which would affect the identification of defects. The sampling interval is described as 5 seconds. The drone takes an image every 5 seconds. All the collected images are smoothed, denoised, and grayscale processed. The processed images are used as the grade-separated highway images at each time.

[0043] Furthermore, semantic segmentation is performed on any three-dimensional highway intersection image to extract the road region from the image. The semantic segmentation is constructed and trained using a CNN network. A training dataset is constructed by professional personnel who label the road regions of a large number of highway images. The loss function is the cross-entropy loss function. The specific process is based on existing semantic segmentation techniques, which are well-known and will not be described in detail in this embodiment. The road regions in each three-dimensional highway intersection image are obtained according to the above method.

[0044] Step S002: Perform edge detection on the road area in the three-dimensional intersection highway image to obtain several road edge segments. Based on the changes and differences in the direction of the road edge segments, select several suspected loop road segments. Analyze the differences in the direction of the road edge segments and their distribution in the road area to obtain several continuous road edges. Combine with the suspected loop road segments, obtain several suspected loop road edges.

[0045] It should be noted that edge detection is performed on road areas, but the edge direction of a loop road differs from that of a straight road. The direction of a loop road changes constantly, while the direction of a straight road is usually fixed. The angle between road edge segments remains at 180°, which is used to screen suspected loop road segments. At the same time, the direction of adjacent suspected loop road segments changes little. Combining the distribution relationship, continuous road edges are combined, and the suspected loop road edges are extracted based on the distribution of suspected loop road segments. This is used for further analysis of their three-dimensional information representation, providing a foundation for the three-dimensional information learning of subsequent disease identification models.

[0046] Preferably, in one embodiment of the present invention, edge detection is performed on the road region in the three-dimensional intersection road image to obtain several road edge segments, and several suspected loop road segments are screened based on the changes and differences in the direction of the road edge segments. The specific method includes:

[0047] Edge detection is performed on the road region in any three-dimensional highway intersection image to obtain several road edges in the road region of the three-dimensional highway intersection image; for any road edge, the chain code of each pixel on the road edge is obtained, and the pixels with consecutive identical chain codes on the road edge constitute a road edge segment, thus obtaining several road edge segments on the road edge; wherein the pixel chain code is obtained using an eight-chain code, and the chain code is a known technology, which will not be described in detail in this embodiment; if the chain code of any pixel is different from the chain codes of its preceding and following pixels, then it is considered as a separate road edge segment.

[0048] It should be noted that by extracting the pixel chain code from the road edge and constructing road edge segments based on consecutive identical chain codes, the road edge is treated as several straight line segments for subsequent bending analysis.

[0049] It should be further explained that by setting a local length, straight segments are combined to form local road segments for analysis. Longer road edge segments are directly used as local road segments, while the rest are combined. At the same time, long road edge segments are avoided from being combined with continuously curved road edge segments. After obtaining the local road segments, chain code difference analysis is further performed. The greater the chain code difference, the more continuously curved the local road segment is, and the greater its overall curvature. This is used to screen suspected loop road segments.

[0050] Furthermore, a local length is preset. In this embodiment, the local length is described as 10. For any road edge, if the number of pixels in any road edge segment is greater than or equal to the local length, the road edge segment is regarded as a local road segment of the road edge. After filtering all road edge segments as local road segments, the remaining road edge segments are traversed clockwise and combined. When the total number of pixels in the combined road edge segments is greater than or equal to the local length, the combination is stopped, and the combined road edge segment is regarded as a local road segment. The remaining road edge segments are then combined again, and so on, to obtain several local road segments of the road edge. If the total number of pixels in the remaining road edge segments between two local road segments is less than the local length, the previous local road segment is obtained by traversing clockwise. If the previous local road segment is a combination, the remaining road edge segments are combined with the previous local road segment to obtain a new local road segment. If the previous local road segment corresponds to a road edge segment, the remaining road edge segments are combined as a local road segment.

[0051] Furthermore, for any local road segment, the absolute value of the difference between the chain codes of any two adjacent road edge segments within that local road segment is obtained (where 0 and 7 are adjacent chain codes, and the result of calculating their absolute difference is 1). The average of the absolute values ​​of the differences between the chain codes of all adjacent road edge segments is taken as the overall curvature of the local road segment. Specifically, if the local road segment contains only one road edge segment, the overall curvature of the local road segment is set to 0. The overall curvature of all local road segments within that road edge is arranged in ascending order, and the difference between the overall curvature of sequentially adjacent local road segments is calculated. The maximum value of the two overall curvatures corresponding to the maximum value of all obtained differences is taken as the curvature threshold of that road edge. Local road segments within that road edge whose overall curvature is greater than or equal to the curvature threshold are considered suspected loop road segments.

[0052] It should be further explained that by extracting continuous suspected loop road segments, the edges of continuous roads are obtained and suspected loop road edges are filtered out.

[0053] Preferably, in one embodiment of the present invention, the differences between the orientations of road edge segments and their distribution in the road area are analyzed to obtain several continuous road edges. Combined with suspected loop road segments, several suspected loop road edges are obtained. The specific method includes:

[0054] For any road edge, a continuous road edge is formed by consecutive suspected loop road segments within that road edge, and a continuous road edge is formed by consecutive non-suspected loop road segments within that road edge; the continuous road edge containing suspected loop road segments is considered a suspected loop road edge.

[0055] At this point, several possible edges of the ring road were obtained.

[0056] Step S003: Analyze the changes in the ordinates of adjacent road edge segments in the three-dimensional highway image of suspected ring road edges to obtain the initial three-dimensionality of each suspected ring road edge in the three-dimensional highway image; combine the differences in the ordinates of each road edge segment in the three-dimensional highway image to obtain the corrected three-dimensionality of each suspected ring road edge.

[0057] It should be noted that after obtaining the suspected loop road edge, it is mainly extracted from the road curvature. In order to learn the subsequent stereo information, it is necessary to further analyze its stereo representation. The larger the mean value of the change of the vertical coordinate of the adjacent road edge segments in the suspected loop road edge in the image coordinate system, and the larger the fluctuation range, i.e. the standard deviation, the larger the actual height change of the suspected loop road edge, thus quantifying the initial stereo degree.

[0058] Preferably, in one embodiment of the present invention, analyzing the changes in the ordinates of two adjacent road edge segments in a stereoscopic intersection image to obtain the initial stereoscopic level of each suspected loop road edge in the stereoscopic intersection image includes the following specific method:

[0059] A coordinate system is constructed with the lower left corner of any three-dimensional road intersection image as the origin, the horizontal axis to the right as the x-axis, and the vertical axis upward as the y-axis. The y-coordinates of each pixel in the three-dimensional road intersection image are obtained. For any road edge segment, the mean of the y-coordinates of all pixels in the road edge segment is taken as the y-coordinate of the road edge segment. For any suspected loop road edge, the absolute value of the difference in y-coordinates between any two adjacent road edge segments in the suspected loop road edge is obtained. The product of the mean and standard deviation of the absolute values ​​of the differences in y-coordinates of all adjacent road edge segments in the suspected loop road edge is taken as the initial three-dimensionality of the suspected loop road edge.

[0060] It should be further explained that, based on the initial three-dimensionality, the edge of the suspected loop road needs to be analyzed for distance from the other edge of the same road. The smaller the distance, the stronger the three-dimensionality of the suspected loop road edge. Combined with the maximum difference in the overall ordinate of the road edge segments in the suspected loop road edge, the initial three-dimensionality is corrected to obtain the corrected three-dimensionality.

[0061] Preferably, in one embodiment of the present invention, the corrected stereoscopic degree of each suspected loop road edge is obtained by combining the differences in the ordinates of each road edge segment in the stereoscopic intersection image. The specific method includes:

[0062] For any road region, a two-dimensional sample space is constructed based on the number of pixels in the local road segments and the overall curvature of the road region. All local road segments in the road region are converted into sample points, and density clustering is performed on all sample points. The distance metric is the Euclidean distance between the sample points, resulting in several clusters. For any local road segment, the mean of the ordinates of all road edge segments in the local road segment is used as the ordinate of the local road segment. The minimum absolute value of the difference between the ordinates of the local road segment and the ordinates of all local road segments in its cluster is obtained as the road ordinate difference of the local road segment.

[0063] Furthermore, the ratio of the initial three-dimensionality of any suspected ring road edge to the minimum difference in the longitudinal coordinates of all local road segments within the suspected ring road edge is used as the adjusted three-dimensionality of the suspected ring road edge; the product of the range of longitudinal coordinates of all road edge segments within the suspected ring road edge and the adjusted three-dimensionality of the suspected ring road edge is used as the corrected three-dimensionality of the suspected ring road edge.

[0064] It should be noted that by performing cluster analysis on local road segments in the road area based on length and overall curvature, the probability that local road segments in the same cluster correspond to the same road is greater. The minimum difference in ordinate between local road segments reflects the distance between the two sides of a road, thereby adjusting the initial three-dimensionality. The smaller the difference, the stronger the three-dimensionality of the suspected loop road edge. Furthermore, by combining the maximum difference in ordinate of the overall ordinate in the suspected loop road edge, the larger the difference, the greater the three-dimensional change of the entire road segment, thus obtaining a corrected three-dimensionality.

[0065] Step S004: Construct a defect identification model based on the images of grade-separated highways, and construct an attention mechanism by combining the correction of the three-dimensionality of the suspected ring road edge to learn the three-dimensional information of the model, thereby realizing defect identification of grade-separated highways.

[0066] It should be noted that after obtaining the corrected stereoscopic level of the suspected ring road edge, the attention mechanism for stereoscopic information is constructed based on the stereoscopic level. At the same time, the stereoscopic crossroad images from multiple perspectives during drone aerial photography can enable the disease identification model to learn more stereoscopic information. Combined with manually labeled road disease areas, the disease identification model is constructed and trained in this way, reducing the impact of stereoscopic structure on disease identification.

[0067] Specifically, a CNN network was used to build a disease identification model, and a large number of collected images of three-dimensional intersections were used as a training dataset to train the disease identification model.

[0068] Furthermore, for any three-dimensional highway intersection image, the corrected stereoscopic degree of all suspected loop road edges in the three-dimensional highway intersection image is obtained as the stereoscopic degree of pixels on the suspected loop road edges, and the stereoscopic degree of pixels in the three-dimensional highway intersection image other than those on the suspected loop road edges is marked as 0. The corrected stereoscopic degree of all pixels in the three-dimensional highway intersection image is linearly normalized, and the product of the normalization result and the upper limit of the gray value range is rounded down to obtain the result as the attention value of each pixel in the three-dimensional highway intersection image. The gray value is replaced with the attention value of each pixel, and the resulting image is used as the attention map of the three-dimensional highway intersection image.

[0069] Furthermore, a large number of collected images of grade-separated highways are used as a training dataset and input into the defect identification model. Simultaneously, the attention maps of each grade-separated highway image are input during training. Defects in each grade-separated highway image are manually labeled, with pixels belonging to defect areas marked as 1 and pixels in other areas marked as 0. The cross-entropy loss function is used to train the defect identification model, simultaneously learning stereo information to obtain a trained defect identification model. After new grade-separated highway images are obtained through UAV inspections, they are input into the trained defect identification model, which can then output the corresponding defect areas, thus achieving defect identification of grade-separated highways.

[0070] This concludes the embodiment.

[0071] Another embodiment of the present invention provides a system for identifying defects in grade-separated highways based on unmanned aerial vehicle (UAV) inspection. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the above-described method steps S001 to S004.

[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying defects in grade-separated highways based on unmanned aerial vehicle (UAV) inspection, characterized in that, The method includes the following steps: The drones were used to fly along a fixed route in the inspection area to take aerial photos at multiple times, and the road areas in each of the three-dimensional intersection images were obtained. Edge detection is performed on the road area in the three-dimensional intersection highway image to obtain several road edge segments. Based on the changes and differences in the direction of the road edge segments, several suspected loop road segments are screened. The differences between the directions of the road edge segments and their distribution in the road area are analyzed to obtain several continuous road edges. Combined with the suspected loop road segments, several suspected loop road edges are obtained. The changes in the ordinates of adjacent road edge segments in the stereoscopic intersection image of suspected ring roads are analyzed to obtain the initial stereoscopic degree of each suspected ring road edge in the stereoscopic intersection image; combined with the differences in the ordinates of each road edge segment in the stereoscopic intersection image, the corrected stereoscopic degree of each suspected ring road edge is obtained. A road defect identification model is constructed based on images of three-dimensional intersections, and an attention mechanism is built by combining the correction of the three-dimensionality of suspected ring road edges to learn the three-dimensional information of the model. The method for obtaining the suspected loop road segment is as follows: A local length is preset. For any road edge, if the number of pixels in any road edge segment is greater than or equal to the local length, that road edge segment is considered a local road segment. After filtering all road edge segments as local road segments, the remaining road edge segments are traversed clockwise. These remaining road edge segments are combined. When the total number of pixels in the combined road edge segments is greater than or equal to the local length, the combination is stopped, and the already combined road edge segment is considered a local road segment. The remaining road edge segments are then combined again, and so on, to obtain several local road segments for that road edge. If the total number of pixels in the remaining road edge segments between two local road segments is less than the local length, the previous local road segment is obtained by traversing clockwise. If the previous local road segment is... The remaining road edge segments are combined with the previous local road segment to obtain a new local road segment. If the previous local road segment corresponds to a road edge segment, the remaining road edge segments are combined into a local road segment. For any local road segment, the absolute value of the difference between the chain codes of any two adjacent road edge segments in the local road segment is obtained. The average of the absolute values ​​of the differences between the chain codes of all two adjacent road edge segments is taken as the overall curvature of the local road segment. The overall curvature of all local road segments in the road edge is arranged in ascending order, and the difference between the overall curvature of sequentially adjacent local road segments is calculated. The maximum value of the two overall curvatures corresponding to the maximum value of all obtained differences is taken as the curvature threshold of the road edge. Local road segments in the road edge whose overall curvature is greater than or equal to the curvature threshold are considered as suspected loop road segments.

2. The method for identifying defects in grade-separated highways based on UAV inspection according to claim 1, characterized in that, The specific method for obtaining the road regions in the images of each grade-separated highway intersection is as follows: Perform semantic segmentation on any three-dimensional highway intersection image to extract the road region from the image.

3. The method for identifying defects in grade-separated highways based on UAV inspection according to claim 1, characterized in that, The specific method for obtaining several road edge segments is as follows: Edge detection is performed on the road region in any stereoscopic intersection road image to obtain several road edges in the road region of the stereoscopic intersection road image; For any road edge, obtain the chain code of each pixel on the road edge, and form a road edge segment by forming consecutive pixels with the same chain code on the road edge, thus obtaining several road edge segments on the road edge.

4. The method for identifying defects in grade-separated highways based on UAV inspection according to claim 1, characterized in that, The specific method for obtaining the edges of the several continuous roads is as follows: For any road edge, consecutive suspected loop road segments within that road edge constitute a continuous road edge, and consecutive non-suspected loop road segments within that road edge constitute a continuous road edge.

5. The method for identifying defects in grade-separated highways based on UAV inspection according to claim 1, characterized in that, The specific method for obtaining several suspected ring road edges is as follows: The edges of continuous roads that have suspected loop road segments are designated as suspected loop road edges.

6. The method for identifying defects in grade-separated highways based on UAV inspection according to claim 1, characterized in that, The initial three-dimensionality of the edges of each suspected loop road in the three-dimensional intersection highway image is obtained by the following method: A coordinate system is constructed with the lower left corner of any three-dimensional road intersection image as the origin, the horizontal axis to the right as the x-axis, and the vertical axis upward as the y-axis. The y-coordinates of each pixel in the three-dimensional road intersection image are obtained. For any road edge segment, the average of the y-coordinates of all pixels in the road edge segment is taken as the y-coordinate of the road edge segment. For any suspected loop road edge, obtain the absolute value of the difference between the ordinates of any two adjacent road edge segments in the suspected loop road edge. The product of the mean and standard deviation of the absolute values ​​of the differences between the ordinates of all adjacent road edge segments in the suspected loop road edge is used as the initial three-dimensionality of the suspected loop road edge.

7. The method for identifying defects in grade-separated highways based on UAV inspection according to claim 1, characterized in that, The specific method for obtaining the corrected three-dimensionality of the edges of each suspected ring road is as follows: For any road region, a two-dimensional sample space is constructed based on the number of pixels in local road segments and the overall curvature of the road region. All local road segments in the road region are converted into sample points. Density clustering is performed on all sample points, and the distance metric is the Euclidean distance between sample points, resulting in several clusters. For any local road segment, the mean of the ordinates of all road edge segments in the local road segment is taken as the ordinate of the local road segment. The minimum absolute value of the difference between the ordinates of the local road segment and the ordinates of all local road segments in its cluster is taken as the difference of the ordinates of the local road segment. The ratio of the initial three-dimensionality of any suspected loop road edge to the minimum difference in the road longitudinal coordinates of all local road segments within that suspected loop road edge is used as the adjusted three-dimensionality of that suspected loop road edge. The product of the vertical coordinate range of all road edge segments in the suspected loop road edge and the adjusted three-dimensionality of the suspected loop road edge is used as the corrected three-dimensionality of the suspected loop road edge.

8. The method for identifying defects in grade-separated highways based on UAV inspection according to claim 1, characterized in that, The specific methods for constructing the attention mechanism by combining the corrected three-dimensionality of the suspected loop road edge are as follows: For any three-dimensional highway intersection image, the corrected stereo degree of all suspected loop road edges in the three-dimensional highway intersection image is obtained as the stereo degree of the pixels on the suspected loop road edges. The stereo degree of all pixels in the three-dimensional highway intersection image except for the pixels on the suspected loop road edges is marked as 0. The corrected stereo degree of all pixels in the three-dimensional highway intersection image is linearly normalized. The product of the normalization result and the upper limit of the gray value range is rounded down to obtain the result as the attention value of each pixel in the three-dimensional highway intersection image. Replace the grayscale values ​​with the attention values ​​of each pixel, and use the resulting image as the attention map of the three-dimensional highway intersection image.

9. A system for identifying defects in grade-separated highways based on unmanned aerial vehicle (UAV) inspection, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for identifying defects in grade-separated highways based on unmanned aerial vehicle (UAV) inspection as described in any one of claims 1-8.

Images