Asphalt pavement crack identification and classification method, device, equipment and medium

Abstract

The invention discloses an asphalt pavement crack recognition and classification method, device and equipment and a medium, and relates to the field of road disease detection, and the method comprises the steps: marking cracks in an asphalt pavement image collected by an industrial line scanning camera through employing a trained target model, and obtaining a marked image; wherein a labeling frame in the labeled image is a target rectangular frame meeting a preset small-size condition; combining the target rectangular frames in the marked image to obtain a polygonal outer frame, and filtering the polygonal outer frame to obtain a filtered polygonal outer frame; judging the main direction of the filtered polygonal outer frame based on a principal component analysis algorithm, and identifying a linear crack in the asphalt pavement image according to the main direction; and judging whether a block-shaped crack exists in the asphalt pavement image through a clustering algorithm, and determining a classification result of the asphalt pavement crack according to a corresponding judgment result and the linear crack. According to the invention, high-precision asphalt pavement crack detection and classification can be realized under the condition of small samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of road disease detection, in particular to a method and device for identifying and classifying asphalt pavement cracks, and a medium. BACKGROUND

[0002] A road detection industrial line scan camera obtains high-definition pavement images by vertical ground shooting, is mainly applied to the field of road detection, and is used for detecting cracks, potholes, rutting and other diseases on the road surface to provide data support for road maintenance and management. In addition, it can also be used for quality detection in the process of road construction, such as detecting pavement flatness and paving thickness. Although its high-resolution characteristics can capture crack details, direct processing has problems such as large consumption of computing resources and easy omission of small-size crack targets. Existing crack detection methods mostly rely on global feature labeling and traditional target detection frameworks, which on the one hand have strong requirements for large-scale labeled data, and on the other hand are difficult to effectively process small crack targets with variable shapes in industrial line scan camera images, especially for complex shapes such as block cracks. In the prior art, due to the lack of targeted labeling of crack micro features and small target detection adaptation schemes, the detection model has defects such as strong data dependence and weak generalization ability. Therefore, how to realize high-precision asphalt pavement crack detection and classification under small sample conditions, reduce the number of crack labels, improve the generalization ability of the model, and improve the detection efficiency, while solving the problem of block crack recognition, is an urgent problem to be solved. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a method and device for identifying and classifying asphalt pavement cracks, which can realize high-precision asphalt pavement crack detection and classification under small sample conditions, reduce the number of crack labels, improve the generalization ability of the model, and improve the detection efficiency, while solving the problem of block crack recognition. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a method for identifying and classifying asphalt pavement cracks, comprising:

[0005] The trained target model is used to label the cracks in the asphalt pavement images collected by the industrial line scan camera to obtain labeled images; wherein the labeling frame in the labeled image is a target rectangular frame meeting the preset small size condition;

[0006] The target rectangular frames in the labeled images are merged to obtain a polygon outer frame, and the polygon outer frame is filtered to obtain a filtered polygon outer frame;

[0007] judging a main direction of the polygon outer frame after filtering based on a principal component analysis algorithm, and identifying linear cracks in the asphalt pavement image according to the main direction; the linear cracks include longitudinal cracks and transverse cracks;

[0008] judging whether block cracks exist in the asphalt pavement image by a clustering algorithm, and determining a classification result of the asphalt pavement cracks according to corresponding judgment results and the linear cracks.

[0009] Optionally, before the cracks in the asphalt pavement image collected by the industrial line-scan camera are labeled by using the trained target model, the method further includes:

[0010] segmenting historical asphalt pavement images collected by the industrial line-scan camera based on target pixels to obtain segmented images;

[0011] manually labeling cracks in the segmented images by using target rectangular frames meeting a preset small size condition to obtain manually labeled images; wherein a complete crack is labeled by a plurality of target rectangular frames.

[0012] Optionally, after the manually labeled images are obtained, the method further includes:

[0013] constructing an initial model by an RT-DETR framework and a slice-assisted super-reasoning framework, training the initial model by using the manually labeled images and transfer learning to obtain a trained target model.

[0014] Optionally, the merging the target rectangular frames in the labeled images to obtain polygon outer frames includes:

[0015] performing inflation processing on the target rectangular frames in the labeled images based on a target multiple to obtain processed rectangular frames;

[0016] extracting outer frame lines of the processed rectangular frames, and determining target outer frame lines that are not covered by other processed rectangular frames;

[0017] matching the target outer frame lines according to end point positions, and merging overlapping intervals of the matched outer frame lines to obtain polygon outer frames.

[0018] Optionally, the filtering the polygon outer frames to obtain polygon outer frames after filtering includes:

[0019] performing filtering on the polygon outer frames by eliminating polygon outer frames with an area smaller than a preset area threshold or a number of edges smaller than a target edge number threshold to obtain polygon outer frames after filtering.

[0020] Optionally, the principal component analysis algorithm is used to determine the main direction of the filtered polygonal frame, and linear cracks in the asphalt pavement image are identified according to the main direction, including:

[0021] The covariance matrix of the coordinate points of the filtered polygonal frame is calculated to obtain eigenvalues and eigenvectors;

[0022] The main direction angle is determined by the eigenvector corresponding to the maximum eigenvalue;

[0023] If the main direction angle is within a first angle range, it is determined that the cracks in the asphalt pavement image are transverse cracks;

[0024] If the main direction angle is within a second angle range, it is determined that the cracks in the asphalt pavement image are longitudinal cracks.

[0025] Optionally, the clustering algorithm is used to determine whether there are block cracks in the asphalt pavement image, and the classification result of the asphalt pavement cracks is determined according to the corresponding determination result and the linear cracks, including:

[0026] The polygon center points of each filtered polygonal frame are determined, and the density-based noise application spatial clustering algorithm is used to cluster the polygon center points, and polygons that meet the preset conditions are determined as the same family;

[0027] The aspect ratio of the minimum circumscribed rectangle corresponding to each family is calculated;

[0028] If the aspect ratio is greater than a first value, it is determined that the cracks in the asphalt pavement image are the linear cracks, and the corresponding cracks are labeled as the transverse crack category or the longitudinal crack category;

[0029] If the aspect ratio is less than or equal to the first value, the ratio of the area of the filtered polygonal frame to the area of the corresponding circumscribed rectangle is calculated;

[0030] If the ratio is greater than a second value, and the area of the circumscribed rectangle meets a preset area ratio condition, it is determined that the cracks in the asphalt pavement image are block cracks, and the corresponding cracks are labeled as the block crack category;

[0031] If the ratio is less than or equal to the second value, it is determined that the cracks in the asphalt pavement image are the linear cracks, and the corresponding cracks are labeled as the transverse crack category or the longitudinal crack category.

[0032] In a second aspect, the present application discloses an asphalt pavement crack identification and classification device, including:

[0033] The annotation module is used to annotate cracks in asphalt pavement images acquired by industrial line scanning cameras using a trained target model, and obtain an annotated image; wherein the annotation box in the annotated image is a target rectangle that meets the preset small size condition;

[0034] A filtering module is used to merge the target rectangles in the labeled image to obtain a polygonal outline, and to filter the polygonal outline to obtain a filtered polygonal outline.

[0035] The identification module is used to determine the principal direction of the filtered polygonal outline based on the principal component analysis algorithm, and to identify linear cracks in the asphalt pavement image according to the principal direction; the linear cracks include longitudinal cracks and transverse cracks.

[0036] The classification result determination module is used to determine whether there are block cracks in the asphalt pavement image through a clustering algorithm, and to determine the classification result of the asphalt pavement cracks based on the corresponding judgment result and the linear cracks.

[0037] Thirdly, this application discloses an electronic device, including:

[0038] Memory, used to store computer programs;

[0039] A processor is used to execute computer programs to implement the asphalt pavement crack identification and classification method described above.

[0040] Fourthly, this application discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned asphalt pavement crack identification and classification method.

[0041] The application firstly labels the cracks in the asphalt pavement image collected by the industrial line scanning camera by using the trained target model to obtain a labeled image; wherein the label frame in the labeled image is a target rectangular frame meeting a preset small size condition; each target rectangular frame in the labeled image is merged to obtain a polygon outer frame, and the polygon outer frame is filtered to obtain a filtered polygon outer frame; the main direction of the filtered polygon outer frame is judged based on a principal component analysis algorithm, and the linear cracks in the asphalt pavement image are identified according to the main direction; the linear cracks include longitudinal cracks and transverse cracks; whether there is a block crack in the asphalt pavement image is judged by a clustering algorithm, and the classification result of the asphalt pavement crack is determined according to the corresponding judgment result and the linear cracks. It can be seen that the application realizes accurate positioning by capturing crack detail features through dense small frame labeling, and avoids directly relying on the high requirement of global features on data volume. In the post-processing stage, the classification is carried out from the geometric features and spatial distribution of the cracks by morphological analysis and clustering algorithm, the accurate judgment of the crack direction and type is realized, and the purpose of realizing high-precision crack detection and classification, improving the model migration generalization ability and reducing the data labeling cost under the condition of small sample is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0043] Figure 1 A flow chart of an asphalt pavement crack identification and classification method disclosed by the present application;

[0044] Figure 2 A frame merging schematic diagram disclosed by the present application;

[0045] Figure 3 A block crack classification schematic diagram disclosed by the present application;

[0046] Figure 4 A structure schematic diagram of an asphalt pavement crack identification and classification device disclosed by the present application;

[0047] Figure 5 A structure diagram of an electronic device disclosed by the present application. DETAILED DESCRIPTION

[0048] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0049] The existing asphalt pavement crack detection scheme has problems of strong dependence on large sample data, high small target missing detection rate, and difficult block crack recognition, especially in industrial line scan camera high-resolution image detection, large calculation resource consumption and insufficient classification accuracy. In order to solve the above technical problems, the present application discloses an asphalt pavement crack recognition and classification method, device, equipment and medium, which can realize high-precision asphalt pavement crack detection and classification under small sample conditions, reduce the number of crack labels, improve the model generalization ability and improve the detection efficiency, and at the same time solve the problem of block crack recognition.

[0050] Referring to Figure 1 The embodiment of the present application discloses an asphalt pavement crack recognition and classification method, which comprises:

[0051] Step S11, using the trained target model to label the cracks in the asphalt pavement image collected by the industrial line scan camera to obtain a labeled image; wherein the label frame in the labeled image is a target rectangular frame meeting the preset small size condition.

[0052] In this embodiment, first, based on the target pixel, the historical asphalt pavement image collected by the industrial line scan camera is divided to obtain a divided image; the cracks in the divided image are manually labeled using the target rectangular frame meeting the preset small size condition to obtain a manually labeled image; wherein a complete crack is labeled by a plurality of target rectangular frames. Specifically, the original industrial line scan camera image is divided into small images of 600x600 pixels, and a small frame dense labeling strategy is used for the cracks in the divided small images: along the crack direction, the cracks are labeled with rectangular small frames that overlap with each other, the size of a single frame is limited to within 40x40 pixels, the overlapping ratio of adjacent frames is controlled to be between 30%-60%, each frame only labels a small part of the crack, and the label category is uniform as "crack". This labeling method can cover the details of the crack by dense coverage, so that the subsequent model can quickly learn the micro features such as local texture and direction of the crack, and lay a data foundation for small target detection.

[0053] After obtaining the artificially labeled images, an initial model is built through an RT-DETR (Real-Time Detection Transformer) framework and a slicing-aided hyper inference framework, the initial model is trained using the artificially labeled images and transfer learning to obtain a trained target model. Transfer learning refers to the process of applying knowledge, skills, or experience obtained from one environment to another different but related environment. In machine learning, it aims to use existing knowledge to help the learning of new tasks and reduce the dependence on a large amount of labeled data. In this application, the RT-DETR framework is used for small target detection in the model training stage, and a 600x600 pixel segmented image is used as input. RT-DETR is a real-time target detection framework optimized for small target detection, with efficient feature extraction and positioning capabilities. It introduces a hybrid encoder to improve the processing capability of objects of different scales, and uses a multi-scale feature fusion strategy to combine low-level detail information and high-level semantic information to help the model better understand the objects in the image. It performs particularly well in small object detection and can achieve model training under small sample conditions through transfer learning mechanism. This framework has advantages in small target feature extraction and positioning, and through the transfer learning mechanism, combined with small sample dense labeling data, it can effectively capture the detailed features of cracks. In the training process, the loss function for small target detection is optimized, and the boundary box regression loss for small targets is multiplied by a weight coefficient (1.2-1.5 times), which strengthens the sensitivity of the model to small frame coordinates and the sensitivity of the model to small size crack targets. This allows the model to learn common features of cracks under limited sample conditions and avoid overfitting caused by insufficient data. During model inference, the input image is the standard size of an industrial line scan camera, and the SAHI (Slicing Aided Hyper Inference) framework is used to divide the large size industrial line scan image into a grid with a size of 600x600 pixels and set an overlap area of 20%-30%. SAHI is a small target detection auxiliary framework that can divide a large image into multiple small blocks and support setting the size and overlap ratio of the slices to share information between slices and reduce boundary effects. It can run a target detection model on each slice to obtain detection results and provides multiple methods to combine the detection results on the slices to generate the final detection results. It also supports post-processing techniques such as non-maximum suppression to eliminate duplicate detections, effectively solving the computational bottleneck problem caused by direct input of high-resolution images.After each subgraph is detected by the RT-DETR model, a weighted box fusion (WBF) algorithm is used to integrate the detection results in the overlapping area. The algorithm first clusters and groups all detection boxes according to their spatial positions and categories. For multiple detection boxes of the same crack, the fusion coordinates are calculated by weighted average according to their confidence levels. At the same time, the final confidence is dynamically adjusted according to the number of boxes in the group to balance the recall rate and accuracy, ensuring that the continuous crack across subgraphs can be completely retained. Finally, the global detection result is output without seamless splicing. Compared with the traditional NMS (Non-maximum suppression) method, WBF effectively solves the problem of repeated boxes while maintaining high recall rate. This process solves the problems of large computational resource consumption and small target missing caused by direct input of high-resolution images of industrial line scan cameras through block processing, ensuring the efficiency and accuracy of detection. In this way, the application captures the micro features of cracks through a refined dense small box labeling strategy, uses the RT-DETR framework suitable for small target detection combined with transfer learning to reduce the dependence on large-scale labeled data.

[0054] Step S12, merging each target rectangular frame in the labeled image to obtain a polygon outer frame, filtering the polygon outer frame to obtain a filtered polygon outer frame.

[0055] In this embodiment, after obtaining the labeled image, the target rectangular frame in the labeled image is dilated based on the target multiple to obtain a processed rectangular frame. The outer frame line of each processed rectangular frame is extracted to determine the target outer frame line that is not covered by other processed rectangular frames. Each target outer frame line is matched according to the end point position, and the overlapping interval of the matched outer frame lines is merged to obtain a polygon outer frame. This process is a frame fusion of the identified dense small rectangular frame, as shown in Figure 2 The final polygon outer frame is formed as the smallest processing unit of the crack. Then, by eliminating the polygon outer frame with an area smaller than a preset area threshold or a number of sides smaller than a target side threshold, the polygon outer frame is filtered to obtain a filtered polygon outer frame. In a specific embodiment, the merged polygon is filtered to eliminate abnormal data with an area that is too small or a number of sides less than 3, ensuring the effectiveness of subsequent processing.

[0056] Step S13, determining the main direction of the filtered polygon outer frame based on a principal component analysis algorithm, and identifying linear cracks in the asphalt pavement image according to the main direction; the linear cracks include longitudinal cracks and transverse cracks.

[0057] In this embodiment, after the labeled image is preliminarily processed, the eigenvalues and eigenvectors are obtained by calculating the covariance matrix of the coordinate points of the filtered polygon frame, the main direction angle is determined by the eigenvector corresponding to the maximum eigenvalue, if the main direction angle is within the first angle range, it is determined that the crack in the asphalt pavement image is a transverse crack, if the main direction angle is within the second angle range, it is determined that the crack in the asphalt pavement image is a longitudinal crack. Specifically, the PCA (Principal Components Analysis) main direction algorithm is used to determine the main direction of the polygon to distinguish the transverse and longitudinal cracks. The eigenvalues and eigenvectors are obtained by calculating the covariance matrix of the coordinate points of the polygon, and the main direction angle is determined by the eigenvector corresponding to the maximum eigenvalue. In a specific embodiment, when the angle is within 0° to 30° or 150° to 180°, it is determined to be a transverse crack, and when the angle is within 31° to 149°, it is a longitudinal crack. This step starts from the morphological characteristics of the crack, and provides basic direction information for subsequent classification.

[0058] In step S14, it is determined whether there is a block crack in the asphalt pavement image by using a clustering algorithm, and the classification result of the asphalt pavement crack is determined according to the corresponding determination result and the linear crack.

[0059] In the embodiment, the linear cracks in the asphalt pavement image are only determined in the previous step, and the existence of block cracks needs to be further determined. In this process, the polygon center points of each filtered polygon outer frame are determined, the polygon center points are clustered by a spatial clustering algorithm based on density, polygons that meet the preset conditions in space are determined as the same family, the aspect ratio of the minimum circumscribed rectangle corresponding to each family is calculated, if the aspect ratio is greater than a first value, it is determined that the crack in the asphalt pavement image is the linear crack, and the corresponding crack is labeled as a horizontal crack category or a vertical crack category, if the aspect ratio is less than or equal to the first value, the ratio of the area of the filtered polygon outer frame to the area of the corresponding circumscribed rectangle is calculated, if the ratio is greater than a second value, and the area of the circumscribed rectangle meets a preset area proportion condition, it is determined that the crack in the asphalt pavement image is a block crack, and the corresponding crack is labeled as a block crack category, if the ratio is less than or equal to the second value, it is determined that the crack in the asphalt pavement image is the linear crack, and the corresponding crack is labeled as the horizontal crack category or the vertical crack category. In a specific embodiment, first, the polygon center points are obtained by a polygon center point acquisition algorithm, the center points are clustered by a DBSCAN algorithm, and polygons close in space are divided into a family. The aspect ratio of the rotating minimum circumscribed rectangle is calculated for each family, if the aspect ratio is greater than 5, it is considered as a normal horizontal or vertical crack, if the aspect ratio is less than or equal to 5, the ratio of the polygon area to the circumscribed rectangle area is further calculated, when the ratio is greater than 0.5 and the circumscribed rectangle area accounts for more than 15% of the total area of the picture, it is determined as a block crack, at this time, the minimum vertical circumscribed rectangle frame of the family is used to replace the internal polygons and labeled as a “block crack” category, and the families that do not meet the conditions remain the original direction classification (horizontal or vertical crack). In this way, through morphological analysis and clustering algorithm, the cracks are classified from the geometric characteristics and spatial distribution, which makes up for the defect of insufficient generalization ability of directly labeling global features.

[0060] In summary, the application first uses the trained target model to label the cracks in the asphalt pavement image collected by the industrial line scan camera to obtain a labeled image; wherein the label frame in the labeled image is a target rectangular frame meeting a preset small size condition; each target rectangular frame in the labeled image is merged to obtain a polygon outer frame, and the polygon outer frame is filtered to obtain a filtered polygon outer frame; the main direction of the filtered polygon outer frame is judged based on a principal component analysis algorithm, and linear cracks in the asphalt pavement image are identified according to the main direction; the linear cracks include longitudinal cracks and transverse cracks; whether there is a block crack in the asphalt pavement image is judged by a clustering algorithm, and a classification result of the asphalt pavement cracks is determined according to the corresponding judgment result and the linear cracks. It can be seen that the application realizes accurate positioning by capturing crack detail features through dense small frame labeling, avoiding direct dependence on the high requirement of data volume for global features. In the post-processing stage, the morphological analysis and the clustering algorithm are used to classify the cracks from the geometric features and spatial distribution of the cracks, to realize accurate judgment of the direction and type of the cracks, and to realize high-precision crack detection and classification under the condition of small sample, improve the model migration generalization ability, and reduce the data labeling cost.

[0061] Based on the previous embodiment, the application discloses a kind of asphalt pavement crack identification classification method, can realize high-precision crack detection and classification, improve the identification precision of complex form such as block crack. Next, the specific crack classification process will be described in detail.

[0062] After the cracks in the original industrial line scan camera image are labeled by the trained model, the crack result processing includes multiple stages of operations such as frame fusion, filtering and direction and type judgment.

[0063] First, the identified dense small rectangular frame is fused: the small rectangular frame is first expanded by 1.1-1.5 times to make adjacent frames as much as possible to contact, and then the outer frame line is extracted by processing the coverage of the four sides of each frame. For the left and right upper and lower frames of each rectangle, find out the line segment that is not covered by other rectangles, merge the overlapping interval, and draw the line segment on the mask to finally form a polygon outer frame, which is used as the smallest processing unit of the crack. Then, the merged polygon is filtered to remove abnormal data with too small area or less than 3 sides, to ensure the effectiveness of subsequent processing.

[0064] Then, the main direction algorithm of PCA is used to determine the main direction of the polygon to distinguish the transverse and longitudinal cracks. The covariance matrix of the polygon coordinate points is calculated to obtain the eigenvalues and eigenvectors, and the eigenvector corresponding to the maximum eigenvalue is used to determine the main direction angle. When the angle is between 0° and 30° or between 150° and 180°, it is determined as a transverse crack, and when the angle is between 31° and 149°, it is determined as a longitudinal crack. This step starts from the morphological characteristics of the cracks and provides basic directional information for the subsequent classification. Finally, the block crack judgment link adopts the combination of DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering and morphological analysis. First, the polygon center point acquisition algorithm is used to obtain the center points of each polygon, and the DBSCAN algorithm is used to cluster the center points to divide the polygons close in space into a family. The length-width ratio of the rotating minimum bounding rectangle of each family is calculated, and if the length-width ratio is greater than 5, it is considered as a normal transverse or longitudinal crack; if the length-width ratio is less than or equal to 5, the area ratio of the polygon to the bounding rectangle is further calculated, and when the area ratio is greater than 0.5 and the area of the bounding rectangle accounts for more than 15% of the total area of the picture, it is determined as a block crack, and the minimum vertical bounding rectangle of the family is used to replace the internal polygons and labeled as the "block crack" category. The specific block crack classification is shown in Figure 3 The families that do not meet the conditions are kept in the original direction classification (transverse or longitudinal crack).

[0065] Through experimental testing, by using the above method, only about 500 densely annotated crack original pictures need to be incrementally trained on the basis of the pre-trained model, so that the crack recognition and classification (covering transverse, longitudinal, and block cracks) with a recall rate of more than 85% can be realized on most highway sections, and good migration and generalization are achieved.

[0066] In this way, by combining the spatial distribution, morphological proportion, and area characteristics of the cracks, the morphological analysis and clustering algorithm are used to accurately determine the direction and type of the cracks, effectively distinguishing the block cracks from the linear cracks, and solving the recognition problem of the block cracks caused by the variable morphology in the traditional method. In this way, the purpose of realizing high-precision crack detection and classification under the condition of small sample, improving the migration and generalization ability of the model, and reducing the data labeling cost is achieved.

[0067] Referring to Figure 4 As shown in

[0068] The labeling module 11 is used to label the cracks in the asphalt pavement images collected by the industrial line scan camera by using the trained target model to obtain the labeled images; wherein the labeling frame in the labeled image is a target rectangular frame meeting the preset small size condition;

[0069] a filtering module 12, configured to merge each of the target rectangular frames in the labeled image to obtain a polygon outer frame, and filter the polygon outer frame to obtain a filtered polygon outer frame;

[0070] a recognition module 13, configured to determine a main direction of the filtered polygon outer frame based on a principal component analysis algorithm, and recognize linear cracks in the asphalt pavement image according to the main direction; the linear cracks include longitudinal cracks and transverse cracks;

[0071] a classification result determination module 14, configured to determine whether block cracks exist in the asphalt pavement image by using a clustering algorithm, and determine a classification result of the asphalt pavement cracks according to a corresponding determination result and the linear cracks.

[0072] In summary, the application first uses a trained target model to label cracks in an asphalt pavement image collected by an industrial line scan camera to obtain a labeled image; a labeled frame in the labeled image is a target rectangular frame meeting a preset small size condition; each of the target rectangular frames in the labeled image is merged to obtain a polygon outer frame, and the polygon outer frame is filtered to obtain a filtered polygon outer frame; a main direction of the filtered polygon outer frame is determined based on a principal component analysis algorithm, and linear cracks in the asphalt pavement image are recognized according to the main direction; the linear cracks include longitudinal cracks and transverse cracks; whether block cracks exist in the asphalt pavement image is determined by using a clustering algorithm, and a classification result of the asphalt pavement cracks is determined according to a corresponding determination result and the linear cracks. It can be seen that the application realizes accurate positioning by capturing crack detail features through dense small frame labeling, and avoids high requirements for data volume by directly relying on global features. In the post-processing stage, the classification is performed from the geometric features and spatial distribution of the cracks by using morphological analysis and a clustering algorithm, accurate determination of the direction and type of the cracks is realized, and the purposes of realizing high-precision crack detection and classification under a small sample condition, improving model migration generalization ability, and reducing data labeling cost are achieved.

[0073] In some specific embodiments, the device can further include:

[0074] an image segmentation module, configured to segment a historical asphalt pavement image collected by the industrial line scan camera based on a target pixel to obtain a segmented image;

[0075] an artificial labeling module, configured to artificially label cracks in the segmented image by using target rectangular frames meeting a preset small size condition to obtain an artificially labeled image; wherein a complete crack is labeled by using a plurality of the target rectangular frames.

[0076] In some specific embodiments, the artificial labeling module can further include:

[0077] The model training unit is configured to construct an initial model based on an RT-DETR framework and a slice-assisted super-reasoning framework, train the initial model by using the artificially labeled image and transfer learning, and obtain a trained target model.

[0078] In some specific embodiments, the filtering module 12 can specifically include:

[0079] The rectangular box processing unit is configured to perform inflation processing on each target rectangular box in the labeled image based on a target magnification, and obtain a processed rectangular box.

[0080] The target outer frame line determination unit is configured to extract an outer frame line of each processed rectangular box, and determine a target outer frame line that is not covered by other processed rectangular boxes.

[0081] The polygon outer frame acquisition unit is configured to match each target outer frame line according to an endpoint position, and merge overlapping intervals of matched outer frame lines to obtain a polygon outer frame.

[0082] In some specific embodiments, the filtering module 12 can specifically include:

[0083] The filtering unit is configured to filter the polygon outer frame by eliminating polygon outer frames with an area less than a preset area threshold or a number of sides less than a target side number threshold, and obtain a filtered polygon outer frame.

[0084] In some specific embodiments, the identification module 13 can specifically include:

[0085] The feature value and feature vector acquisition unit is configured to obtain a feature value and a feature vector by calculating a covariance matrix of coordinate points of the filtered polygon outer frame.

[0086] The main direction angle determination unit is configured to determine a main direction angle by using a feature vector corresponding to a maximum feature value.

[0087] The first determination unit is configured to determine that a crack in the asphalt pavement image is a transverse crack if the main direction angle is within a first angle range.

[0088] The second determination unit is configured to determine that a crack in the asphalt pavement image is a longitudinal crack if the main direction angle is within a second angle range.

[0089] In some specific embodiments, the classification result determination module 14 can specifically include:

[0090] The clustering unit is configured to determine polygon center points of the filtered polygon outlines, and to cluster the polygon center points by using a spatial clustering algorithm based on density, and to determine polygons that meet preset conditions in space as the same family.

[0091] The aspect ratio calculation unit is configured to calculate an aspect ratio of a minimum circumscribed rectangle corresponding to each family.

[0092] The third determination unit is configured to determine that the crack in the asphalt pavement image is the linear crack if the aspect ratio is greater than a first value, and to label the corresponding crack as a transverse crack category or a longitudinal crack category.

[0093] The ratio calculation unit is configured to calculate a ratio of an area of the filtered polygon outline to an area of a circumscribed rectangle corresponding to the filtered polygon outline if the aspect ratio is less than or equal to the first value.

[0094] The fourth determination unit is configured to determine that the crack in the asphalt pavement image is the block crack if the ratio is greater than a second value and the area of the circumscribed rectangle meets a preset area proportion condition, and to label the corresponding crack as a block crack category.

[0095] The fifth determination unit is configured to determine that the crack in the asphalt pavement image is the linear crack if the ratio is less than or equal to the second value, and to label the corresponding crack as a transverse crack category or a longitudinal crack category.

[0096] Further, the embodiment of the present application further discloses an electronic device, Figure 5 is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the contents in the figure cannot be considered as any limitation on the use range of the present application.

[0097] Figure 5 A structural schematic diagram of an electronic device 20 provided by the embodiment of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, the computer program is loaded and executed by the processor 21 to realize the related steps in the asphalt pavement crack identification and classification method disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the embodiment can be an electronic computer.

[0098] In this embodiment, the power supply 23 is configured to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 is configured to create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which will not be specifically limited herein; the input and output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which will not be specifically limited herein.

[0099] In addition, the memory 22 as a carrier of resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0100] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the asphalt pavement crack identification and classification method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0101] Further, the present application also discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the foregoing disclosed asphalt pavement crack identification and classification method. For the specific steps of the method, reference can be made to the corresponding content disclosed in the foregoing embodiments, which will not be described here again.

[0102] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. For the same or similar parts between each embodiment, reference can be made to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts are described in the method part.

[0103] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0104] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM, flash memory, ROM, electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. The

[0105] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and do not imply or require any such actual relationship or order. Moreover, the terms "include", "contain", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0106] The above provides a detailed description of the technical solutions of the present application. The principles and implementation modes of the present application are described by applying specific examples. The above description of the embodiments is only to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. An asphalt pavement crack identification and classification method, characterized by, The method comprises the steps of: annotating the cracks in the asphalt pavement image collected by the industrial line scanning camera using the trained target model to obtain an annotated image; wherein the annotation box in the annotated image is a target rectangular box meeting a preset small size condition; merging each target rectangular box in the annotated image to obtain a polygon outer frame, and filtering the polygon outer frame to obtain a filtered polygon outer frame; determining the main direction of the filtered polygon outer frame based on a principal component analysis algorithm, and identifying linear cracks in the asphalt pavement image according to the main direction; the linear cracks include longitudinal cracks and transverse cracks; determining whether there are block cracks in the asphalt pavement image through a clustering algorithm, and determining the classification result of the asphalt pavement cracks according to the corresponding determination result and the linear cracks.

2. The asphalt pavement crack identification and classification method of claim 1, wherein, Before the step of annotating the cracks in the asphalt pavement image collected by the industrial line scanning camera using the trained target model, the method further comprises the steps of: segmenting historical asphalt pavement images collected by the industrial line scanning camera based on target pixels to obtain segmented images; manually annotating the cracks in the segmented images using target rectangular boxes meeting a preset small size condition to obtain manually annotated images; wherein a complete crack is annotated by a plurality of target rectangular boxes.

3. The asphalt pavement crack identification and classification method of claim 2, wherein, After the step of obtaining the manually annotated images, the method further comprises the steps of: constructing an initial model through an RT-DETR framework and a slice-assisted super-reasoning framework, training the initial model using the manually annotated images and transfer learning to obtain a trained target model.

4. The asphalt pavement crack identification and classification method of claim 1, wherein, The step of merging each target rectangular box in the annotated image to obtain a polygon outer frame comprises the steps of: performing inflation processing on each target rectangular box in the annotated image based on a target multiple to obtain processed rectangular boxes; extracting the outer frame lines of each processed rectangular box, and determining target outer frame lines that are not covered by other processed rectangular boxes; matching each target outer frame line according to the end point positions, and merging the overlapping intervals of the matched outer frame lines to obtain a polygon outer frame.

5. The asphalt pavement crack identification and classification method of claim 1, wherein, The step of filtering the polygon outer frame to obtain a filtered polygon outer frame comprises the steps of: filtering the polygon outer frame by removing polygon outer frames with an area smaller than a preset area threshold or a number of sides smaller than a target side threshold to obtain a filtered polygon outer frame.

6. The asphalt pavement crack identification and classification method of claim 1, wherein, The step of determining the main direction of the filtered polygon outer frame based on a principal component analysis algorithm, and identifying linear cracks in the asphalt pavement image according to the main direction comprises the steps of: calculating the covariance matrix of the coordinate points of the filtered polygon outer frame to obtain eigenvalues and eigenvectors; determining the main direction angle through the eigenvector corresponding to the maximum eigenvalue; if the main direction angle is within a first angle range, it is determined that the cracks in the asphalt pavement image are transverse cracks; if the main direction angle is within a second angle range, it is determined that the cracks in the asphalt pavement image are longitudinal cracks.

7. The asphalt pavement crack identification and classification method according to any one of claims 1 to 6, characterized in that, The step of determining whether there are block cracks in the asphalt pavement image through a clustering algorithm, and determining the classification result of the asphalt pavement cracks according to the corresponding determination result and the linear cracks comprises the steps of: Determine polygon center points of each of the filtered polygon outer frames, and cluster the polygon center points by a density-based noise application spatial clustering algorithm to determine polygons meeting preset conditions as the same family; Calculate aspect ratios of minimum circumscribed rectangles corresponding to each family; If the aspect ratio is greater than a first value, determine that the crack in the asphalt pavement image is the linear crack, and label the corresponding crack as the transverse crack category or the longitudinal crack category; If the aspect ratio is less than or equal to the first value, calculate a ratio of an area of the filtered polygon outer frame to an area of a corresponding circumscribed rectangle; If the ratio is greater than a second value, and the area of the circumscribed rectangle meets a preset area proportion condition, determine that the crack in the asphalt pavement image is the block crack, and label the corresponding crack as the block crack category; If the ratio is less than or equal to the second value, determine that the crack in the asphalt pavement image is the linear crack, and label the corresponding crack as the transverse crack category or the longitudinal crack category.

8. An asphalt pavement crack identification and classification device, characterized by, Comprise: The labeling module is used for labeling the cracks in the asphalt pavement image collected by the industrial line scan camera by using the trained target model to obtain a labeled image; wherein the labeling frame in the labeled image is a target rectangular frame meeting a preset small size condition; The filtering module is used for merging each of the target rectangular frames in the labeled image to obtain a polygon outer frame, and filtering the polygon outer frame to obtain a filtered polygon outer frame; The identification module is used for determining the main direction of the filtered polygon outer frame based on a principal component analysis algorithm, and identifying the linear crack in the asphalt pavement image according to the main direction; the linear crack comprises a longitudinal crack and a transverse crack; The classification result determination module is used for determining whether the block crack exists in the asphalt pavement image by a clustering algorithm, and determining the classification result of the asphalt pavement crack according to the corresponding determination result and the linear crack.

9. An electronic device, comprising: Comprise: The memory is used for storing the computer program; The processor is used for executing the computer program to realize the steps of the asphalt pavement crack identification and classification method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to realize the steps of the asphalt pavement crack identification and classification method in any one of claims 1 to 7.

Images