Asphalt concrete pavement early crack detection method and system

Abstract

The invention discloses an asphalt concrete pavement early crack detection method and system, and the method comprises the steps: S1, obtaining pavement three-dimensional point cloud data to form a texture image, recognizing a gray level change region, and marking the region as a suspected crack region as a first crack degree; s2, segmenting a suspected crack region through a deep learning model Mask R-CNN, determining a crack type, and measuring geometric parameters of the crack as a second crack degree; s3, in combination with historical data and environmental factors, the development trend and the damage degree of the crack are evaluated through a machine learning model to serve as a third crack degree; s4, establishing a crack detection mechanism, and recording and tracking the state change of the early asphalt concrete pavement crack; and S5, setting and comparing crack state evaluation thresholds, and performing early warning when the crack state evaluation thresholds exceed a threshold boundary mechanism. According to the invention, an early crack state tracking and early warning mechanism is realized, and the efficiency and quality of pavement maintenance work are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pavement crack detection, and in particular to a method and system for detecting early cracks in an asphalt concrete pavement. Background Art

[0002] With the continuous growth of transportation volume and vehicle load, asphalt concrete pavement is prone to early cracks during use. These cracks not only affect the quality and appearance of the road surface, but also accelerate the aging of the pavement structure, reduce its service life, and pose a threat to driving safety. Traditional pavement maintenance methods often rely on regular manual inspections, which are inefficient and difficult to detect early small-scale cracks in time, resulting in missing the best time for repair and increasing the cost and complexity of later maintenance. Therefore, there is an urgent need for an efficient and accurate automatic detection method to quickly identify and evaluate early cracks in asphalt concrete pavements.

[0003] Traditional technical solutions mainly rely on manual visual inspection or simple image processing technology to detect pavement cracks. The former requires a lot of manpower input, the detection results are greatly affected by personal experience, and it is difficult to fully cover minor cracks; although the latter introduces digital image processing methods, it is limited by the limitations of the algorithm and environmental factors (such as changes in lighting conditions), the detection accuracy is not high, and the false alarm rate is high. Some existing automated detection systems are also based on two-dimensional image analysis, lack the effective use of three-dimensional spatial information, and cannot accurately capture the true shape and depth information of cracks. In addition, traditional technologies rarely consider the dynamic characteristics of crack development and the impact of the external environment on it, making it difficult to provide a scientific prediction and early warning mechanism.

[0004] Existing pavement crack detection technologies have been automated to a certain extent, for example, by collecting pavement images through cameras installed on vehicles and using computer vision algorithms for preliminary analysis. However, such methods can usually only identify relatively obvious cracks, and are not effective in detecting small or hidden cracks. They also fail to fully utilize the powerful pattern recognition capabilities of modern machine learning and deep learning. In addition, the current technical solutions are not sophisticated enough in terms of crack type classification and geometric parameter measurement, and lack a complete full-process solution from data acquisition to crack status assessment. This limits its effectiveness and reliability in practical applications, especially in large-scale highway network monitoring.

[0005] To address the above-mentioned problems, this technical solution proposes an automatic detection method for early cracks in asphalt concrete pavements based on three-dimensional point cloud data and deep learning models. It can more accurately detect and classify early cracks, provide detailed crack information to support decision-making, and establish an effective crack status tracking and early warning mechanism, significantly improving the efficiency and quality of pavement maintenance work. Summary of the invention

[0006] The present invention provides a method for detecting early cracks in an asphalt concrete pavement, comprising:

[0007] S1, obtaining the 3D point cloud data of the road surface to form a texture image, scanning the texture image by texture analysis technology, identifying the area with grayscale changes in the texture image, marking it as a suspected crack area as the first crack level;

[0008] S2, in the suspected crack area marked by the first crack level, the suspected crack area is segmented by the deep learning model Mask R-CNN, the crack type is determined and the geometric parameters of the crack are measured as the second crack level;

[0009] S3, based on the crack type and geometric parameter information provided by the second crack index, combined with historical data and environmental factors, the development trend of the cracks and their damage degree are evaluated through a machine learning model as the third crack index;

[0010] S4, establishing a crack detection mechanism through the third crack degree, and using the crack detection mechanism to record and track the state changes of early asphalt concrete pavement cracks;

[0011] S5. Set a crack status assessment threshold, compare the crack status output by the crack detection mechanism with the crack status assessment threshold, issue an early warning through the threshold boundary mechanism, and form early crack detection of asphalt concrete pavement.

[0012] Preferably, the texture image in S1 is formed by acquiring three-dimensional point cloud data and projecting it onto a two-dimensional plane to form a texture image; the three-dimensional point cloud data is formed by a mobile detection device equipped with a laser scanner driving along the asphalt concrete pavement to be detected, and the laser scanner is used to capture the three-dimensional coordinate data of the pavement surface in the form of a high-density point cloud.

[0013] Preferably, the texture image formed by using texture analysis technology is scanned, and the grayscale value correlation between the pixel point and other pixels in its neighborhood is calculated by constructing a grayscale co-occurrence matrix GLCM; for a given pixel point (x, y), the frequency of occurrence of grayscale value pairs between the pixel point and the neighboring pixels with a distance of d and a direction of θ is counted to form a co-occurrence matrix GLCM (d, θ); the element p (i, j|d, θ) in the co-occurrence matrix is ​​the probability of the co-occurrence of a pixel with grayscale level i and a pixel with grayscale level j in its neighborhood at a distance of d and a direction of θ; the texture feature contrast and correlation are calculated by the co-occurrence matrix, and an area with high contrast or low correlation is identified as an area with significant grayscale changes, and marked as a suspected crack area; the contrast formula is: C D =∑ i,j (ij) 2 p(i,j|d,θ), the correlation formula is: where μ i , μ j are the grayscale means of i and j, σ i , σ j are the grayscale standard deviations of i and j respectively.

[0014] Preferably, in S2, the deep learning model Mask R-CNN is used to process the texture image marked as a suspected crack area, and the texture image is input into the Mask R-CNN model trained with a large amount of annotated data. The Mask R-CNN model identifies and generates a pixel-level mask for each suspected crack area to achieve accurate segmentation of the crack area; the crack type is matched with a predefined crack template library through the mask output by the Mask R-CNN model, and the crack type is identified as a longitudinal crack, a transverse crack, an oblique crack or a tortoise crack by calculating the similarity and geometric features between the mask shape features and the template; the geometric parameters of the crack are measured by the mask boundary extraction technology, a curve or a straight line segment is fitted along the edge of the crack, and then the length of the crack is calculated, the maximum width of the crack is determined by analyzing the grayscale change gradient in the crack width direction, and the direction angle of the crack is calculated according to the main axis direction of the crack mask to form a second crack degree.

[0015] Preferably, the Mask R-CNN model identifies and generates a pixel-level mask for each suspected crack area and then accurately segments the crack area; the texture image marked as a suspected crack area by texture analysis technology is input into a pre-trained deep learning model Mask R-CNN, and image features are extracted using a convolutional neural network (CNN), and candidate areas containing cracks are generated through a region proposal network (RPN); for each candidate area, Mask R-CNN performs a classification task to determine whether it is a crack, and uses a fully convolutional network (FCN) branch to generate a corresponding pixel-level binary mask for each category to define the boundary of the crack, and uses a non-maximum suppression (NMS) algorithm to remove redundant overlapping masks and retain high-score masks of real cracks; the crack area in the original texture image is accurately segmented based on the generated pixel-level mask to ensure that each crack area is independently and completely marked.

[0016] Preferably, the identification of the crack type is performed by matching the pixel-level mask output by the Mask R-CNN model with a predefined crack template library; the matching identification is performed by calculating the similarity and geometric features between the mask shape features and the template, and the shape features of the calculated mask include but are not limited to area A, perimeter P, main axis direction θ, aspect ratio Wherein L and W are the length and width of the mask respectively; for cracks of the type of longitudinal cracks, transverse cracks, oblique cracks or cracks, the corresponding templates are defined according to typical geometric features, such as the high aspect ratio and nearly vertical direction angle of longitudinal cracks, the low aspect ratio and horizontal direction angle of transverse cracks, the intermediate aspect ratio and non-orthogonal direction angle of oblique cracks, and the complex branching structure recognition of cracks, using the similarity function Where m represents the mask, t represents the template, and w 1 、w 2 and w 3 are the area weight coefficient, perimeter weight coefficient and main axis weight coefficient, σ A and σ P They are the area standard deviation parameter and the perimeter standard deviation parameter, which are used to quantify the similarity between the mask and the template; by comparing the similarity scores between the mask and various crack templates, the crack type corresponding to the template with the highest score is selected as the final recognition result, thereby achieving matching and classification of crack types.

[0017] Preferably, the third crack dimension is used to evaluate the development trend of cracks and the degree of damage thereof; the third crack dimension collects the crack type and geometric parameter information provided by the second crack dimension, including the length, width, direction and type of the crack, and combines historical data, such as the crack development records at the same or similar positions in the past, and environmental factors including but not limited to temperature changes, humidity levels, and traffic flow, to construct a multidimensional feature space; the multidimensional features are input into a pre-trained machine learning model, and a time series analysis model is used to capture the change pattern of cracks over time, and the crack expansion rate R and damage index DI are calculated; the expansion rate R is calculated from the historical data of crack growth, and the damage index DI comprehensively considers the influence of geometric parameters and environmental factors, and generates the future development trend of the cracks according to the output results of the machine learning model.

[0018] Preferably, the calculation of the crack growth rate R and the damage index DI is performed by inputting multidimensional features into a pre-trained machine learning model for calculation; the multidimensional features include crack type and geometric parameters, the geometric parameters include length L, width W, direction θ, historical data and environmental factors, wherein the environmental factors include temperature change T, humidity level H, and traffic flow F; the crack growth rate R is calculated as follows: Wherein ΔL(t) represents the change in crack length within the time interval Δt, and quantifies the growth rate of the crack; the damage index DI is calculated by the time series comprehensive evaluation model, and the formula is: Among them, X t Represents the multidimensional feature vector at time point t, including geometric parameters and environmental factors; f i (X t) is a nonlinear function that evaluates the impact of each feature on damage; i is with f i The associated weight coefficient; g(H t ) is a nonlinear function of historical data, φ is a scaling factor that adjusts the impact of historical data on current damage assessment, and n is the number of features.

[0019] Preferably, the setting of the crack state assessment threshold is based on historical data and experience to determine the initial threshold T of different crack types. C , the initial threshold T C Including the critical values ​​of geometric parameters, expansion rate R and damage index DI, the crack state data output by the crack detection mechanism and the initial threshold T C Real-time comparison of crack monitoring points, using the formula D = | DI-T C |, where D represents the difference between the current damage index and the threshold, which is used to quantify the degree to which the crack state deviates from the normal range. When D exceeds the preset safety boundary D b , that is, D>D b , triggers the alarm.

[0020] An asphalt concrete pavement early crack detection system includes a data acquisition module, an image generation module, a texture analysis module, a Mask R-CNN module, an evaluation module, an early warning module and a feedback module;

[0021] The data acquisition module acquires the three-dimensional point cloud data of the road surface by driving a mobile detection device equipped with a laser scanner along the asphalt concrete road surface to be detected to collect data, and transmits the collected three-dimensional point cloud data to the image generation module;

[0022] The image generation module projects the three-dimensional point cloud data onto a two-dimensional plane to form a texture image; the image generation module is connected to the data acquisition module to receive the three-dimensional point cloud data provided by the data acquisition module; and the generated texture image is sent to the texture analysis module;

[0023] The texture analysis module uses a gray level co-occurrence matrix to analyze gray level changes in the texture image, identifies suspected crack areas, receives the texture image from the image generation module, and outputs the marked suspected crack area image to the deep learning processing module;

[0024] The Mask R-CNN module segments the suspected crack area, determines the crack type and measures the geometric parameters; the Mask R-CNN module accepts input from the texture analysis module and passes the result to the evaluation module;

[0025] The evaluation module combines historical data, environmental factors and information provided by the second crack degree to predict the development trend and damage degree of the cracks using a machine learning model; the evaluation module receives crack type and geometric parameter information from the deep learning processing module, and provides the evaluation results to the early warning module;

[0026] The early warning module sets a crack state assessment threshold, compares the current crack state with the threshold, and triggers an alarm when the safety boundary is exceeded. The early warning module is connected to the assessment module, receives the assessment result, and issues an alarm according to the set rules;

[0027] The feedback module records and tracks the state changes of early asphalt concrete pavement cracks through a crack detection mechanism, including a database or log system for long-term storage and management of crack information.

[0028] Compared with the prior art, the technical solution of this application has the following technical effects:

[0029] The present invention projects three-dimensional point cloud data onto a two-dimensional plane to form a texture image, and uses a gray-level co-occurrence matrix (GLCM) to analyze these images, thereby solving the problem that traditional two-dimensional image analysis methods are difficult to capture subtle cracks in the road surface. Through statistical analysis of the correlation between grayscale values ​​of pixel points and their neighborhoods, areas with significant grayscale changes can be identified as suspected crack areas. The method based on high-density three-dimensional coordinate data not only improves the accuracy and sensitivity of early crack detection, but also reduces the false alarm rate caused by environmental factors such as changes in lighting conditions, thereby ensuring more reliable crack detection results.

[0030] The present invention introduces the deep learning model Mask R-CNN to process texture images marked as suspected crack areas, thereby solving the problems of insufficiently fine classification of crack types and inaccurate measurement of geometric parameters in the prior art. Mask R-CNN can generate a pixel-level mask for each suspected crack area, achieve accurate segmentation of the crack area, and accurately identify the crack type by matching with a predefined crack template library, thereby improving the accuracy of crack type recognition. It can also fit curves or straight line segments along the crack edge to calculate the specific geometric parameters of the crack, such as length, width, and direction angle, thereby providing detailed data support for subsequent crack status assessment, greatly enhancing the scientificity and reliability of crack detection.

[0031] The present invention solves the technical problem that traditional methods cannot effectively predict the development trend and damage degree of cracks by constructing a multi-dimensional feature space and applying a pre-trained machine learning model. Combining the crack type and geometric parameter information provided by the second crack dimension, as well as the influence of historical data and environmental factors, the machine learning model can capture the change pattern of cracks over time and calculate the crack expansion rate and damage index. This allows us to not only evaluate the current state of cracks, but also predict future development trends, helping road management departments to formulate reasonable maintenance plans in advance, optimize resource allocation, reduce unnecessary maintenance costs, and improve the safety and service life of roads.

[0032] The present invention solves the problem of the lack of effective early warning mechanism in traditional monitoring methods by setting a crack status assessment threshold and comparing it with real-time monitoring data. When the crack status exceeds the preset safety boundary, the system will immediately trigger an alarm and notify relevant departments to take necessary measures, thereby achieving dynamic tracking and timely response to early cracks in asphalt concrete pavements, and avoiding the occurrence of small cracks turning into big problems due to delayed repairs. In addition, the ability of the feedback module to record and manage crack information also provides strong support for long-term road health monitoring, promoting refined management and scientific decision-making in highway maintenance work.

[0033] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application so that it can be implemented in accordance with the contents of the specification, and to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings as follows.

[0034] Based on the detailed description of the specific embodiments of the present application in combination with the accompanying drawings below, those skilled in the art will become more aware of the above and other objects, advantages and features of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale.

[0036] Figure 1 This is a flow chart of an automatic detection method for early cracks in an asphalt concrete pavement according to the present invention;

[0037] Figure 2A structural diagram of an automatic detection method for early cracks in an asphalt concrete pavement according to the present invention;

[0038] Figure 3 This is a structural diagram of a Mask R-CNN model of an automatic detection method for early cracks in asphalt concrete pavement according to the present invention;

[0039] Figure 4 This is a structural diagram of an automatic detection system for early cracks in asphalt concrete pavement according to the present invention. DETAILED DESCRIPTION

[0040] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, rather than all of the embodiments. In the following description, specific details such as specific configuration and components are provided only to help fully understand the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, for clarity and brevity, the description of known functions and structures is omitted in the embodiment.

[0041] It should be understood that the references to "one embodiment" or "this embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "one embodiment" or "this embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0042] In addition, the present application may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplicity and clarity, and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0043] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" in this article describes another type of association object relationship, indicating that there can be two relationships. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0044] The term "at least one" in this article is merely a description of the association relationship of associated objects, indicating that there may be three relationships. For example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0045] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusions.

[0046] Example 1

[0047] This embodiment mainly describes a method for detecting early cracks in an asphalt concrete pavement. Figure 1-Figure 2 As shown, the following steps are included:

[0048] S1, obtaining the 3D point cloud data of the road surface to form a texture image, scanning the texture image by texture analysis technology, identifying the area with grayscale changes in the texture image, marking it as a suspected crack area as the first crack level;

[0049] Furthermore, the texture image is formed by acquiring three-dimensional point cloud data and projecting it onto a two-dimensional plane to form a texture image; the three-dimensional point cloud data is formed by a mobile detection device equipped with a laser scanner driving along the asphalt concrete pavement to be inspected, and the laser scanner is used to capture the three-dimensional coordinate data of the pavement surface in the form of a high-density point cloud.

[0050] Furthermore, the texture analysis technology is used to scan the formed texture image, and the grayscale value correlation between the pixel point and other pixels in its neighborhood is calculated by constructing the grayscale co-occurrence matrix GLCM; for a given pixel point (x, y), the frequency of occurrence of grayscale value pairs between the pixel point and the neighboring pixels with a distance of d and a direction of θ is counted to form the co-occurrence matrix GLCM(d,θ); the element p(i,j|d,θ) in the co-occurrence matrix is ​​the probability of the co-occurrence of a pixel with grayscale level i and a pixel with grayscale level j in its neighborhood at a distance of d and a direction of θ; the texture feature contrast and correlation are calculated by the co-occurrence matrix, and the areas with high contrast or low correlation are identified as areas with significant grayscale changes and marked as suspected crack areas; the contrast formula is: C D =∑ i,j (ij) 2 p(i,j|d,θ), the correlation formula is: where μ i , μ j are the grayscale means of i and j, σ i , σ jare the grayscale standard deviations of i and j respectively.

[0051] S2, in the suspected crack area marked by the first crack level, the suspected crack area is segmented by the deep learning model Mask R-CNN, the crack type is determined and the geometric parameters of the crack are measured as the second crack level;

[0052] Furthermore, the deep learning model Mask R-CNN is used to process texture images marked as suspected crack areas, and the texture images are input into the Mask R-CNN model that has been trained with a large amount of annotated data. The Mask R-CNN model identifies and generates pixel-level masks for each suspected crack area to achieve accurate segmentation of the crack area. The crack type is matched with the predefined crack template library through the mask output by the Mask R-CNN model, and the crack type is identified as longitudinal crack, transverse crack, oblique crack or tortoise crack by calculating the similarity and geometric features between the mask shape features and the template. The geometric parameters of the crack are measured by mask boundary extraction technology, and curves or straight line segments are fitted along the crack edge to calculate the length of the crack. The maximum width of the crack is determined by analyzing the grayscale change gradient in the crack width direction, and the orientation angle of the crack is calculated according to the main axis direction of the crack mask to form the second crack degree.

[0053] Further, if Figure 3 As shown in the figure, the Mask R-CNN model identifies and generates a pixel-level mask for each suspected crack area, and then accurately segments the crack area; the texture image marked as a suspected crack area by texture analysis technology is input into the pre-trained deep learning model Mask R-CNN, and the convolutional neural network CNN is used to extract image features, and the region proposal network RPN is used to generate candidate regions containing cracks; for each candidate region, Mask R-CNN performs a classification task to determine whether it is a crack, and uses the fully convolutional network FCN branch to generate a corresponding pixel-level binary mask for each category to define the boundary of the crack, and uses the non-maximum suppression NMS algorithm to remove redundant overlapping masks and retain the high-score masks of the real cracks; the crack area in the original texture image is accurately segmented according to the generated pixel-level mask to ensure that each crack area is independently and completely marked.

[0054] Furthermore, the crack type is identified by matching the pixel-level mask output by the Mask R-CNN model with the predefined crack template library; matching identification calculates the shape features of the mask by calculating the similarity and geometric features between the mask shape features and the template, including but not limited to the area A, perimeter P, main axis direction θ, aspect ratio Where L and W are the length and width of the mask, respectively. For cracks of the longitudinal, transverse, oblique or crazing types, the corresponding templates are defined according to typical geometric features, such as the high aspect ratio and near-vertical orientation angle of the longitudinal crack, the low aspect ratio and horizontal orientation angle of the transverse crack, the intermediate aspect ratio and non-orthogonal orientation angle of the oblique crack, and the complex branching structure recognition of the crazing, using the similarity function Where m represents the mask, t represents the template, and w 1 、w 2 and w 3 are the area weight coefficient, perimeter weight coefficient and main axis weight coefficient, σ A and σ P They are the area standard deviation parameter and the perimeter standard deviation parameter, which are used to quantify the similarity between the mask and the template; by comparing the similarity scores between the mask and various crack templates, the crack type corresponding to the template with the highest score is selected as the final recognition result to achieve matching and classification of crack types.

[0055] S3, based on the crack type and geometric parameter information provided by the second crack index, combined with historical data and environmental factors, the development trend of the cracks and their damage degree are evaluated through a machine learning model as the third crack index;

[0056] Furthermore, the third crack dimension is used to evaluate the development trend of cracks and the degree of damage. The third crack dimension collects the crack type and geometric parameter information provided by the second crack dimension, including the length, width, direction and type of the crack, and combines historical data, such as the crack development records at the same or similar locations in the past, and environmental factors including but not limited to temperature changes, humidity levels, and traffic flow, to construct a multidimensional feature space. The multidimensional features are input into a pre-trained machine learning model, and a time series analysis model is used to capture the change pattern of cracks over time, and the crack expansion rate R and damage index DI are calculated. The expansion rate R is calculated from the historical data of crack growth, and the damage index DI comprehensively considers the influence of geometric parameters and environmental factors, and generates the future development trend of cracks based on the output results of the machine learning model.

[0057] Furthermore, the calculation of crack growth rate R and damage index DI is performed by inputting multidimensional features into a pre-trained machine learning model for calculation; the multidimensional features include crack type and geometric parameters, the geometric parameters include length L, width W, direction θ, historical data and environmental factors, where the environmental factors include temperature change T, humidity level H, and traffic flow F; the crack growth rate R is calculated as Where ΔL(t) represents the change in crack length within the time interval Δt, which quantifies the growth rate of the crack; the damage index DI is calculated by the time series comprehensive evaluation model, and the formula is: Among them, Xt Represents the multidimensional feature vector at time point t, including geometric parameters and environmental factors; f i (X t ) is a nonlinear function that evaluates the impact of each feature on damage; i is with f i The associated weight coefficient; g(H t ) is a nonlinear function of historical data, φ is a scaling factor that adjusts the impact of historical data on current damage assessment, and n is the number of features.

[0058] S4, establishing a crack detection mechanism through the third crack degree, and using the crack detection mechanism to record and track the state changes of early asphalt concrete pavement cracks;

[0059] A crack detection mechanism is established using the third crack degree. Based on the crack type and its geometric parameter information provided by the second crack degree, combined with historical data and environmental factors (such as temperature changes and traffic flow), a machine learning system is used to evaluate the development trend and degree of damage of cracks. Specifically, a recurrent neural network is used to identify and classify crack patterns and track their changes over time. The recurrent neural network model is trained with a large number of samples to accurately capture the subtle differences in crack development, provide accurate trend predictions, and enhance the monitoring capability of crack state changes.

[0060] S5. Setting a crack state assessment threshold, comparing the crack state output by the crack detection mechanism with the crack state assessment threshold, and issuing an alarm through the threshold boundary mechanism to form early crack detection of the asphalt concrete pavement;

[0061] Furthermore, the setting of the crack state assessment threshold is based on historical data and experience to determine the initial threshold T for different crack types. C , initial threshold T C Including the critical values ​​of geometric parameters, expansion rate R and damage index DI, the crack state data output by the crack detection mechanism and the initial threshold T C Real-time comparison of crack monitoring points, using the formula D = | DI-T C |, where D represents the difference between the current damage index and the threshold, which is used to quantify the degree to which the crack state deviates from the normal range. When D exceeds the preset safety boundary D b , that is, D>D b , triggers the alarm.

[0062] This embodiment uses a laser scanner to obtain high-density three-dimensional coordinate data of the road surface, and after forming a texture image, uses advanced texture analysis techniques such as gray-level co-occurrence matrix (GLCM) to identify suspected crack areas, and uses advanced deep learning models such as Mask R-CNN to perform pixel-level segmentation on suspected crack areas, accurately determine the crack type and its geometric parameters, and combine historical data and environmental factors. The development trend and degree of damage of the cracks are evaluated through a machine learning prediction system, which achieves accurate detection and classification of early cracks, provides detailed crack information, and improves the efficiency and quality of road maintenance work.

[0063] Example 2

[0064] This embodiment describes in detail the optimization scheme of the deep learning model Mask R-CNN, which specifically includes:

[0065] When the suspected crack area is marked by the first crack degree, the Mask R-CNN model is used to segment the suspected crack area. If the Mask R-CNN model fails to detect any cracks, there are actually no cracks in the area or the cracks are so small that they do not reach the recognition threshold of the model. At this time, the regional situation is recorded by the Mask R-CNN model as the second micro-crack degree, and a comprehensive analysis is performed using historical data and environmental factors. For example, in some cases, no obvious cracks are found through the segmentation of the Mask R-CNN model, but considering the road conditions at the same location or other similar conditions in the past, as well as the influence of external factors such as temperature changes, humidity levels, and traffic flow, there is a potential risk of cracks. Therefore, by setting a micro-crack degree mechanism, when no cracks are found in multiple consecutive tests but potential risks are shown in other indicators, the threshold is automatically lowered to increase the sensitivity, and the regional information is segmented to determine the crack type and measure the geometric parameters of the crack as the second micro-crack degree.

[0066] Through the second micro-crack information, a secondary verification is performed using a multi-source data fusion method. Through infrared thermal imaging or ultrasonic testing, information different from optical images is provided. Even if the Mask R-CNN model fails to directly identify cracks, it can be cross-verified with the help of multiple detection methods to ensure that no factors affecting road safety are missed. At the same time, this multimodal data collection method also provides a richer information basis for subsequent data analysis, helping to better understand the health status of the road surface and make scientific and reasonable maintenance decisions.

[0067] This embodiment describes in detail the situation where Mask R-CNN fails to find cracks. It lowers the recognition threshold to ensure that potential risks are not missed. It uses multi-source data fusion methods, such as combining infrared thermal imaging or ultrasonic detection results, to perform secondary verification to find early damage that is not visually noticeable. It combines multiple verifications, multimodal data analysis, and the establishment of long-term monitoring archives to significantly improve the accuracy and comprehensiveness of crack detection, and can effectively monitor the health of the road surface even when no obvious cracks are found in the early stage.

[0068] Example 3

[0069] This embodiment is based on Embodiment 1 and describes in detail the optimization scheme of threshold warning, which specifically includes:

[0070] To ensure the accuracy and timeliness of the alarm, the threshold warning includes a multi-level threshold mechanism, which are the warning threshold T W and the emergency threshold T J , corresponding to different degrees of crack development. When the crack state reaches the warning threshold TL, a warning message is generated; and when the crack state exceeds the emergency threshold T J , the emergency response procedure is immediately initiated to ensure that the road management department can take prompt action. The entire process forms a complete closed loop for early crack detection of asphalt concrete pavement, from setting the evaluation threshold to issuing an alarm through the threshold boundary mechanism, thus achieving effective monitoring and management of pavement cracks.

[0071] Warning threshold T L It is used to identify cracks in the early development stage. When the state parameters of the crack (such as damage index DI or expansion rate R) reach T L When cracks are detected, early warning information is automatically generated and sent to the road maintenance team. This enables maintenance personnel to take preventive measures at an early stage to prevent the cracks from further deteriorating.

[0072] When the crack state parameter exceeds the emergency threshold T J , indicating that the cracks have reached a level that requires immediate attention and treatment. At this point, the emergency response procedure is immediately initiated, triggering an alarm to notify relevant departments and provide detailed information such as the location, type and current status of the cracks. This immediate response mechanism ensures that road management departments can take quick action to arrange necessary repairs and avoid potential safety hazards. By distinguishing between early warning and emergency situations, the multi-level threshold system improves the flexibility and response speed of the system and ensures accurate monitoring of the development of pavement cracks.

[0073] This embodiment provides an optimization scheme for threshold warning, which realizes effective monitoring and management of road cracks through a multi-level threshold mechanism, from setting evaluation thresholds to issuing warnings through threshold boundary mechanisms. It can not only capture the changing trend of cracks in real time, but also automatically adjust the alarm level according to the predefined threshold, thereby ensuring that maintenance personnel always have the latest road conditions. In this way, the detection method not only improves the efficiency of crack monitoring, but also improves the level of road safety management, providing solid technical support for timely and effective road maintenance.

[0074] Example 4

[0075] This embodiment describes in detail an automatic detection system for early cracks in asphalt concrete pavement. Figure 4 As shown, it includes a data acquisition module, an image generation module, a texture analysis module, a Mask R CNN module, an evaluation module, an early warning module and a feedback module; through a multi-level processing flow, efficient and accurate detection and management of early cracks in asphalt concrete pavement can be achieved.

[0076] The data acquisition module is responsible for acquiring the three-dimensional point cloud data of the road surface. It collects data by driving along the asphalt concrete road surface to be inspected through a mobile detection device equipped with a high-precision laser scanner (LiDAR) system, capturing the precise three-dimensional coordinate information of the road surface, and combining with the inertial measurement unit (IMU) to ensure the position accuracy during the data collection process. The data acquisition module passes the collected three-dimensional point cloud data to the image generation module to provide basic data support for subsequent processing.

[0077] The image generation module receives the 3D point cloud data from the data acquisition module and projects it onto a 2D plane using an advanced point cloud data to image conversion algorithm to form a texture image with grayscale or color information. This process not only preserves the spatial geometric characteristics of the original point cloud, but also enhances the visualization of potential crack features. The generated texture image is then sent to the texture analysis module for further crack identification and classification.

[0078] The texture analysis module uses the gray-level co-occurrence matrix (GLCM) technique to analyze the grayscale changes in the texture image to identify locations that exhibit abnormal texture features. These areas are marked as suspected crack areas. The texture analysis module receives the texture image from the image generation module, uses an adaptive filtering algorithm to remove noise and improve image quality, then applies GLCM to calculate the grayscale value correlation between each pixel and other pixels in its neighborhood, and finally outputs the marked suspected crack area image to the deep learning processing module (i.e., Mask R-CNN module).

[0079] The Mask R-CNN module is based on a deep learning model to accurately segment suspected crack areas, determine the crack type (longitudinal cracks, transverse cracks, oblique cracks or fissures), and measure the geometric parameters of the cracks (such as length, width and direction). This module accepts input from the texture analysis module, uses a pre-trained convolutional neural network (CNN) to extract image features, and generates candidate regions containing cracks through a region proposal network (RPN). For each candidate region, Mask R-CNN performs a classification task to determine whether it is a crack, and uses a fully convolutional network (FCN) branch to generate a corresponding pixel-level binary mask for each category, thereby achieving accurate segmentation of the crack area. The segmentation results and geometric parameter information are then passed to the evaluation module.

[0080] The evaluation module combines historical data, environmental factors, and information provided by the second crack degree to use a machine learning model to predict the development trend of cracks and their degree of damage. The evaluation module receives crack type and geometric parameter information from the deep learning processing module and constructs a multidimensional feature space, which includes the geometric parameters, expansion rate R, and damage index DI of the cracks, as well as environmental factors such as temperature changes, humidity levels, and traffic flow. Through time series analysis models such as long short-term memory networks (LSTM) or autoregressive integrated moving average models (ARIMA), the evaluation module captures the change pattern of cracks over time, calculates the crack expansion rate R and damage index DI, and passes the evaluation results to the early warning module.

[0081] The early warning module sets the crack status assessment threshold, including the early warning threshold T L and the emergency threshold T J , corresponding to different degrees of crack development. When the crack state reaches the warning threshold T L When the crack state exceeds the emergency threshold T J , the emergency response procedure is immediately initiated, an alarm is triggered to notify relevant departments, and detailed information such as the crack location, type and current status is provided. The early warning module is connected to the assessment module, receives the assessment results, and issues an alarm according to the set rules to ensure that the road management department can take prompt action to avoid potential safety hazards.

[0082] The feedback module records and tracks the status changes of early asphalt concrete pavement cracks through the crack detection mechanism, integrates the data output of all modules, and also includes a database or log system for long-term storage and management of crack information. The feedback module regularly updates the crack status and dynamically adjusts the crack assessment threshold to ensure that the crack status output each time is the latest and accurate. In addition, the feedback module also supports maintenance personnel to review the historical crack development and assist in the formulation of long-term road maintenance plans, thus forming a complete early crack detection closed loop and realizing effective monitoring and management of pavement cracks.

[0083] This implementation provides a full-process solution from data collection to crack detection, evaluation and early warning by describing the early crack automatic detection system in detail. It also ensures the accurate recording and long-term management of crack information, greatly improving the efficiency and reliability of early crack detection in asphalt concrete pavements.

[0084] The above are only preferred embodiments of the present invention, which do not limit the scope of protection of the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any changes, modifications, replacements, integrations and parameter changes to these embodiments within the spirit and principles of the present invention through conventional substitutions or without departing from the principles and spirit of the present invention fall within the scope of protection of the present invention.

Claims

1. A method for detecting early cracks in asphalt concrete pavement, characterized in that: include: S1, obtaining the 3D point cloud data of the road surface to form a texture image, scanning the texture image by texture analysis technology, identifying the area with grayscale changes in the texture image, marking it as a suspected crack area as the first crack level; S2, in the suspected crack area marked by the first crack level, the suspected crack area is segmented by the deep learning model Mask R-CNN, the crack type is determined and the geometric parameters of the crack are measured as the second crack level; S3, based on the crack type and geometric parameter information provided by the second crack index, combined with historical data and environmental factors, the development trend of the cracks and their damage degree are evaluated through a machine learning model as the third crack index; S4, establishing a crack detection mechanism through the third crack degree, and using the crack detection mechanism to record and track the state changes of early asphalt concrete pavement cracks; S5. Set a crack status assessment threshold, compare the crack status output by the crack detection mechanism with the crack status assessment threshold, issue an early warning through the threshold boundary mechanism, and form early crack detection of asphalt concrete pavement.

2. The method for detecting early cracks in an asphalt concrete pavement according to claim 1, characterized in that: The texture image in S1 is formed by acquiring three-dimensional point cloud data and projecting it onto a two-dimensional plane to form a texture image; the three-dimensional point cloud data is formed by a mobile detection device equipped with a laser scanner driving along the asphalt concrete road surface to be detected, and the laser scanner is used to capture the three-dimensional coordinate data of the road surface in the form of a high-density point cloud.

3. A method for detecting early cracks in an asphalt concrete pavement according to claim 1 or 2, characterized in that: The texture analysis technology is used to scan the formed texture image, and the grayscale value correlation between the pixel point and other pixel points in its neighborhood is calculated by constructing a grayscale co-occurrence matrix GLCM; for a given pixel point (x, y), the frequency of occurrence of grayscale value pairs between the pixel point and neighboring pixels with a distance of d and a direction of θ is counted to form a co-occurrence matrix GLCM (d, θ); the element p (i, j|d, θ) in the co-occurrence matrix is ​​the probability of co-occurrence of a pixel with grayscale level i and a pixel with grayscale level j in its neighborhood at a distance of d and a direction of θ; The texture feature contrast and correlation are calculated by the co-occurrence matrix, and the area with high contrast or low correlation is identified as the area with significant grayscale change, and marked as the suspected crack area; the contrast formula is: C D =∑ i,j (ij) 2 p(i,j|d,θ), the correlation formula is: where μ i , μ j are the grayscale means of i and j, σ i , σ j are the grayscale standard deviations of i and j respectively.

4. The method for detecting early cracks in an asphalt concrete pavement according to claim 1, characterized in that: In S2, the deep learning model Mask R-CNN is used to process the texture image marked as a suspected crack area, and the texture image is input into the Mask R-CNN model trained with a large amount of annotated data. The Mask R-CNN model identifies and generates a pixel-level mask for each suspected crack area to achieve accurate segmentation of the crack area; the crack type is matched with a predefined crack template library through the mask output by the Mask R-CNN model, and the crack type is identified as a longitudinal crack, a transverse crack, an oblique crack or a crack by calculating the similarity and geometric features between the mask shape features and the template; the geometric parameters of the crack are measured by the mask boundary extraction technology, and a curve or a straight line segment is fitted along the edge of the crack to calculate the length of the crack, and the maximum width of the crack is determined by analyzing the grayscale change gradient in the crack width direction, and the direction angle of the crack is calculated according to the main axis direction of the crack mask to form a second crack degree.

5. The method for detecting early cracks in an asphalt concrete pavement according to claim 4, characterized in that: The Mask R-CNN model identifies and generates a pixel-level mask for each suspected crack area and then accurately segments the crack area; the texture image marked as a suspected crack area by texture analysis technology is input into the pre-trained deep learning model Mask R-CNN, and the convolutional neural network CNN is used to extract image features, and the candidate area containing cracks is generated by the region proposal network RPN; for each candidate area, Mask R-CNN performs a classification task to determine whether it is a crack, and uses the fully convolutional network FCN branch to generate a corresponding pixel-level binary mask for each category to define the boundary of the crack, and uses the non-maximum suppression NMS algorithm to remove redundant overlapping masks and retain the high-score mask of the real crack; the crack area in the original texture image is accurately segmented according to the generated pixel-level mask to ensure that each crack area is independently and completely marked.

6. The method for detecting early cracks in an asphalt concrete pavement according to claim 4, characterized in that: The identification of the crack type is performed by matching the pixel-level mask output by the Mask R-CNN model with a predefined crack template library; the matching identification is performed by calculating the similarity and geometric features between the mask shape features and the template, and the shape features of the calculated mask include but are not limited to area A, perimeter P, main axis direction θ, aspect ratio Wherein L and W are the length and width of the mask respectively; for cracks of the type of longitudinal cracks, transverse cracks, oblique cracks or cracks, the corresponding templates are defined according to typical geometric features, such as the high aspect ratio and nearly vertical direction angle of longitudinal cracks, the low aspect ratio and horizontal direction angle of transverse cracks, the intermediate aspect ratio and non-orthogonal direction angle of oblique cracks, and the complex branching structure recognition of cracks, using the similarity function Where m represents the mask, t represents the template, w1, w2 and w3 are the area weight coefficient, perimeter weight coefficient and main axis weight coefficient respectively, σ A and σ P They are the area standard deviation parameter and the perimeter standard deviation parameter, which are used to quantify the similarity between the mask and the template; by comparing the similarity scores between the mask and various crack templates, the crack type corresponding to the template with the highest score is selected as the final recognition result, thereby achieving matching and classification of crack types.

7. The method for detecting early cracks in an asphalt concrete pavement according to claim 1, characterized in that: The third crack dimension is used to evaluate the development trend of cracks and the degree of damage; the third crack dimension collects the crack type and geometric parameter information provided by the second crack dimension, including the length, width, direction and type of the crack, and combines historical data, such as the crack development records of the same or similar positions in the past, and environmental factors including but not limited to temperature changes, humidity levels, and traffic flow, to construct a multidimensional feature space; the multidimensional features are input into a pre-trained machine learning model, and a time series analysis model is used to capture the change pattern of cracks over time, and the crack expansion rate R and damage index DI are calculated; the expansion rate R is calculated from the historical data of crack growth, and the damage index DI comprehensively considers the influence of geometric parameters and environmental factors, and generates the future development trend of the cracks according to the output results of the machine learning model.

8. The method for detecting early cracks in an asphalt concrete pavement according to claim 7, characterized in that: The calculation of the crack growth rate R and the damage index DI is performed by inputting multidimensional features into a pre-trained machine learning model for calculation; the multidimensional features include crack type and geometric parameters, the geometric parameters include length L, width W, direction θ, historical data and environmental factors, wherein the environmental factors include temperature change T, humidity level H, and traffic flow F; the crack growth rate R is calculated as follows: Wherein ΔL(t) represents the change in crack length within the time interval Δt, and quantifies the growth rate of the crack; the damage index DI is calculated by the time series comprehensive evaluation model, and the formula is: Among them, X t Represents the multidimensional feature vector at time point t, including geometric parameters and environmental factors; f i (X t ) is a nonlinear function that evaluates the impact of each feature on damage; w i is with f i The associated weight coefficient; g(H t ) is a nonlinear function of historical data, φ is a scaling factor that adjusts the impact of historical data on current damage assessment, and n is the number of features.

9. The method for detecting early cracks in an asphalt concrete pavement according to claim 1, characterized in that: The setting of the crack state assessment threshold is based on historical data and experience to determine the initial threshold T of different crack types. C , the initial threshold T C Including the critical values ​​of geometric parameters, expansion rate R and damage index DI, the crack state data output by the crack detection mechanism and the initial threshold T C Real-time comparison of crack monitoring points, using the formula D = | DI-T C |, where D represents the difference between the current damage index and the threshold, which is used to quantify the degree to which the crack state deviates from the normal range. When D exceeds the preset safety boundary D b , that is, D>D b , triggers the alarm.

10. An early crack detection system for asphalt concrete pavement, characterized in that: It includes data acquisition module, image generation module, texture analysis module, Mask R-CNN module, evaluation module, warning module and feedback module; The data acquisition module acquires the three-dimensional point cloud data of the road surface by driving a mobile detection device equipped with a laser scanner along the asphalt concrete road surface to be detected to collect data, and transmits the collected three-dimensional point cloud data to the image generation module; The image generation module projects the three-dimensional point cloud data onto a two-dimensional plane to form a texture image; the image generation module is connected to the data acquisition module to receive the three-dimensional point cloud data provided by the data acquisition module; and the generated texture image is sent to the texture analysis module; The texture analysis module uses a gray level co-occurrence matrix to analyze gray level changes in the texture image, identifies suspected crack areas, receives the texture image from the image generation module, and outputs the marked suspected crack area image to the deep learning processing module; The Mask R-CNN module segments the suspected crack area, determines the crack type and measures the geometric parameters; the Mask R-CNN module accepts input from the texture analysis module and passes the result to the evaluation module; The evaluation module combines historical data, environmental factors and information provided by the second crack degree to predict the development trend and damage degree of the cracks using a machine learning model; the evaluation module receives crack type and geometric parameter information from the deep learning processing module, and provides the evaluation results to the early warning module; The early warning module sets a crack state assessment threshold, compares the current crack state with the threshold, and triggers an alarm when the safety boundary is exceeded. The early warning module is connected to the assessment module, receives the assessment result, and issues an alarm according to the set rules; The feedback module records and tracks the state changes of early asphalt concrete pavement cracks through a crack detection mechanism, including a database or log system for long-term storage and management of crack information.

Images