Highway pavement condition intelligent monitoring system and method
Through artificial intelligence technology based on deep learning, high-definition cameras and feature extraction algorithms are used to realize automated detection of highway road cracks, solving the high cost, time-consuming and safety hazards of traditional manual inspection, and improving the detection efficiency and results stability.
Patent Information
- Application Number
- CN202410985776.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-12
AI Technical Summary
Traditional manual testing methods are costly, time-consuming and have safety hazards. Due to inconsistent subjective testing standards and environmental factors, the detection results are unstable, making it difficult to effectively monitor cracks on highway road surfaces.
Using artificial intelligence monitoring technology based on deep learning, high-speed road images are obtained through high-definition cameras, image preprocessing, feature extraction and enhancement, and features are extracted and fused using spatial attention mechanism and lightweight convolution modules with different hollow rates. Finally, Use the classifier to determine whether there are cracks on the road surface.
Automatic road crack detection is realized, the dependence on manual inspection is reduced, the detection efficiency is improved, and the adaptability is better to different lighting and environmental conditions.
Smart Images

Figure CN118942053A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of road monitoring, and more specifically, to an intelligent monitoring system and method for highway pavement conditions. Background Art
[0002] China's expressways are developing rapidly, with increasing mileage, which has played an important role in promoting economic development and social progress. Cracks, as the most common defect among highway diseases, have long been a major challenge for road maintenance and management. These cracks not only destroy the integrity of the road surface, but also become a channel for corrosive substances to enter the interior of the road surface, thereby accelerating the corrosion process of the road surface. Over time, the cracks gradually expand, seriously weakening the tensile strength of the road, making it unable to withstand the increasing traffic pressure. This not only affects the aesthetics of the road, but more importantly, the presence of cracks poses a serious threat to driving safety. When driving at high speeds, cracks may cause the vehicle to lose control or even cause a traffic accident.
[0003] Therefore, regular inspection and management of road cracks are particularly important. However, traditional crack detection methods mainly rely on manual inspection, which is not only costly and time-consuming, but also poses a considerable safety hazard to operators. Due to the different shapes and sizes of cracks, different inspectors may have different test results due to inconsistent subjective test standards. In addition, manual inspection is also affected by environmental factors such as weather and light, making the test results more unstable.
[0004] Therefore, an optimized intelligent monitoring solution for highway pavement conditions is needed. Summary of the invention
[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a highway pavement condition intelligent monitoring system and method, which adopts artificial intelligence monitoring technology based on deep learning, firstly obtains the highway pavement image to be detected through a camera, then extracts the features of the highway pavement image to be detected in a high-dimensional space to obtain a road condition detection classification feature vector, then optimizes the road condition detection classification feature vector to obtain an optimized road condition detection classification feature vector, and finally uses the optimized road condition detection classification feature vector through a classifier to judge whether there are cracks in the road surface. In this way, automated pavement crack detection can be achieved, which reduces the dependence on manual detection and greatly improves the detection efficiency.
[0006] According to one aspect of the present application, there is provided a highway pavement condition intelligent monitoring system, which includes:
[0007] A road surface image data acquisition module is used to acquire the image of the high-speed road surface to be detected taken by a high-definition camera;
[0008] A road surface image data preprocessing module, used for performing image preprocessing on the to-be-detected highway road surface image to obtain a preprocessed highway road surface image;
[0009] A road surface image feature encoding module, used for performing feature encoding on the pre-processed highway road surface image to obtain a road condition detection classification feature map;
[0010] The road condition monitoring result generation module is used to determine whether there are cracks on the highway road surface to be detected based on the road condition detection classification feature map.
[0011] In the above-mentioned intelligent highway pavement condition monitoring system, the pavement image feature encoding module includes: a pavement feature extraction unit, which is used to extract pavement features from the preprocessed highway pavement image to obtain a highway pavement feature map; a pavement feature enhancement unit, which is used to enhance the features of the highway pavement feature map to obtain a highway pavement enhanced feature map; and a road condition detection classification feature generation unit, which is used to fuse the highway pavement feature map and the highway pavement enhanced feature map to obtain the road condition detection classification feature map.
[0012] In the above-mentioned highway pavement condition intelligent monitoring system, the pavement feature extraction unit is used to: pass the pre-processed highway pavement image through a pavement feature extractor based on a spatial attention mechanism to obtain the highway pavement feature map.
[0013] In the above-mentioned intelligent monitoring system for highway pavement conditions, the pavement feature enhancement unit is used to: obtain the highway pavement enhanced feature map by using lightweight convolution modules with different void rates on the highway pavement feature map.
[0014] In the above-mentioned intelligent monitoring system for highway pavement conditions, the pavement feature extractor based on the spatial attention mechanism is a first convolutional neural network model using the spatial attention mechanism, and the lightweight convolution module using different void rates includes a convolution kernel with a first void rate of a first scale and a convolution kernel with a second void rate of a first scale.
[0015] In the above-mentioned intelligent highway pavement condition monitoring system, the pavement condition monitoring result generation module includes: a feature map dimension reduction unit, which is used to pool the various feature matrices along the channel dimension of the road condition detection classification feature map to obtain a road condition detection classification feature vector; a feature vector optimization unit, which is used to perform category adaptive matching on the road condition detection classification feature vector based on weight learning parameters to obtain an optimized road condition detection classification feature vector; a classification unit, which is used to pass the optimized road condition detection classification feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether there are cracks on the highway pavement to be detected.
[0016] In the above-mentioned intelligent monitoring system for highway pavement conditions, the feature vector optimization unit is used to: calculate the road condition detection classification autocorrelation association matrix between the road condition detection classification feature vector and its transposed vector; perform matrix multiplication of the weight parameter matrix of the classifier with the road condition detection classification autocorrelation association matrix to obtain a weight parameter modulation classification autocorrelation association matrix; calculate the correlation perception factor between any two row vectors in the weight parameter modulation classification autocorrelation association matrix to obtain a metric correlation perception matrix; multiply the metric correlation perception matrix with the road condition detection classification feature vector to obtain the optimized road condition detection classification feature vector.
[0017] In the above-mentioned intelligent monitoring system for highway pavement conditions, calculating the correlation perception factor between any two row vectors in the weight parameter modulation classification autocorrelation association matrix to obtain a metric correlation perception matrix, including: calculating the correlation perception factor between any two row vectors in the weight parameter modulation classification autocorrelation association matrix by the following formula to obtain a metric correlation perception matrix;
[0018] Wherein, the formula is:
[0019]
[0020] Among them, V i is the row vector of the i-th row in the classification autocorrelation matrix modulated by the weight parameter, V j is the row vector of the jth row in the autocorrelation matrix of the weight parameter modulation classification, V j T is the transposed vector of the row vector of the jth row in the weight parameter modulation classification autocorrelation association matrix, L is the total number of rows in the weight parameter modulation classification autocorrelation association matrix, m i,j is the row vector V i and the row vector V j The correlation perception coefficient between them, ||·||2 is the bi-norm of the feature vector, represents matrix multiplication, and M' is the metric correlation-aware matrix.
[0021] According to another aspect of the present application, a method for intelligently monitoring a highway pavement condition is provided, comprising:
[0022] Acquire the image of the highway road to be inspected taken by a high-definition camera;
[0023] Performing image preprocessing on the to-be-detected highway road surface image to obtain a preprocessed highway road surface image;
[0024] Performing feature encoding on the pre-processed highway road surface image to obtain a road condition detection classification feature map;
[0025] Based on the road condition detection classification feature map, it is determined whether there are cracks on the highway road surface to be detected.
[0026] Compared with the prior art, the highway pavement condition intelligent monitoring system and method provided by the present application adopts artificial intelligence monitoring technology based on deep learning, firstly, the highway pavement image to be detected is obtained through a camera, then the high-dimensional space feature extraction is performed on the highway pavement image to be detected to obtain a road condition detection classification feature vector, then the road condition detection classification feature vector is optimized to obtain an optimized road condition detection classification feature vector, and finally the optimized road condition detection classification feature vector is used by a classifier to determine whether there are cracks on the pavement. In this way, automatic pavement crack detection can be realized, which reduces the dependence on manual detection and greatly improves the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0028] Figure 1 4 is a system block diagram of an intelligent highway pavement condition monitoring system according to an embodiment of the present application.
[0029] Figure 2 4 is an architecture diagram of an intelligent highway pavement condition monitoring system according to an embodiment of the present application.
[0030] Figure 3 It is a block diagram of a road surface image feature encoding module in an intelligent highway road condition monitoring system according to an embodiment of the present application.
[0031] Figure 4 It is a block diagram of a road surface condition monitoring result generation module in the highway road surface condition intelligent monitoring system according to an embodiment of the present application.
[0032] Figure 5 Flow chart of a method for intelligently monitoring highway pavement conditions according to an embodiment of the present application.
[0033] Figure 6 A block diagram of a computing device according to an embodiment of the present application. DETAILED DESCRIPTION
[0034] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.
[0035] As mentioned in the background technology above, the rapid development of China's expressways has provided important support for economic and social progress. Cracks are the most common defect in highway diseases. They allow corrosive substances to reach the inside of the road surface, accelerate corrosion, and reduce the tensile strength of the highway. This not only affects the appearance, but more importantly, it affects driving safety in serious cases. Therefore, it is extremely important to regularly check the severity of road cracks. Early crack detection uses manual detection, which is not only costly and time-consuming, but also poses a safety hazard to operators. The detection results also depend on the subjective detection standards of different personnel. Therefore, an optimized intelligent monitoring solution for highway pavement conditions is expected.
[0036] In recent years, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. In addition, deep learning and neural networks have also demonstrated a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields. The development of deep learning and neural networks has provided new solutions and solutions for intelligent monitoring of highway pavement conditions.
[0037] Figure 1 4 is a system block diagram of an intelligent highway pavement condition monitoring system according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of an intelligent highway pavement condition monitoring system according to an embodiment of the present application. Figure 1 and Figure 2 As shown, in the highway pavement condition intelligent monitoring system 100, it includes: a pavement image data acquisition module 110, which is used to acquire the highway pavement image to be detected taken by a high-definition camera; a pavement image data preprocessing module 120, which is used to perform image preprocessing on the highway pavement image to be detected to obtain a preprocessed highway pavement image; a pavement image feature encoding module 130, which is used to perform feature encoding on the preprocessed highway pavement image to obtain a road condition detection classification feature map; and a pavement condition monitoring result generation module 140, which is used to determine whether there are cracks on the highway pavement to be detected based on the road condition detection classification feature map.
[0038] Specifically, in the technical solution of the present application, first, obtain the image of the high-speed road surface to be detected taken by a high-definition camera. It should be well understood that the image of the high-speed road surface to be detected taken by the high-definition camera is the data source of the crack detection algorithm and the basis for subsequent image processing and analysis. It contains information about actual road surface cracks and is the key data for algorithm learning and identifying cracks. The road surface image of the high-speed road surface to be detected contains rich information, such as road surface texture, color, texture, etc., which is crucial for crack detection. By shooting high-definition images, detailed road surface features can be captured, which helps the algorithm to accurately identify cracks. The high-definition camera can provide high-resolution images, which can capture the subtle features of road surface cracks, help improve the accuracy of the crack detection algorithm, and it also reduces information loss, which is conducive to the algorithm to more accurately locate and identify cracks. It is also worth mentioning that the high-definition camera can obtain high-speed road surface images in real time, realize timely monitoring and diagnosis of road surface cracks, help to quickly discover and deal with road surface problems, and improve the safety and reliability of highways.
[0039] The camera itself has a certain degree of noise, which may be caused by the non-ideal performance of the components themselves, electronic interference and thermal noise. There may also be various interference factors in the acquisition environment of the highway road image, such as lighting changes, shadows, rain, fog, etc. These factors may cause noise in the image, so it is necessary to perform image preprocessing on the highway road image to be detected to obtain a preprocessed highway road image. By performing image preprocessing on the highway road image to be detected, noise and irrelevant information in the image can be removed, and interference can be reduced, so that the highway road image is clearer and easier to analyze, which helps to improve the robustness of the algorithm and make it better adaptable to different lighting, angles and other conditions. By optimizing the quality of the highway road image, the computational complexity of the subsequent algorithm can also be reduced, and the processing efficiency can be improved.
[0040] In an embodiment of the present application, a possible implementation method of performing image preprocessing on the highway road image to be detected to obtain a preprocessed highway road image may be: first, a denoising operation is performed on the highway road image to be detected, and the image is smoothed using a Gaussian filter to reduce high-frequency noise in the image, that is, a convolution operation is performed on the image using a Gaussian kernel function, and the kernel size and standard deviation are adjusted to control the degree of smoothness; then, an image enhancement operation is performed on the denoised image, by adjusting the grayscale distribution of the image, increasing the contrast and brightness of the image, highlighting the details, and applying a contrast enhancement algorithm, such as linear stretching or histogram matching, to adjust the contrast of the image so that different areas in the image are easier to distinguish.
[0041] Figure 3 FIG. 1 is a block diagram of a road surface image feature encoding module in an intelligent highway road surface condition monitoring system according to an embodiment of the present application. Figure 3As shown, the road surface image feature encoding module 130 includes: a road surface feature extraction unit 131, which is used to extract road surface features from the preprocessed highway road surface image to obtain a highway road surface feature map; a road surface feature enhancement unit 132, which is used to enhance the features of the highway road surface feature map to obtain a highway road surface enhanced feature map; and a road condition detection classification feature generation unit 133, which is used to fuse the highway road surface feature map and the highway road surface enhanced feature map to obtain the road condition detection classification feature map.
[0042] In order to more effectively capture the key features in the pre-processed highway road image, the pre-processed highway road image is passed through a road feature extractor based on a spatial attention mechanism to obtain a highway road feature map. In the technical solution of the present application, the essence of the road feature extractor based on the spatial attention mechanism is a convolutional neural network model using a spatial attention mechanism. Those skilled in the art should know that the convolutional neural network model performs well in image feature extraction, and it extracts features in the image through convolution operations. The use of a spatial attention mechanism can help the model focus on important areas and features in the image, improve the efficiency of extracting key information, and help accurately detect targets such as cracks. In short, the spatial attention mechanism can dynamically adjust the weight of the feature according to the importance of each position, so that the network pays more attention to meaningful features and improves the effectiveness of feature representation. Through the spatial attention mechanism, the model can learn the spatial relationship between different positions, which helps to better understand the structural information in the image and improve the representation ability of the feature. The spatial attention mechanism helps the model automatically filter out redundant information in the image, extract more representative features, reduce attention to irrelevant information, and improve the efficiency of the algorithm. The preprocessed highway road image is processed by a road feature extractor based on the spatial attention mechanism to obtain the road features in the image, such as the shape and texture of the cracks. This information is included in the produced highway road feature map.
[0043] In an embodiment of the present application, a possible implementation method of passing the preprocessed highway road surface image through a road surface feature extractor based on a spatial attention mechanism to obtain a highway road surface feature map may be: using each layer of the road surface feature extractor based on the spatial attention mechanism to perform the following on the input data in the forward pass of the layer: performing convolution processing based on a convolution kernel on the input data to obtain a convolution feature map; passing the convolution feature map through a spatial attention unit to obtain a spatial attention map; calculating the point-by-point multiplication of the convolution feature map and the spatial attention map to obtain a spatial attention feature map; inputting the spatial attention feature map into a nonlinear activation unit to obtain an activation feature map; wherein the input of the first layer of the road surface feature extractor based on the spatial attention mechanism is the preprocessed highway road surface image, and the output of the last layer of the road surface feature extractor based on the spatial attention mechanism is the highway road surface feature map.
[0044] Next, the highway feature map is processed by using lightweight convolution modules with different dilation rates to obtain a highway enhanced feature map. In the technical solution of the present application, the lightweight convolution modules using different dilation rates include a convolution kernel with a first dilation rate of a first scale and a convolution kernel with a second dilation rate of a first scale. The dilated convolution kernel can expand the receptive field of the convolution kernel by introducing the dilation rate, so that the network can capture feature information in a larger range, which is helpful for extracting global information. Compared with ordinary convolution kernels, the dilated convolution kernel can maintain resolution while expanding the receptive field, without losing the detail information of the image, which is conducive to better processing of image details. The use of dilated convolution kernels can expand the receptive field without increasing the number of parameters, improve the efficiency of feature extraction, and reduce the complexity and calculation amount of the model. By further processing the highway feature map using lightweight convolution modules with different dilation rates, features of different scales can be extracted and the generalization ability of the model can be improved. Convolution kernels with different void ratios can also adapt to scenes of different complexities, thereby better processing various details and texture information in road images and improving the adaptability of the algorithm.
[0045] In an embodiment of the present application, a possible implementation method of obtaining a highway surface enhancement feature map by using lightweight convolution modules with different void ratios may be: performing a dilated convolution process on the highway surface feature map using a convolution kernel with a first void ratio at the first scale, a mean pooling process based on a local feature matrix, and a nonlinear activation process to obtain a highway surface enhancement feature map of a first scale; performing a dilated convolution process on the highway surface feature map using a convolution kernel with a second void ratio at the first scale, a mean pooling process based on a local feature matrix, and a nonlinear activation process to obtain a highway surface enhancement feature map of a second scale; and fusing the first-scale highway surface enhancement feature map and the second-scale highway surface enhancement feature map to obtain the highway surface enhancement feature map.
[0046] The highway pavement feature map and the highway pavement enhancement feature map respectively capture highway pavement feature information of different levels and scales. In order to comprehensively utilize the information of the two, the highway pavement feature map and the highway pavement enhancement feature map are fused in the technical solution of the present application to obtain a road condition detection classification feature map. It should be easy to understand that by fusing the two feature maps, the feature characterization capability can be enhanced, so that the road condition detection algorithm can better understand and distinguish different road surface features, and improve the accuracy and comprehensiveness of road condition detection. In other words, fusing the information of different feature maps can make up for their respective shortcomings and give full play to their advantages, thereby improving the performance and effect of the road condition detection algorithm.
[0047] Figure 4 FIG. 1 is a block diagram of a road surface condition monitoring result generation module in an intelligent highway road surface condition monitoring system according to an embodiment of the present application. Figure 4 As shown, the road condition monitoring result generation module 140 includes: a feature map dimensionality reduction unit 141, which is used to pool the various feature matrices along the channel dimension of the road condition detection classification feature map to obtain a road condition detection classification feature vector; a feature vector optimization unit 142, which is used to perform category adaptive matching on the road condition detection classification feature vector based on weight learning parameters to obtain an optimized road condition detection classification feature vector; a classification unit 143, which is used to pass the optimized road condition detection classification feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether there are cracks on the highway road surface to be detected.
[0048] The road condition detection classification feature map usually contains a large amount of feature information. In order to reduce the amount of data, reduce the computational complexity, and extract the main features at the same time, the technical solution of the present application performs pooling of the feature matrices along the channel dimension on the road condition detection classification feature map to obtain a road condition detection classification feature vector.
[0049] In particular, in the technical solution of the present application, it is considered that due to the diversity of road conditions and differences in shooting conditions (such as illumination, angle, and distance), the features of different images may be inconsistent in spatial distribution. There may be many different types of cracks and other defects on the highway road surface, and these complexities may lead to local differences in the feature extraction process. The road feature extractor based on the spatial attention mechanism may not be able to fully capture all subtle features related to road cracks, especially for cracks of different types and degrees. Lightweight convolution modules using different void rates may be overly sensitive or insensitive to certain features, resulting in local differences in the feature distribution in the enhanced feature map. When fusing the highway road feature map and the enhanced feature map, the weights and importance of different features may not be properly processed, which may cause the fused road condition detection classification feature vector to show inconsistency in local areas. The road condition detection classification feature vector has local distribution differences, resulting in a class harmony offset relative to the predetermined class label of the classifier when it is classified by the classifier, affecting the classification iteration effect. Based on this, the road condition detection classification feature vector is subjected to class adaptive matching based on weight learning parameters.
[0050] Specifically, in an embodiment of the present application, the road condition detection classification feature vector is subjected to category adaptive matching based on weight learning parameters to obtain an optimized road condition detection classification feature vector, including: calculating a road condition detection classification autocorrelation association matrix between the road condition detection classification feature vector and its transposed vector; performing matrix multiplication of the weight parameter matrix of the classifier with the road condition detection classification autocorrelation association matrix to obtain a weight parameter modulated classification autocorrelation association matrix; calculating the correlation perception factor between any two row vectors in the weight parameter modulated classification autocorrelation association matrix to obtain a metric correlation perception matrix; multiplying the metric correlation perception matrix with the road condition detection classification feature vector to obtain the optimized road condition detection classification feature vector.
[0051] More specifically, calculating the correlation perception factor between any two row vectors in the weight parameter modulation classification autocorrelation association matrix to obtain a metric correlation perception matrix includes: calculating the correlation perception factor between any two row vectors in the weight parameter modulation classification autocorrelation association matrix using the following formula to obtain a metric correlation perception matrix; wherein the formula is:
[0052]
[0053] Among them, V i is the row vector of the i-th row in the classification autocorrelation matrix modulated by the weight parameter, V j is the row vector of the jth row in the autocorrelation matrix of the weight parameter modulation classification, V j Tis the transposed vector of the row vector of the jth row in the weight parameter modulation classification autocorrelation association matrix, L is the total number of rows in the weight parameter modulation classification autocorrelation association matrix, m i,j is the row vector V i and the row vector V j The correlation perception coefficient between them, ||·||2 is the bi-norm of the feature vector, represents matrix multiplication, and M' is the metric correlation-aware matrix.
[0054] In an embodiment of the present application, another expression of performing class adaptive matching based on weight learning parameters on the road condition detection classification feature vector to obtain an optimized road condition detection classification feature vector includes: processing the road condition detection classification feature vector according to the following formula to obtain the optimized road condition detection classification feature vector; wherein the formula is:
[0055]
[0056]
[0057] Among them, V c is the road condition detection classification feature vector, V c T is the transposed vector of the road condition detection classification feature vector, M c is the road condition detection classification autocorrelation matrix, M p is the weight parameter matrix of the classifier, M t is the weight parameter modulated classification autocorrelation matrix, V i is the row vector of the i-th row in the classification autocorrelation matrix modulated by the weight parameter, V j is the row vector of the jth row in the autocorrelation matrix of the weight parameter modulation classification, V j T is the transposed vector of the row vector of the jth row in the weight parameter modulation classification autocorrelation association matrix, L is the total number of rows in the weight parameter modulation classification autocorrelation association matrix, m i,j is the row vector V i and the row vector V j The correlation perception coefficient between them, ||·||2 is the bi-norm of the feature vector, represents matrix multiplication, M' is the metric correlation perception matrix, R is a set of real numbers, and V' is the optimized road condition detection classification feature vector.
[0058] In the technical solution of the present application, it is considered that the road condition detection classification feature vector has local distribution differences, which leads to the problem that when the classification judgment is performed by the classifier, there will be a class harmony offset relative to the predetermined class label of the classifier, which affects the classification iteration effect. Based on this, in the technical solution of the present application, based on the autocorrelation dimension of the road condition detection classification feature vector, the spatial sub-dimensional complexity of the high-dimensional feature space of the feature distribution is used to perform correlation expansion to obtain the road condition detection classification autocorrelation association matrix. Furthermore, the road condition detection classification autocorrelation association matrix is subjected to metric correlation perception based on weight learning parameters. By introducing weight learning parameters, the local distribution differences of different road condition detection classification feature vectors can be effectively quantified and adjusted, thereby reducing the class harmony offset caused by these local distribution differences when the classifier performs classification judgment.
[0059] Finally, the optimized road condition detection classification feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether there are cracks on the highway road surface to be detected. It should be well understood that the optimized road condition detection classification feature vector contains the road surface feature information after feature extraction and fusion, and these features are of great significance for the identification of cracks. The classifier can judge the state of the road surface based on these features, including whether there are cracks. By learning the feature distribution between different categories in the training data set, the classifier can learn an appropriate decision boundary, so that different categories (with cracks and without cracks) can be effectively distinguished. The use of a classifier can realize the automatic identification of whether there are cracks on the highway road surface to be detected, reduce manual intervention and improve detection efficiency. The classification result directly reflects the system's judgment on the state of the highway road surface, which is intuitive, clear and easy to understand.
[0060] In summary, the highway pavement condition intelligent monitoring system 100 based on the embodiment of the present application is explained, which adopts artificial intelligence monitoring technology based on deep learning, firstly obtains the highway pavement image to be detected through the camera, then extracts the features of the highway pavement image to be detected in high-dimensional space to obtain the road condition detection classification feature vector, then optimizes the road condition detection classification feature vector to obtain the optimized road condition detection classification feature vector, and finally uses the optimized road condition detection classification feature vector through the classifier to judge whether there are cracks in the road surface. In this way, automatic road crack detection can be realized, which reduces the dependence on manual detection and greatly improves the detection efficiency.
[0061] As described above, the intelligent monitoring system 100 for the road surface condition of a highway according to the embodiment of the present application can be implemented in various terminal devices, such as a server for intelligent monitoring of the road surface condition of a highway. In one example, the intelligent monitoring system 100 for the road surface condition of a highway according to the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the intelligent monitoring system 100 for the road surface condition of a highway can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the intelligent monitoring system 100 for the road surface condition of a highway can also be one of the many hardware modules of the terminal device.
[0062] Alternatively, in another example, the highway pavement condition intelligent monitoring system 100 and the terminal device may also be separate devices, and the highway pavement condition intelligent monitoring system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.
[0063] Figure 5 Flow chart of the method for intelligent monitoring of highway pavement conditions according to an embodiment of the present application. Figure 5 As shown, in the method for intelligent monitoring of highway pavement conditions, the method includes: S110, acquiring an image of the highway pavement to be detected taken by a high-definition camera; S120, performing image preprocessing on the image of the highway pavement to be detected to obtain a preprocessed highway pavement image; S130, performing feature encoding on the preprocessed highway pavement image to obtain a road condition detection classification feature map; S140, judging whether there are cracks on the highway pavement to be detected based on the road condition detection classification feature map.
[0064] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned method for intelligent monitoring of highway pavement conditions have been described in detail above. Figures 1 to 3 The invention has been introduced in detail in the description of the intelligent monitoring system for highway pavement conditions, and therefore, its repeated description will be omitted.
[0065] In summary, the intelligent monitoring method of the highway pavement condition based on the embodiment of the present application is explained, which adopts the artificial intelligence monitoring technology based on deep learning, firstly obtains the image of the highway pavement to be detected through the camera, then extracts the features of the highway pavement to be detected in high-dimensional space to obtain the road condition detection classification feature vector, then optimizes the road condition detection classification feature vector to obtain the optimized road condition detection classification feature vector, and finally uses the optimized road condition detection classification feature vector through the classifier to judge whether there are cracks in the road surface. In this way, automatic road crack detection can be realized, which reduces the dependence on manual detection and greatly improves the detection efficiency.
[0066] Below, reference Figure 6 To describe a computing device according to an embodiment of the present application. Figure 6 A block diagram of a computing device according to an embodiment of the present application.
[0067] like Figure 6 As shown, the computing device 10 includes a processing system 11, one or more computer-readable media 12, and one or more I / O interfaces 13 that are communicatively coupled to each other. The computing device 10 also includes a bus system and / or other forms of connection mechanisms (not shown).
[0068] Processing system 11 represents the functionality of performing one or more operations using hardware. Thus, processing system 11 is illustrated as including hardware elements 14 that may be configured as processors, functional blocks, etc. Hardware elements 14 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, a processor may be composed of (multiple) semiconductors and / or transistors (e.g., electronic integrated circuits (ICs)). The processor may control other components in computing device 10 to perform desired functions.
[0069] The computer-readable medium 12 is illustrated as including a memory 15. The memory 15 represents a memory associated with one or more computer-readable media. The memory 15 may include volatile media (such as random access memory (RAM)) and / or non-volatile media (such as read-only memory (ROM), flash memory, optical disk, magnetic disk, etc.). The memory 15 may include fixed media (e.g., RAM, ROM, fixed hard disk drive, etc.) and removable media (e.g., flash memory, removable hard disk drive, optical disk, etc.). The computer-readable medium 12 may store one or more computer program instructions to implement the above-described highway pavement condition intelligent monitoring method of the various embodiments of the present application and / or other desired functions.
[0070] One or more I / O interfaces 13 represent functions that allow a user to input commands and information to the computing device 10 and optionally also allow information to be presented to the user and / or other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone (e.g., for voice input), a scanner, a touch function (e.g., a capacitive or other sensor configured to detect physical touch), a camera (e.g., a visible or invisible wavelength (such as infrared frequency) can be used to detect motion that does not involve touch as a gesture), etc. Examples of output devices include a display device (e.g., a monitor or projector), a speaker, a printer, a network card, a tactile response device, etc. The output device can output various information to the outside, including the result of whether there are cracks in the highway road surface to be detected, etc.
[0071] Of course, to simplify, Figure 6Only some of the components related to the present application in the computing device 10 are shown, and components such as buses, etc. are omitted. In addition, according to specific application situations, the computing device 10 may also include any other appropriate components.
[0072] In addition to the above methods and systems, embodiments of the present application may also be computer-readable media. Computer-readable media may include various media that can be accessed by computing device 10. As an example and not limitation, computer-readable media may include "computer-readable storage media" and "computer-readable signal media".
[0073] In contrast to a simple signal transmission, carrier wave, or signal itself, "computer-readable storage medium" refers to a medium and / or device capable of persistently storing information, and / or a tangible storage device. Therefore, a computer-readable storage medium refers to a non-signal-bearing medium. Computer-readable storage media include hardware such as volatile and non-volatile, removable and non-removable media and / or storage devices implemented in a method or technology suitable for storing information (such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data). Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage device, hard disk, cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, or other storage device, tangible medium, or article of manufacture suitable for storing desired information and accessible by a computer.
[0074] In the several embodiments provided in this application, it should be understood that the disclosed systems, methods, etc. can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0075] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0076] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0077] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application.
[0078] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present application is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application. Any attached figure mark in the claims should not be regarded as limiting the claims involved.
[0079] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.
[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit of the technical solution of the present application.
Claims
1. An intelligent monitoring system for highway pavement conditions, characterized in that: include: A road surface image data acquisition module is used to acquire the image of the high-speed road surface to be detected taken by a high-definition camera; A road surface image data preprocessing module, used for performing image preprocessing on the to-be-detected highway road surface image to obtain a preprocessed highway road surface image; A road surface image feature encoding module, used for performing feature encoding on the pre-processed highway road surface image to obtain a road condition detection classification feature map; The road condition monitoring result generation module is used to determine whether there are cracks on the highway road surface to be detected based on the road condition detection classification feature map.
2. The intelligent highway pavement condition monitoring system according to claim 1 is characterized in that: The road surface image feature encoding module comprises: A road surface feature extraction unit, used for extracting road surface features from the pre-processed highway road surface image to obtain a highway road surface feature map; A road surface feature enhancement unit, used for performing feature enhancement on the highway road surface feature map to obtain a highway road surface enhanced feature map; A road condition detection classification feature generation unit is used to fuse the highway road feature map and the highway road enhancement feature map to obtain the road condition detection classification feature map.
3. The intelligent highway pavement condition monitoring system according to claim 2 is characterized in that: The road surface feature extraction unit is used to: pass the pre-processed highway road surface image through a road surface feature extractor based on a spatial attention mechanism to obtain the highway road surface feature map.
4. The intelligent highway pavement condition monitoring system according to claim 3 is characterized in that: The road surface feature enhancement unit is used to: obtain the highway road surface enhanced feature map by using lightweight convolution modules with different void rates on the highway road surface feature map.
5. The intelligent highway pavement condition monitoring system according to claim 4 is characterized in that: The road feature extractor based on the spatial attention mechanism is a first convolutional neural network model using the spatial attention mechanism, and the lightweight convolution module using different void rates includes a convolution kernel with a first void rate of a first scale and a convolution kernel with a second void rate of the first scale.
6. The intelligent highway pavement condition monitoring system according to claim 5 is characterized in that: The road condition monitoring result generating module comprises: A feature map dimensionality reduction unit, used for performing pooling of each feature matrix along the channel dimension on the road condition detection classification feature map to obtain a road condition detection classification feature vector; A feature vector optimization unit, used for performing class adaptive matching on the road condition detection classification feature vector based on weight learning parameters to obtain an optimized road condition detection classification feature vector; The classification unit is used to pass the optimized road condition detection classification feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether there are cracks on the highway road surface to be detected.
7. The intelligent highway pavement condition monitoring system according to claim 6 is characterized in that: The feature vector optimization unit is used for: Calculating a road condition detection classification autocorrelation matrix between the road condition detection classification feature vector and its transposed vector; Performing matrix multiplication of the weight parameter matrix of the classifier and the road condition detection classification autocorrelation matrix to obtain a weight parameter modulation classification autocorrelation matrix; Calculating the correlation perception factor between any two row vectors in the weight parameter modulation classification autocorrelation association matrix to obtain a metric correlation perception matrix; The metric correlation perception matrix is multiplied by the road condition detection classification feature vector to obtain the optimized road condition detection classification feature vector.
8. The intelligent highway pavement condition monitoring system according to claim 7 is characterized in that: Calculating the correlation perception factor between any two row vectors in the weight parameter modulation classification autocorrelation association matrix to obtain a metric correlation perception matrix, including: calculating the correlation perception factor between any two row vectors in the weight parameter modulation classification autocorrelation association matrix using the following formula to obtain a metric correlation perception matrix; Wherein, the formula is: Among them, V i is the row vector of the i-th row in the classification autocorrelation matrix modulated by the weight parameter, V j is the row vector of the jth row in the autocorrelation matrix of the weight parameter modulation classification, V j T is the transposed vector of the row vector of the jth row in the weight parameter modulation classification autocorrelation association matrix, L is the total number of rows in the weight parameter modulation classification autocorrelation association matrix, m i,j is the row vector V i and the row vector V j The correlation perception coefficient between them, ||·||2 is the bi-norm of the feature vector, represents matrix multiplication, and M' is the metric correlation-aware matrix.
9. A method for intelligent monitoring of highway pavement conditions, characterized in that: include: Acquire the image of the highway road to be inspected taken by a high-definition camera; Performing image preprocessing on the to-be-detected highway road surface image to obtain a preprocessed highway road surface image; Performing feature encoding on the pre-processed highway road surface image to obtain a road condition detection classification feature map; Based on the road condition detection classification feature map, it is determined whether there are cracks on the highway road surface to be detected.
10. The method for intelligent monitoring of highway pavement conditions according to claim 9, characterized in that: Performing feature encoding on the pre-processed highway road surface image to obtain a road condition detection classification feature map includes: Extracting road surface features from the pre-processed highway road surface image to obtain a highway road surface feature map; Performing feature enhancement on the highway road feature map to obtain a highway road enhanced feature map; The highway road feature map and the highway road enhancement feature map are fused to obtain the road condition detection classification feature map.
Citation Information
Cited By
Pavement performance evaluation method and system based on joint judgment
CN120088244A
A pavement performance evaluation method and system based on joint judgment
CN120088244B
