Road surface crack monitoring method, system and device, unmanned aerial vehicle, medium and product

Abstract

The invention relates to the technical field of image analysis, and discloses a pavement crack monitoring method, system and device, an unmanned aerial vehicle, a medium and a product, the pavement crack monitoring method comprises the following steps: acquiring environment information of a target area, and selecting a first shooting parameter according to the environment information; the target area is an area including a road to be monitored; shooting a to-be-monitored road in the target area according to the first shooting parameter to obtain a target image; performing crack identification on the target image to obtain an identification result; judging whether the road to be monitored has cracks or not according to the recognition result; if the crack exists in the road to be monitored, the target crack of the target road with the crack is monitored, crack recognition is carried out on the shot target image, the target crack of the target road with the crack is monitored, and the road crack monitoring efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of image analysis technology, and in particular to a method, system, device, drone, medium and product for monitoring pavement cracks. Background Art

[0002] Monitoring and analyzing pavement cracks is a crucial component of modern road maintenance and management. Road health directly impacts people's quality of life and regional economic development. Pavement cracks are not only an early sign of road degradation but also impact road safety and comfort. Therefore, timely and accurate monitoring of pavement cracks and maintenance of cracked pavement are crucial to preventing further deterioration, effectively extending road life, reducing maintenance costs, and ensuring traffic safety.

[0003] At present, traditional methods for monitoring pavement cracks mainly rely on manual visual inspection and photogrammetry. This method is not only time-consuming and labor-intensive, but also subject to constraints such as weather conditions and road conditions. In addition, the monitoring results are easily affected by the experience and subjective judgment of technicians, making it difficult to ensure the consistency and accuracy of the monitoring results. Summary of the Invention

[0004] In view of this, the present invention provides a method, system, device, drone, medium and product for monitoring pavement cracks to solve the problem that traditional pavement crack monitoring methods are difficult to ensure the consistency and accuracy of monitoring results.

[0005] In a first aspect, the present invention provides a method for monitoring road cracks, comprising: obtaining environmental information of a target area, and selecting a first shooting parameter based on the environmental information; the target area is an area including a road to be monitored; photographing the road to be monitored in the target area based on the first shooting parameter to obtain a target image; performing crack identification on the target image to obtain an identification result; judging whether cracks exist in the road to be monitored based on the identification result; if cracks exist in the road to be monitored, monitoring target cracks in the target road where cracks exist.

[0006] The present invention obtains environmental information and selects a first shooting parameter based on the environmental information. Since different environmental conditions will affect the shooting effect, the present invention obtains environmental information to specifically select shooting parameters to ensure the quality stability of the target image obtained subsequently, improve the adaptability of the application of the present invention, avoid the problem of repeated shooting caused by using fixed parameters, and improve the efficiency of subsequent shooting and reduce the waste of storage resources by selecting shooting parameters. The present invention shoots the road to be monitored in the target area according to the first shooting parameter to obtain a target image. The target image obtained by the present invention based on the first shooting parameter adapted to the environment can truly reflect the actual situation of the road. The present invention identifies cracks in the target image to obtain an identification result. Based on the identification result, it is determined whether cracks exist in the road to be monitored. Based on the presence of cracks in the road to be monitored, the target cracks in the target road with cracks are monitored. Compared with related technologies, the present invention is not affected by human factors and improves the efficiency and accuracy of road crack identification and monitoring. The present invention timely discovers changes in the target cracks by monitoring the target cracks, so as to take preventive measures and ensure traffic safety.

[0007] In an optional embodiment, target cracks on a target road where cracks exist are monitored, including: determining a preliminary path based on the position of the target road and with the goal of maximizing accumulated rewards according to a preset reward function; optimizing the preliminary path according to pheromone concentration and heuristic information to obtain a target path; the pheromone concentration is a variable in the ant colony algorithm used to characterize the attractiveness of the path, and the heuristic information is information used for path selection in the ant colony algorithm; focusing and photographing the target road at a target shooting point according to a second shooting parameter based on the target path to monitor target cracks on the target road; the second shooting parameter is a shooting parameter obtained by precision adjustment of the first shooting parameter.

[0008] The present invention aims to maximize the accumulated rewards, and according to a preset reward function, determines a preliminary path to ensure that the path planning revolves around the core needs of crack monitoring, so that the preliminary path is more in line with the actual application scenario. The present invention optimizes the preliminary path according to the pheromone concentration and the heuristic information to obtain the target path, ensures that the target path adapts to the changes in the environment, and ensures the rationality and effectiveness of the target path. The present invention focuses on shooting the target road according to the second shooting parameters at the target shooting point according to the target path to monitor the target cracks on the target road, and reaches the target shooting point according to the target path to ensure that the target cracks on the target road are accurately positioned and shot. The second shooting parameter is obtained after the precision adjustment of the first shooting parameter. For the specific task of crack monitoring, the shooting parameters are optimized, which helps to obtain more accurate crack feature information. Shooting is performed according to the planned target path and precise shooting parameters. While meeting the crack monitoring accuracy requirements, unnecessary waste of shooting resources is avoided, and efficient use of resources is achieved.

[0009] In an optional embodiment, crack recognition is performed on a target image to obtain a recognition result, including: extracting features from the target image to obtain multiple feature maps; determining spatial position weights of the multiple feature maps, and fusing the multiple feature maps based on the spatial position weights to obtain multiple target feature maps; inputting the multiple target feature maps into a trained road crack extraction model to obtain a recognition result; the input of the road crack extraction model is the target feature map, and the output of the road crack extraction model is the recognition result; the pavement crack monitoring method also includes a training process for the road crack extraction model; the training process for the road crack extraction model includes: obtaining a road crack target sample library, labeling multiple target samples in the road crack target sample library to obtain multiple target labeled samples; and training the road crack extraction model based on the multiple target labeled samples.

[0010] The present invention extracts features from a target image to generate multiple feature maps. Each feature map can capture different aspects of the image's features, achieving the goal of capturing these multiple features. The image is analyzed from multiple dimensions, providing a more comprehensive description of the target image's features, thereby providing a rich data foundation for accurate road crack identification. The present invention considers the differences in the spatial importance of different features, highlighting spatial location information critical to crack identification while suppressing irrelevant or interfering information. Through weighted fusion, the features of key areas are more fully reflected in the fused target feature map, further improving the accuracy of subsequent crack identification. The present invention inputs the target feature map into a trained road crack extraction model to generate identification results. After training with a large amount of data, the road crack extraction model learns the characteristic patterns and regularities of road cracks. It can quickly and accurately analyze and determine the input target feature map, identifying the presence of cracks and related information within the map. This significantly improves the efficiency and accuracy of crack identification and reduces the workload and error of manual identification. The present invention labels multiple target samples in a road crack target sample library to obtain multiple labeled target samples. The road crack extraction model is trained based on these multiple labeled target samples, thereby improving the recognition efficiency of the road crack extraction model.

[0011] In an optional embodiment, after photographing the road to be monitored in the target area according to the first shooting parameter and obtaining the target image, the method for monitoring road cracks further includes: inputting the target image into a trained deep convolutional neural network to perform noise reduction and image enhancement processing on the target image, and the target image for crack identification is the image after noise reduction and image enhancement processing.

[0012] The present invention inputs the target image into a trained deep convolutional neural network to perform noise reduction and image enhancement on the target image, overcoming problems such as image blur, color distortion, and difficulty in identifying cracks caused by insufficient imaging conditions and complex road conditions, and is conducive to constructing high-definition, high-resolution images with clear target boundaries.

[0013] In an optional embodiment, after crack recognition is performed on the target image and recognition results are obtained, the pavement crack monitoring method further includes: formatting the recognition results and environmental information, establishing a directory structure based on the recognition results and environmental information after format conversion; and storing the recognition results and environmental information in the directory structure.

[0014] The present invention converts the format of recognition results and environmental information to ensure compatibility with different systems and tools, facilitates reading, parsing, and processing by different programming languages ​​and platforms, and achieves data standardization. The present invention establishes a directory structure that can organize the recognition results and environmental information after format conversion in a logical and clear manner. When searching for specific data, users can quickly locate the corresponding directory location, greatly improving the efficiency of data retrieval. Compared with the storage method of conventional storage structures, it avoids the trouble of blindly searching through large amounts of data and saves time.

[0015] In a second aspect, the present invention provides a pavement crack monitoring system, which includes a mobile terminal, an unmanned device equipped with a supercomputing board module, a hyperspectral camera module, and a communication module, and a backend server; the mobile terminal is used to send a pavement crack monitoring instruction input by a user to the unmanned device; the supercomputing board module is used to obtain environmental information of a target area based on the pavement crack monitoring instruction and select a first shooting parameter based on the environmental information; the target area is an area including a road to be monitored; the hyperspectral camera module is used to shoot the road to be monitored in the target area according to the first shooting parameter to obtain a target image; the supercomputing board module is used to identify cracks in the target image to obtain an identification result; the supercomputing board module is used to determine whether cracks exist in the road to be monitored based on the identification result; the hyperspectral camera module is used to monitor target cracks in the target road where cracks exist based on the presence of cracks in the road to be monitored; the communication module is used to send the monitoring results to the backend server; the backend server is used to convert the monitoring results into a preset vector layer and to visualize the preset vector layer on a preset map; the backend server is used to send a preset map including the preset vector layer to the mobile terminal; and the mobile terminal is used to display the preset map including the preset vector layer.

[0016] In a third aspect, the present invention provides a device for monitoring road cracks, comprising: a first shooting parameter selection unit, for acquiring environmental information of a target area, and selecting a first shooting parameter based on the environmental information; the target area is an area including a road to be monitored; a target image determination unit, for photographing the road to be monitored in the target area according to the first shooting parameter, to obtain a target image; an intelligent interpretation unit, for identifying cracks in the target image, to obtain an identification result; a crack judgment unit, for judging whether there are cracks in the road to be monitored based on the identification result; and a crack monitoring unit, for monitoring target cracks in the target road where cracks exist based on the presence of cracks in the road to be monitored.

[0017] In a fourth aspect, the present invention provides a drone comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for monitoring pavement cracks according to the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0018] In a fifth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for monitoring pavement cracks according to the first aspect or any corresponding embodiment thereof.

[0019] In a sixth aspect, the present invention provides a computer program product comprising computer instructions for causing a computer to execute the method for monitoring pavement cracks according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in related technologies, the following briefly introduces the drawings required for use in the specific embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 4 is a flow chart of a method for monitoring pavement cracks according to an embodiment of the present invention.

[0022] Figure 2 4 is a flow chart of another method for monitoring pavement cracks according to an embodiment of the present invention.

[0023] Figure 3 2 is a schematic diagram of the operation process of the road crack extraction model according to an embodiment of the present invention.

[0024] Figure 4 2 is a schematic structural diagram of a pavement crack monitoring system according to an embodiment of the present invention.

[0025] Figure 5 1 is a schematic diagram of the working process of a pavement crack monitoring system according to an embodiment of the present invention.

[0026] Figure 6 2 is a schematic diagram of the working process of another pavement crack monitoring system according to an embodiment of the present invention.

[0027] Figure 7 4 is a structural block diagram of a device for monitoring pavement cracks according to an embodiment of the present invention.

[0028] Figure 8 4 is a schematic diagram of the hardware structure of the UAV according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0030] Monitoring and analyzing pavement cracks is a crucial component of modern road maintenance and management. The health of the road network directly impacts people's quality of life. Pavement cracks are not only an early sign of road aging, but also affect road safety and comfort.

[0031] Therefore, timely and accurate monitoring of pavement cracks is crucial to preventing further deterioration of pavement cracks, which can effectively extend the service life of roads, reduce maintenance costs, and ensure traffic safety.

[0032] At present, traditional pavement crack monitoring mainly relies on manual visual inspection and photogrammetry. This method is not only time-consuming and labor-intensive, but also subject to constraints such as weather conditions and road conditions. In addition, the results are easily affected by the experience and subjective judgment of the inspectors, making it difficult to ensure the consistency and accuracy of the monitoring results.

[0033] An embodiment of the present invention provides a method for monitoring road cracks, which monitors target cracks on a target road where cracks exist by identifying cracks in a captured target image, thereby improving the efficiency and accuracy of road crack monitoring.

[0034] According to an embodiment of the present invention, an embodiment of a method for monitoring pavement cracks is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] In this embodiment, a method for monitoring road cracks is provided, which can be used for unmanned equipment equipped with a server. Figure 1 FIG. 1 is a flow chart of a method for monitoring pavement cracks according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0036] Step S101 , obtaining environmental information of a target area, and selecting a first shooting parameter according to the environmental information; the target area is an area including a road to be monitored.

[0037] Environmental information refers to data related to the roads to be monitored in the target area and their surrounding environment. For example, this environmental information includes weather conditions, current location, wind force, wind speed, and time. First shooting parameters are various parameters used to control the operating state and imaging quality of the unmanned equipment during shooting operations. For example, these first shooting parameters include shooting altitude, shooting frequency, and imaging mode. In embodiments of the present invention, the number of roads to be monitored in the target area can be one or more.

[0038] In some optional implementations, the unmanned device is a device that performs specific tasks without on-site operator operation through remote control, automatic control, or preset programs. In embodiments of the present invention, the unmanned device may be a drone, an unmanned vehicle with a camera function, or the like.

[0039] In some optional embodiments, a first shooting parameter corresponding to the environmental information is selected based on the environmental information. For example, when the light intensity in the environmental information is high, a lower shooting height, a smaller aperture, a higher shutter speed, etc. are selected.

[0040] Step S102 : photographing the road to be monitored in the target area according to the first photographing parameter to obtain a target image.

[0041] In some optional embodiments, the road to be monitored in the target area is photographed according to the first shooting parameter to obtain a target image, including: setting the first shooting parameter to a hyperspectral camera module of the unmanned equipment, controlling the hyperspectral camera module to photograph the road to be monitored in the target area, and obtaining the target image.

[0042] In some optional embodiments, after photographing the road to be monitored in the target area according to the first shooting parameter and obtaining the target image, the method for monitoring road cracks further includes: inputting the target image into a trained deep convolutional neural network to perform noise reduction and image enhancement processing on the target image, and the target image for crack identification is the image after noise reduction and image enhancement processing.

[0043] In some optional embodiments, the deep convolutional neural network can be DnCNN (Denoising Convolutional Neural Network, denoising convolutional neural network), which adopts a deep convolutional neural network architecture, including multiple convolutional layers, batch normalization layers and ReLU (Rectified Linear Unit) activation functions. DnCNN adopts a residual learning strategy, and the network learns the noise instead of the denoised image. DnCNN measures the difference between the estimated noise image and the real noise image by the root mean square error, thereby removing noise.

[0044] In embodiments of the present invention, unmanned equipment may be affected by external factors such as wind and vibration during operation, resulting in noise in the captured target image. Road surface stains, debris, and lighting conditions can all reduce the clarity and quality of the target image, affecting the precise identification of road cracks. Embodiments of the present invention utilize DnCNN to process target images for noise reduction and image enhancement. This overcomes issues such as image blur, color distortion, and difficulty identifying cracks caused by insufficient imaging conditions and complex road conditions, facilitating the creation of high-definition, high-resolution images with clear target boundaries.

[0045] Step S103: performing crack recognition on the target image to obtain a recognition result.

[0046] In some optional embodiments, crack recognition is performed on a target image to obtain a recognition result, including: extracting features from the target image to obtain multiple feature maps; determining spatial position weights of the multiple feature maps, and fusing the multiple feature maps based on the spatial position weights to obtain multiple target feature maps; inputting the multiple target feature maps into a trained road crack extraction model to obtain a recognition result; the input of the road crack extraction model is the target feature map, and the output of the road crack extraction model is the recognition result.

[0047] In some optional embodiments, a three-dimensional convolutional neural network (3D CNN) is used to extract features of the target image to obtain multiple feature maps; specifically, the target image is input into the 3D CNN to obtain multiple feature maps.

[0048] In some optional embodiments, a channel attention mechanism and a spatial attention mechanism are used to fuse features of multiple feature maps in the channel dimension to obtain multiple target feature maps; specifically, the spatial position weights of the multiple feature maps are calculated, and the multiple feature maps are weighted and summed according to the spatial position weights to obtain multiple target feature maps; wherein, the channel attention mechanism is a mechanism that highlights or suppresses the feature information contained in different channels by analyzing and weighting each channel of the feature map; the spatial attention mechanism is a mechanism for focusing on the importance of different spatial positions of the input feature map.

[0049] In some optional embodiments, multiple target feature maps are input into a trained road crack extraction model to obtain recognition results. Specifically, the recognition results include three situations: the presence of cracks, the absence of cracks, and uncertainty. When the recognition result is the presence of cracks, the recognition result includes the bounding box and category of the cracks.

[0050] In some optional embodiments, after crack recognition is performed on the target image and recognition results are obtained, the pavement crack monitoring method further includes: formatting the recognition results and environmental information, establishing a directory structure based on the recognition results and environmental information after format conversion; and storing the recognition results and environmental information in the directory structure.

[0051] The recognition result and the environment information are format-converted to obtain the recognition result and the environment information in a preset format. The preset format may be JSON or XML format.

[0052] In some optional embodiments, a directory structure is established based on the recognition results and environmental information after format conversion, wherein the directory structure is a structure in which the supercomputing board module processes the environmental information, identifies the cracks, and hierarchically displays the environmental information, recognition results, and the relationship between the two. Exemplarily, the directory structure can be: first-level directory: latitude, longitude, and altitude; second-level sub-directory: original image; third-level sub-directory: recognition results.

[0053] The present invention converts the format of recognition results and environmental information to ensure compatibility with different systems and tools, facilitates reading, parsing, and processing by different programming languages ​​and platforms, and achieves data standardization. The present invention establishes a directory structure that can organize the recognition results and environmental information after format conversion in a logical and clear manner. When searching for specific data, users can quickly locate the corresponding directory location, greatly improving the efficiency of data retrieval. Compared with the storage method of conventional storage structures, it avoids the trouble of blindly searching through large amounts of data and saves time.

[0054] Step S104: judging whether there are cracks on the road to be monitored based on the recognition result.

[0055] In some optional embodiments, when the recognition result is that cracks exist, it is judged that cracks exist on the road to be monitored; when the recognition result is that cracks do not exist, it is judged that cracks do not exist on the road to be monitored; when the recognition result is uncertain, it is judged that it is uncertain whether cracks exist on the road to be monitored and further identification is required.

[0056] Step S105 : If cracks exist on the road to be monitored, target cracks on the target road with cracks are monitored.

[0057] If no cracks exist on the road to be monitored, first prompt information is generated, and the first prompt information is used to prompt that no cracks exist.

[0058] In some optional embodiments, target cracks on a target road with cracks are monitored, including: using a path planning algorithm (ACO-YOLO Path Planner, AYPP) that integrates an ant colony algorithm and a target detection algorithm to perform path planning to obtain a target path, and focusing and photographing the target road at a target shooting point according to a second shooting parameter based on the target path to monitor target cracks on the target road.

[0059] Specifically, based on the position of the target road, with the goal of maximizing the accumulated reward, a preliminary path is determined according to a preset reward function; the preliminary path is optimized according to the pheromone concentration and heuristic information to obtain the target path; the pheromone concentration is a variable used to characterize the attractiveness of the path in the ant colony algorithm, and the heuristic information is the information used for path selection in the ant colony algorithm; according to the target path, the target road is focused and photographed at the target shooting point according to the second shooting parameter to monitor the target cracks on the target road; the second shooting parameter is the shooting parameter obtained after the precision adjustment of the first shooting parameter.

[0060] Exemplarily, the second shooting parameter is obtained by adjusting the first shooting parameter in multiple aspects, such as height reduction adjustment, aperture reduction adjustment, and focus mode selection.

[0061] The present embodiment provides a method for monitoring road cracks, which obtains environmental information and selects a first shooting parameter based on the environmental information. Since different environmental conditions can affect the shooting effect, the present invention obtains environmental information to specifically select shooting parameters, ensuring the quality stability of the target image subsequently obtained, improving the adaptability of the present invention, and avoiding the problem of repeated shooting caused by using fixed parameters. By selecting shooting parameters, the efficiency of subsequent shooting is improved and the waste of storage resources is reduced. The present embodiment of the present invention shoots the road to be monitored in the target area according to the first shooting parameter to obtain a target image. The target image obtained by the present embodiment based on the first shooting parameter adapted to the environment can truly reflect the actual situation of the road. The present embodiment of the present invention performs crack identification on the target image to obtain an identification result. Based on the identification result, it is determined whether cracks exist in the road to be monitored. Based on the presence of cracks in the road to be monitored, target cracks are monitored on the target road with cracks. Compared with related technologies, the present embodiment of the present invention is not affected by human factors and improves the efficiency and accuracy of road crack identification and monitoring. By monitoring the target cracks, the present embodiment of the present invention can timely detect changes in the target cracks so that preventive measures can be taken to ensure traffic safety.

[0062] In this embodiment, a method for monitoring road cracks is provided, which can be used for unmanned equipment equipped with a server. Figure 2 FIG. 1 is a flow chart of another method for monitoring pavement cracks according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0063] Step S201: Obtain environmental information of the target area and select a first shooting parameter based on the environmental information; the target area is the area including the road to be monitored. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0064] Step S202: photograph the road to be monitored in the target area according to the first photographing parameter to obtain a target image. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0065] Step S203: performing crack recognition on the target image to obtain a recognition result.

[0066] Specifically, the above step S203 includes:

[0067] Step S2031: extract features from the target image to obtain multiple feature maps.

[0068] Among them, the target image is input into the 3D CNN to obtain multiple feature maps.

[0069] Step S2032: determine the spatial position weights of the multiple feature maps, and fuse the multiple feature maps according to the spatial position weights to obtain multiple target feature maps.

[0070] The spatial position weights of the multiple feature maps are calculated, and the multiple feature maps are weighted and summed according to the spatial position weights to obtain multiple target feature maps.

[0071] Step S2033: Input multiple target feature maps into the trained road crack extraction model to obtain a recognition result; the input of the road crack extraction model is the target feature map, and the output of the road crack extraction model is the recognition result.

[0072] For example, Figure 3 The figure shows a schematic diagram of the operation process of the road crack extraction model. The road crack extraction model uses a deep separable convolutional neural network and a residual network. The target image composed of visible light images and thermal infrared images obtained by the hyperspectral camera module is processed by the deep separable convolutional neural network and the residual network to obtain multiple feature maps. The multiple feature maps are then fused through the channel attention mechanism and the spatial attention mechanism to obtain multiple target feature maps. Finally, the multiple target feature maps are input into the trained road crack extraction model to obtain the recognition results. Among them, the channel attention mechanism is a mechanism that analyzes and weights each channel of the feature map to highlight or suppress the feature information contained in different channels; the spatial attention mechanism is a mechanism used to focus on the importance of different spatial positions of the input feature map.

[0073] In some optional embodiments, the pavement crack monitoring method also includes a training process for a road crack extraction model; the training process for the road crack extraction model includes: obtaining a road crack target sample library, labeling multiple target samples in the road crack target sample library to obtain multiple target labeled samples; and training the road crack extraction model based on the multiple target labeled samples.

[0074] Among them, the road crack target sample library contains multiple target samples of road cracks of various shapes, types, colors, etc. Multiple target samples are used to train the road crack extraction model. During the training process of the road crack extraction model, parameters are continuously adjusted to improve the road crack extraction model's ability to identify and extract road cracks.

[0075] In some optional implementations, data augmentation techniques, such as random cropping, rotation, and scaling, are used during training to improve the generalization capabilities of the road crack extraction model. For target samples with higher recognition accuracy, the road crack target recognition sample library is rewarded accordingly based on their contribution to improving overall recognition performance. This reward mechanism may include, but is not limited to, increasing the weight of the target sample so that it is sampled more frequently in future training, or marking samples with significant contributions so that researchers can more easily identify and analyze these high-quality samples. When the recognition accuracy of certain target samples is low, a penalty mechanism is introduced to reduce the weight of the target samples with lower recognition accuracy during training, reducing their frequency of use in subsequent iterations.

[0076] After training, the road crack extraction model is used to identify cracks, rapidly processing large numbers of target images to initially identify and annotate potential road cracks. The identification results are then added to the road crack target sample library for the next round of road crack extraction model training. This iterative process continuously expands and optimizes the road crack target sample library, while also continuously improving the performance of the road crack extraction model.

[0077] In some optional implementations, the road crack extraction model can also be used to identify road cracks in video streams and images, for example, by using the road crack extraction model to map a user-defined base map.

[0078] The present invention extracts features from a target image to generate multiple feature maps. Each feature map can capture different aspects of the image's features, capturing these multiple features. This allows for analysis of the image from multiple dimensions, providing a more comprehensive description of the target image's features and providing a rich data foundation for accurate road crack identification. The present invention considers the varying importance of different features in spatial location, highlighting key spatial location information for crack identification while suppressing irrelevant or interfering information. Through weighted fusion, features of key regions are more fully reflected in the fused target feature map, further improving the accuracy of subsequent crack identification. The present invention inputs the target feature map into a trained road crack extraction model to generate identification results. After training with a large amount of data, the road crack extraction model learns the characteristic patterns and patterns of road cracks. It can quickly and accurately analyze and determine the presence of cracks and related information within the input target feature map, significantly improving the efficiency and accuracy of crack identification and reducing the workload and errors of manual identification. The present invention labels multiple target samples in a road crack target sample library to obtain multiple labeled target samples. The road crack extraction model is trained based on these multiple labeled target samples, improving the recognition efficiency of the road crack extraction model.

[0079] Step S204: Based on the recognition result, determine whether there are cracks on the road to be monitored. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0080] Step S205 : If cracks exist on the road to be monitored, target cracks on the target road with cracks are monitored.

[0081] Specifically, the above step S205 includes:

[0082] Step S2051 : Based on the location of the target road and with the goal of maximizing the accumulated reward, a preliminary path is determined according to a preset reward function.

[0083] Among them, the Deep Reinforcement Learning (DRL) algorithm is used to maximize the accumulated rewards and determine the preliminary path according to the preset reward function. The reward function setting includes path efficiency reward, safety reward and monitoring efficiency reward; the path efficiency reward encourages the selection of the shortest path, the safety reward encourages the avoidance of obstacles and dangerous areas, and the monitoring efficiency reward encourages the selection of a path that can effectively monitor the road conditions.

[0084] Step S2052: Optimize the preliminary path based on the pheromone concentration and heuristic information to obtain the target path; the pheromone concentration is a variable used to characterize the attractiveness of the path in the ant colony algorithm, and the heuristic information is information used for path selection in the ant colony algorithm.

[0085] The ant colony algorithm is used to simulate the behavior of ants in searching for food, and the pheromone concentration and heuristic information are used to gradually construct the optimal path to optimize the preliminary path and obtain the target path.

[0086] Step S2053 , focusing and photographing the target road at the target photographing point according to the target path according to the second photographing parameter, so as to monitor the target cracks on the target road; the second photographing parameter is a photographing parameter obtained by precision-adjusting the first photographing parameter.

[0087] The pavement crack monitoring method provided in this embodiment aims to maximize the accumulated reward, and determines a preliminary path according to a preset reward function to ensure that the path planning revolves around the core needs of crack monitoring, so that the preliminary path is more in line with the actual application scenario. The present invention optimizes the preliminary path according to the pheromone concentration and the heuristic information to obtain the target path, ensures that the target path adapts to changes in the environment, and ensures the rationality and effectiveness of the target path. The present invention focuses on shooting the target road according to the second shooting parameter at the target shooting point according to the target path to monitor the target cracks on the target road, and reaches the target shooting point according to the target path to ensure that the target cracks on the target road are accurately positioned and shot. The second shooting parameter is obtained after the precision adjustment of the first shooting parameter. For the specific task of crack monitoring, the shooting parameter is optimized, which helps to obtain more accurate crack feature information. Shooting is performed according to the planned target path and precise shooting parameters. While meeting the crack monitoring accuracy requirements, unnecessary shooting resource waste is avoided, and efficient resource utilization is achieved.

[0088] In this embodiment, a road crack monitoring system is provided. Figure 4 FIG. 1 is a schematic structural diagram of a road crack monitoring system according to an embodiment of the present invention. Figure 4 As shown in the figure, the system includes a mobile terminal, an unmanned device equipped with a supercomputing board module, a hyperspectral camera module, and a communication module, as well as a backend server. Figure 5 The following is a schematic diagram of the workflow of the pavement crack monitoring system. The workflow of the system includes:

[0089] Step S501: The mobile terminal is used to send the pavement crack monitoring instruction input by the user to the unmanned equipment.

[0090] Step S502: The supercomputing board module is used to obtain environmental information of the target area based on the road crack monitoring instruction and select a first shooting parameter according to the environmental information; the target area is the area including the road to be monitored. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0091] Step S503: The hyperspectral camera module is used to shoot the road to be monitored in the target area according to the first shooting parameter to obtain a target image. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0092] Step S504: The supercomputing board module is used to identify cracks in the target image and obtain the identification result. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0093] Step S505: The supercomputing board module is used to determine whether there are cracks on the road to be monitored based on the recognition results. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0094] Step S506: The hyperspectral camera module is used to monitor target cracks on the target road where cracks exist, based on the presence of cracks on the road to be monitored. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0095] Step S507: The communication module is used to send the monitoring results to the backend server.

[0096] Step S508: The backend server is used to convert the monitoring results into a preset vector layer, and to visualize the preset vector layer on a preset map.

[0097] The preset vector layer can be an OpenStreetMap vector layer, and the preset map can be a Tiantu map. In an embodiment of the present invention, the detection results are visualized on the Tiantu map base map as an OpenStreetMap vector layer, with dangerous roads highlighted in red to alert users. At the same time, statistical attributes of the vector layer (such as shooting time and crack area) are added to the vector layer's fields. A user click interaction event is also designed. By clicking on the crack layer of interest, the road crack is highlighted, and its attributes are displayed using an information box.

[0098] Step S509: the backend server is configured to send the preset map including the preset vector layer to the mobile terminal.

[0099] Step S5010: The mobile terminal is used to display a preset map including a preset vector layer.

[0100] Among them, the mobile terminal can realize the download function of the vector layer.

[0101] In some optional implementations, the hyperspectral camera module is further configured to perform photography according to photography parameters provided by the mobile terminal.

[0102] In an embodiment of the present invention, users can filter images of interesting times and areas of interest in a mobile terminal according to shooting date, designated administrative area, free-drawn area, etc., and can observe the bird's-eye view of the target road to promptly discover roads with damage or cracks.

[0103] The pavement crack monitoring system of this embodiment visualizes monitoring results, clearly showing the location, direction, and distribution of cracks in geographic space. This helps identify potential patterns and trends in crack distribution, providing accurate data support for road maintenance and repair. This allows personnel to quickly and accurately understand crack conditions and take timely repair measures, thereby improving road safety and service life.

[0104] In this embodiment, a road crack monitoring system is provided. Figure 6 FIG. 1 is a flowchart of the unmanned equipment pavement crack monitoring system according to an embodiment of the present invention. Figure 6 As shown in the figure, the system includes: unmanned equipment equipped with a hyperspectral camera module and a supercomputing board module, and an urban road intelligent management system. The specific workflow is as follows:

[0105] The supercomputing board module is used to obtain environmental information of the target area based on the road crack monitoring instruction and select the first shooting parameter according to the environmental information.

[0106] The hyperspectral camera module is used to shoot the road to be monitored in the target area according to the first shooting parameter to obtain a target image.

[0107] The storage unit in the supercomputing board module is used to store the target image.

[0108] The preprocessing unit in the supercomputing board module is used to preprocess the target image.

[0109] The intelligent interpretation unit in the supercomputing board module is used to identify cracks in the target image and obtain the recognition results.

[0110] The intelligent path optimization unit in the supercomputing board module is used to plan the path based on the recognition results to obtain the target path, and control the hyperspectral camera module of the unmanned equipment according to the target path to monitor the target cracks on the target road where cracks exist.

[0111] Urban road intelligent management system, used to visualize target images and recognition results.

[0112] In some optional implementations, the storage unit is further configured to store recognition results and environmental information in a preset format.

[0113] In some optional embodiments, the communication unit in the supercomputing board module is used to send the recognition results and environmental information in a preset format stored in the storage unit to a road state fine recognition library. The road state fine recognition library also stores target images including visible light images and thermal infrared images. The road state fine recognition library is also used to transmit the stored content to a road crack target sample library. The road crack target sample library is used to train the road crack extraction model of the intelligent interpretation unit.

[0114] In an embodiment of the present invention, the unmanned vehicle also includes an operation control module, which is used to control the operation of the unmanned vehicle according to the target path, thereby enabling command and dispatch of the unmanned vehicle. The operation control module first obtains the initial posture and speed captured by the unmanned vehicle as initial values, and then adjusts the operating posture and speed in real time to respond to environmental changes and achieve real-time road monitoring.

[0115] In an embodiment of the present invention, the road state fine recognition database adopts a file system storage method to establish a corresponding directory structure for storing target images captured, processed, and extracted by the road crack intelligent acquisition and interpretation system, as well as XML and TXT files corresponding to the target images, and recording the paths of the corresponding files.

[0116] In an embodiment of the present invention, the urban road intelligent management system also includes a mobile terminal and a back-end server. The back-end server is used to receive and transmit road crack monitoring instructions sent by the mobile terminal, and to send the instructions to the drone device so that the drone device executes the road crack monitoring instructions and feeds back the monitoring results to the back-end server. The back-end server then returns the monitoring results to the mobile terminal for display.

[0117] The backend server communicates with the unmanned device via the Micro Air Vehicle Link (MAVLink) protocol. The mobile terminal generates control instructions (such as takeoff, landing, hovering, route, and optimal route) based on user-entered or pre-set operational plans, encapsulating them in JSON format and sending them to the backend server. The backend server encodes the control instructions into the MAVLink protocol format and sends them to the unmanned device via a communication unit. The unmanned device's operational control module then controls the flight trajectory and attitude of the unmanned device based on the parsed instructions.

[0118] This embodiment also provides a pavement crack monitoring device for implementing the aforementioned embodiments and preferred implementations. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. While the devices described in the following embodiments are preferably implemented using software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0119] This embodiment provides a monitoring device for road cracks, such as Figure 7 Shown, including:

[0120] The first shooting parameter selection unit 701 is used to obtain environmental information of a target area and select a first shooting parameter according to the environmental information; the target area is an area including the road to be monitored.

[0121] The target image determining unit 702 is configured to photograph the road to be monitored in the target area according to the first photographing parameter to obtain a target image.

[0122] The intelligent interpretation unit 703 is used to perform crack recognition on the target image and obtain a recognition result.

[0123] The crack determination unit 704 is configured to determine whether cracks exist on the road to be monitored based on the recognition result.

[0124] The crack monitoring unit 705 is configured to monitor target cracks on a target road where cracks exist, based on the presence of cracks on the road to be monitored.

[0125] In some optional implementations, the intelligent interpretation unit 703 includes:

[0126] The feature extraction subunit is used to extract features from the target image and obtain multiple feature maps.

[0127] The feature map fusion subunit is used to determine the spatial position weights of multiple feature maps, and fuse the multiple feature maps according to the spatial position weights to obtain multiple target feature maps.

[0128] The recognition result determination subunit is used to input multiple target feature maps into the trained road crack extraction model to obtain the recognition result; the input of the road crack extraction model is the target feature map, and the output of the road crack extraction model is the recognition result.

[0129] In some optional embodiments, the crack monitoring unit 705 includes:

[0130] The preliminary path determination subunit is used to determine the preliminary path based on the location of the target road, with the goal of maximizing the accumulated reward and according to a preset reward function.

[0131] The target path determination subunit is used to optimize the preliminary path according to the pheromone concentration and heuristic information to obtain the target path; the pheromone concentration is the variable used to characterize the path attractiveness in the ant colony algorithm, and the heuristic information is the information used for path selection in the ant colony algorithm.

[0132] The crack monitoring subunit is used to focus and shoot the target road according to the target path at the target shooting point according to the second shooting parameter to monitor the target cracks on the target road; the second shooting parameter is the shooting parameter obtained by precision adjustment of the first shooting parameter.

[0133] In some optional embodiments, the pavement crack monitoring device further includes:

[0134] The road crack extraction model training unit is used to obtain a road crack target sample library, label multiple target samples in the road crack target sample library, and obtain multiple target labeling samples; and train the road crack extraction model according to the multiple target labeling samples.

[0135] In some optional embodiments, the pavement crack monitoring device further includes:

[0136] The preprocessing unit is used to input the target image into the trained deep convolutional neural network to perform noise reduction and image enhancement on the target image. The target image for crack identification is the image after noise reduction and image enhancement.

[0137] In some optional embodiments, the pavement crack monitoring device further includes:

[0138] The information storage unit is used to convert the format of the recognition results and environmental information, establish a directory structure based on the recognition results and environmental information after the format conversion, and store the recognition results and environmental information in the directory structure.

[0139] The further functional description of each of the above units and sub-units is the same as that of the above corresponding embodiments and will not be repeated here.

[0140] The pavement crack monitoring device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0141] The embodiment of the present invention also provides a drone having the above Figure 7 The monitoring device for road cracks shown.

[0142] See also Figure 8 , Figure 8 : is a schematic structural diagram of a drone provided by an optional embodiment of the present invention, such as Figure 8As shown, the drone includes: one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the drone, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used with multiple memories and multiple memories. Similarly, multiple drones can be connected, with each device providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0143] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0144] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0145] The memory 20 may include a program storage area and a data storage area. The program storage area may store an operating system and applications required for at least one function; the data storage area may store data generated during the use of the drone, etc. Furthermore, the memory 20 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some optional embodiments, the memory 20 may optionally include a memory located remotely from the processor 10, and such remote memory may be connected to the drone via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0146] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0147] The drone also includes a communication interface 30 for the drone to communicate with other devices or communication networks.

[0148] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0149] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0150] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for monitoring road cracks, characterized in that: The method comprises: Acquiring environmental information of a target area, and selecting a first shooting parameter according to the environmental information; the target area is an area including a road to be monitored; photographing the road to be monitored in the target area according to the first photographing parameter to obtain a target image; Performing crack recognition on the target image to obtain a recognition result; Determining whether there are cracks on the road to be monitored based on the recognition result; If cracks exist on the road to be monitored, target cracks on the target road with cracks are monitored.

2. The method according to claim 1, characterized in that The monitoring of target cracks on a target road having cracks includes: Based on the location of the target road, determining a preliminary path according to a preset reward function with the goal of maximizing the accumulated reward; The preliminary path is optimized according to the pheromone concentration and heuristic information to obtain a target path; the pheromone concentration is a variable used to characterize the attractiveness of the path in the ant colony algorithm, and the heuristic information is information used for path selection in the ant colony algorithm; The target road is focused and photographed at a target photographing point according to the target path according to a second photographing parameter to monitor the target cracks on the target road; the second photographing parameter is a photographing parameter obtained by precision adjustment of the first photographing parameter.

3. The method according to claim 1 or 2, characterized in that The performing crack recognition on the target image to obtain a recognition result includes: Performing feature extraction on the target image to obtain multiple feature maps; Determining spatial position weights of the multiple feature maps, and fusing the multiple feature maps according to the spatial position weights to obtain multiple target feature maps; Inputting the plurality of target feature maps into a trained road crack extraction model to obtain the recognition result; the input of the road crack extraction model is the target feature map, and the output of the road crack extraction model is the recognition result; The method further includes a training process for the road crack extraction model; the training process for the road crack extraction model includes: Obtaining a road crack target sample library, and labeling a plurality of target samples in the road crack target sample library to obtain a plurality of target labeled samples; The road crack extraction model is trained according to the multiple target labeled samples.

4. The method according to claim 1 or 2, characterized in that After photographing the road to be monitored in the target area according to the first photographing parameter to obtain a target image, the method further includes: The target image is input into a trained deep convolutional neural network to perform noise reduction and image enhancement on the target image. The target image for crack identification is an image after noise reduction and image enhancement.

5. The method according to claim 1 or 2, characterized in that After performing crack recognition on the target image and obtaining a recognition result, the method further includes: Performing format conversion on the recognition result and the environmental information, and establishing a directory structure according to the recognition result and the environmental information after the format conversion; The recognition result of the directory structure and the environment information are stored.

6. A road crack monitoring system, characterized in that: The system includes a mobile terminal, an unmanned device equipped with a supercomputing board module, a hyperspectral camera module, and a communication module, and a backend server; The mobile terminal is used to send the pavement crack monitoring instruction input by the user to the unmanned equipment; The supercomputing board module is configured to obtain environmental information of a target area based on the pavement crack monitoring instruction and select a first shooting parameter according to the environmental information; the target area is an area including the road to be monitored; The hyperspectral camera module is configured to photograph the road to be monitored in the target area according to the first photographing parameters to obtain a target image; The supercomputing board module is used to perform crack recognition on the target image to obtain a recognition result; The supercomputing board module is used to determine whether there are cracks on the road to be monitored based on the recognition result; The hyperspectral camera module is used to monitor target cracks on a target road having cracks according to the presence of cracks on the road to be monitored; The communication module is used to send the monitoring results to the back-end server; The backend server is configured to convert the monitoring results into a preset vector layer, and to visualize the preset vector layer on a preset map; The backend server is configured to send the preset map including the preset vector layer to the mobile terminal; The mobile terminal is used to display the preset map including the preset vector layer.

7. A monitoring device for road cracks, characterized in that: The device comprises: A first shooting parameter selection unit is configured to obtain environmental information of a target area and select a first shooting parameter based on the environmental information; the target area is an area including the road to be monitored; a target image determining unit, configured to photograph the road to be monitored in the target area according to the first photographing parameter to obtain a target image; An intelligent interpretation unit, configured to perform crack recognition on the target image and obtain a recognition result; a crack judging unit, configured to judge whether cracks exist on the road to be monitored according to the recognition result; The crack monitoring unit is used to monitor target cracks on a target road where cracks exist according to the presence of cracks on the road to be monitored.

8. A drone, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the pavement crack monitoring method according to any one of claims 1 to 5 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the pavement crack monitoring method according to any one of claims 1 to 5.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method for monitoring pavement cracks according to any one of claims 1 to 5.

Images