A road disease identification method based on AI and multi-scale fusion

Abstract

The present application relates to the technical field of road disease identification, and relates to a road disease identification method based on AI and multi-scale fusion. The present application adjusts the image acquisition width of the bottom-mounted platform through the actual width of the road, collects global images covering the road surface in real time, evaluates whether the global image quality is qualified based on the feature data of the global image, screens abnormal areas with unqualified quality, collects detail images of the abnormal areas again, identifies whether the abnormal area is a crack disease based on the abnormal morphology data set extracted from the detail images, obtains the depth value and extension length of the crack when it is a crack disease, analyzes the actual coverage contour of the crack based on the constructed three-dimensional space model of the crack, generates a road disease detection report, realizes accurate determination of the crack type, reduces the misjudgment probability of non-crack diseases, provides comprehensive and accurate data support for crack expansion trend prediction, helps to develop targeted repair schemes, and reduces maintenance costs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road disease identification, and relates to a road disease identification method based on AI and multi-scale fusion. BACKGROUND

[0002] With the continuous growth of traffic flow and the extension of service life, various diseases such as cracks and pits may occur on the road surface. Cracks are the most common and serious disease type of road, and as an early form of road surface structure damage, cracks can directly damage the integrity of the road surface. If not timely and accurately identified and disposed, it will affect the service life of the road and cause traffic safety hazards. Therefore, efficient and accurate identification of crack diseases is a key link in road maintenance and management.

[0003] The prior art such as Chinese patent publication No. CN114266892A discloses a road disease identification method and system based on multi-source data deep learning. The method uses a vehicle-mounted road inspection vehicle to cruise at a fixed speed, carries a ground penetrating radar and an industrial camera to obtain real-time road inspection RGB images and radar images, processes the RGB images, and uses the YOLO algorithm to analyze the diseases. If it is judged to be a certain disease, the corresponding numbered radar image is queried, the crack depth is calculated based on the CascadeR-CNN algorithm, and finally a road disease report containing disease type, depth and positioning is generated.

[0004] Based on the above prior art, the following problems exist: 1. The prior art uses a vehicle-mounted device to fixedly collect images and radar data, and does not dynamically adjust the collection parameters according to the actual width of the road, which causes problems such as data redundancy in narrow roads and incomplete coverage in wide roads, resulting in low collection efficiency and possible omission of disease information in the edge area, affecting the comprehensiveness of disease identification.

[0005] 2. The prior art directly uses the YOLO algorithm to analyze the collected images for diseases without multi-scale evaluation and abnormal area revalidation of the collected images. If the images have problems such as blur and texture distortion, it will lead to inaccurate disease feature extraction, thereby reducing the accuracy of disease identification and increasing the risk of misjudgment.

[0006] 3. The prior art relies on the corresponding relationship between radar images and RGB images to calculate the crack depth, and does not use a UAV to perform laser radar depth detection, making it difficult to accurately obtain the depth distribution and extension trajectory coverage of the cracks, resulting in incomplete disease data identification and affecting the scientificity of subsequent maintenance scheme development. SUMMARY

[0007] In view of the problems existing in the prior art, the present application provides a road disease identification method based on AI and multi-scale fusion, which realizes efficient and accurate identification of road diseases.

[0008] The technical scheme adopted by the present application to solve its technical problems is: a road disease identification method based on AI and multi-scale fusion, comprising: acquiring the actual width of the current inspection road through a mobile inspection vehicle, dynamically adjusting the image acquisition width of the bottom-mounted platform based on the actual width, and collecting global images covering the road surface in real time.

[0009] Layered feature extraction is performed on the global image to obtain global image feature data, and the global image quality is evaluated in combination with a normal reference image set of the inspection road to screen abnormal areas with unqualified quality.

[0010] According to the contour boundary of the abnormal area, a fixed-point collection instruction is sent to a drone mounted on the mobile inspection vehicle, the drone is controlled to hover above the center point of the abnormal area to collect a detail image, and an abnormal morphology data set in the detail image is extracted.

[0011] The abnormal morphology data set is matched with the reference morphology data range of all preset crack type diseases to identify whether the abnormal area is a crack disease.

[0012] When the abnormal area is a crack disease, the depth value and extension length of the crack are obtained through deep detection of the drone, and a three-dimensional space model of the crack is constructed.

[0013] Based on the three-dimensional space model of the crack, the actual coverage contour of the crack is analyzed, and a road disease detection report is generated.

[0014] Compared with the prior art, the present application has the following beneficial effects: (1) The present application acquires the actual width of the road through a distance sensor, dynamically adjusts the image acquisition width of the bottom-mounted platform, avoids the problem of redundancy or missing collection caused by fixed collection parameters, ensures the complete coverage of the global image on the road surface, accurately captures the fine cracks in the edge area of the road, and improves the comprehensiveness of crack collection.

[0015] (2) The present application performs layered feature extraction on the global image at the pixel level and the region level, analyzes the definition score of the pixel level scale image and the texture feature vector of each local block, screens abnormal areas in combination with a normal reference image set of the inspection road, and then collects detail images through a drone, effectively eliminates the interference of blurred and distorted images, accurately extracts abnormal morphology data of the abnormal area, and significantly improves the accuracy of later crack identification.

[0016] (3) The present application matches the abnormal morphology data set in the detail image with the reference morphology data range of all preset crack type diseases to identify whether the abnormal area is a crack disease and the crack type disease, realizes accurate determination of the crack type, and reduces the misjudgment probability of non-crack diseases.

[0017] (4) The unmanned aerial vehicle laser radar depth detection is used to obtain the crack depth value and extension length, a three-dimensional space model of the crack is constructed, and actual coverage contour is analyzed, comprehensive and accurate data support is provided for crack expansion trend prediction, targeted maintenance scheme is helped to be made, and maintenance cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0019] Figure 1 It is a schematic diagram of the method steps of the present application.

[0020] Figure 2 It is a schematic diagram of the step flow of the abnormal morphology data set extraction in the present application.

[0021] Figure 3 It is a schematic diagram of the step flow of identifying whether the abnormal area is a crack disease in the present application. DETAILED DESCRIPTION

[0022] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments are not limiting to the scope of the present application unless otherwise specifically stated. It should also be understood that the dimensions of the various parts shown in the drawings are not drawn to scale for the sake of convenience of description.

[0023] The following description of at least one example embodiment is merely illustrative in nature and is in no way limiting to the scope of the application and its applications or uses. Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered part of the specification where appropriate.

[0024] In all examples shown and discussed herein, any specific values should be interpreted as merely illustrative and not as limiting. Thus, other examples of the example embodiments can have different values.

[0025] Reference Figure 1 As shown, the present application provides a road disease identification method based on AI and multi-scale fusion, comprising: S1, acquiring the actual width of the current inspection road through a mobile inspection vehicle, dynamically adjusting the image acquisition width of the bottom-mounted platform based on the actual width, and collecting global images covering the road surface in real time.

[0026] In an embodiment of the present application, the method of dynamically adjusting the image acquisition width of the bottom-mounted platform is as follows: first, the distance sensor mounted on the mobile inspection vehicle acquires the spatial distance of the boundaries on both sides of the current inspection road, which is taken as the actual width of the current inspection road.

[0027] Then, the fixed shooting angle range and the acquisition overlap rate of the image acquisition device in the bottom-mounted platform of the mobile inspection vehicle are called, and the effective shooting width of the image acquisition device is determined based on the fixed shooting angle range and the acquisition overlap rate.

[0028] It should be noted that the calculation formula of the effective shooting width of the image acquisition device is as follows: .

[0029] Wherein, represents the effective shooting width of the image acquisition device, represents the acquisition overlap rate, which is set when the image acquisition device is installed, to avoid gaps when multiple devices acquire images and to ensure the continuity of the global image; represents the actual shooting width corresponding to the fixed shooting angle range, which can be derived by a trigonometric function from the installation height of the image acquisition device and the mean value of the fixed shooting angle range of the device lens factory calibration, for example, the installation height of the image acquisition device is 2 meters, the fixed shooting angle range is , and the mean value of the fixed shooting angle range is , then the actual shooting width is .

[0030] Finally, the actual width of the current inspection road is compared with the effective shooting width to obtain the required number of image acquisition devices in the bottom-mounted platform, the image acquisition width of the bottom-mounted platform is determined according to the required number of image acquisition devices, and the current image acquisition width of the bottom-mounted platform is dynamically adjusted.

[0031] As an example, the required number of image acquisition devices is obtained as follows: when the ratio is an integer, the required number is the integer; if the ratio is a non-integer, the required number is the integer part of the non-integer plus 1, to ensure that the acquisition width completely covers the actual width of the road and avoids edge missing.

[0032] The image acquisition width of the bottom-mounted platform is the product of the installation spacing between the image acquisition devices on the bottom-mounted platform and the required number of image acquisition devices.

[0033] After the dynamic adjustment of the image acquisition width is completed, all image acquisition devices are started to acquire real-time road surface images, and finally a global image covering the entire road surface is formed by splicing, providing a complete data basis for subsequent abnormal area screening and disease identification.

[0034] The present application acquires the actual width of the road through the distance sensor, dynamically adjusts the image acquisition width of the bottom-mounted platform, avoids the redundancy or missing problems caused by fixed acquisition parameters, ensures the complete coverage of the global image on the road surface, accurately captures the fine cracks in the edge area of the road, and improves the comprehensiveness of crack collection.

[0035] S2, the global image is subjected to hierarchical feature extraction to obtain global image feature data, and the global image quality is evaluated in combination with a normal reference image set of the inspection road to determine whether the global image quality is qualified, and an abnormal area with unqualified quality is screened.

[0036] Considering that single-scale feature extraction cannot balance image details and overall rules, the present application first performs scale hierarchical processing on the global image, synchronously converts the global image into a pixel-level scale image and a region-level scale image, the pixel-level scale image retains each pixel information of the image, and the image definition is quantified through gradient amplitude analysis; the region-level scale image divides the image into a plurality of local blocks for analyzing the consistency of the overall texture. The two complement each other to comprehensively cover the image quality evaluation dimensions and avoid the limitations of single-scale evaluation.

[0037] In an embodiment of the present application, the evaluation method of whether the global image quality is qualified is as follows: first, the global image is subjected to scale hierarchical processing to obtain a pixel-level scale image and a region-level scale image.

[0038] Second, the pixel-level scale image is subjected to gray processing, a gradient operator is used to convolve the gray-processed image to obtain a horizontal direction gradient map and a vertical direction gradient map, the gradient amplitude of each pixel point is calculated, the pixel points with a gradient amplitude greater than a set gradient amplitude threshold value are selected as effective pixel points, the gradient amplitudes of all effective pixel points are squared and summed to obtain the definition score of the pixel-level scale image.

[0039] As an example, the set gradient amplitude threshold value can be obtained according to historical clear image statistics, specifically: a plurality of historical road clear images are selected from a road historical database, the gradient amplitudes of the plurality of historical road clear images are obtained, and the minimum gradient amplitude is selected as the set gradient amplitude threshold value.

[0040] It should be noted that the above-mentioned gradient operator can be a Sobel operator, and the calculation of the gradient amplitude of each pixel point in the image through the Sobel operator is a prior art means, which will not be described in detail.

[0041] Third, the region-level scale image is subjected to gray processing, the processed region-level scale image is divided into a plurality of non-overlapping local blocks, a plurality of directions containing distance and angle are selected in each local block, a multi-direction gray co-occurrence matrix is calculated, and a texture feature vector of each local block in the region-level scale image is determined.

[0042] The texture feature vector is composed of contrast, energy, entropy, correlation and homogeneity.

[0043] In the fourth step, based on the definition of the pixel-level scale image and the texture feature vector of each local block, the normal reference image set of the inspection road recorded in the road history database is compared to determine whether the global image quality is qualified.

[0044] It should be noted that the above scale layering processing is a dimensional splitting of the global image analysis, which does not change the pixel information of the global image. The scale layering processing is the existing image processing technology, which will not be repeated here.

[0045] In an embodiment of the present application, the way to determine whether the global image quality is qualified is to extract each reference image in the normal reference image set of the inspection road, obtain the reference definition score and the reference texture feature vector of the corresponding image based on each reference image, and screen and count the minimum reference definition score and the reference texture feature range.

[0046] Each reference texture feature range is a range composed of the maximum value and the minimum value selected from the reference texture feature vectors of all reference images.

[0047] If the definition score of the pixel-level scale image is less than the minimum reference definition score, or the texture feature value in the texture feature vector of a certain local block is outside the corresponding reference texture feature range, it is determined that the global image quality is unqualified, otherwise it is determined that the global image quality is qualified.

[0048] It should be noted that the normal reference image set of the inspection road needs to be supplemented with new disease-free images regularly to avoid changes in texture features caused by road aging, which affects the accuracy of the reference benchmark. After the inspection road is maintained and repaired, the normal reference image set of the road needs to be reconstructed to ensure that the reference benchmark matches the current road condition.

[0049] The abnormal area screening content is as follows: first, all pixel points with gradient amplitude less than a set gradient amplitude threshold are screened, and a circumscribed polygon containing all pixel points is circled based on the positions of all pixel points, which is recorded as the corresponding suspected abnormal area of the pixel-level scale image.

[0050] Secondly, the local block whose texture feature value in the texture feature vector is outside the reference texture feature range is recorded as a candidate local block.

[0051] Then, the Euclidean deviation distance of the texture feature of all candidate local blocks from the average of the corresponding reference texture feature range is counted, and the candidate local block with the Euclidean deviation distance greater than the reference Euclidean deviation distance is screened, and the regions are merged to obtain the corresponding suspected abnormal area of the regional-level scale image.

[0052] Finally, the pixel-level scale image and the region-level scale image are overlapped and compared to the suspected abnormal region, and the overlapping region is taken as the final abnormal region.

[0053] As an example, the reference Euclidean deviation distance is set by: removing all candidate local blocks from the region-level scale image to obtain the remaining local blocks, obtaining the Euclidean deviation distance between the texture feature of the remaining local block and the range mean of the corresponding reference texture feature, and screening the maximum Euclidean distance as the reference Euclidean deviation distance.

[0054] S3, according to the contour boundary of the abnormal region, a point collection instruction is sent to the unmanned aerial vehicle carried by the mobile inspection vehicle, the unmanned aerial vehicle is controlled to hover above the center point of the abnormal region to collect a detail image, and an abnormal morphology data set in the detail image is extracted.

[0055] In an embodiment of the present application, the diagonal intersection method is used to obtain the center point coordinates of the contour boundary of the abnormal region, and the center point coordinates of the contour boundary are converted into actual geographic coordinates in combination with the real-time GPS position of the mobile inspection vehicle, and the actual geographic coordinates are taken as the point collection hovering position of the unmanned aerial vehicle.

[0056] Considering that the mobile inspection vehicle collects images by uniform speed movement, if the images have problems such as blur and texture distortion, it may not be possible to clearly capture the feature data of the abnormal region, while the unmanned aerial vehicle has the advantages of flexible hovering and close-range shooting, and can obtain high-resolution images with vertical angles above the abnormal region, and can clearly obtain abnormal information.

[0057] Based on this, as shown in the figure, Figure 2 The abnormal morphology data set in the detail image includes: S31, performing noise reduction processing and image enhancement processing on the collected detail image, and extracting the abnormal edge contour of the processed detail image by using an image edge algorithm.

[0058] The noise reduction processing, image enhancement processing and image edge algorithm in the above are all prior art, and will not be described in detail.

[0059] S32, reading the contour surrounding area, contour boundary perimeter, longest straight line distance and minimum circumscribed rectangle corresponding to the abnormal edge contour.

[0060] Considering that the contour surrounding area, contour boundary perimeter, longest straight line distance and minimum circumscribed rectangle are all basic contour parameters, which can only reflect the original size and cannot be directly used for crack type matching, for example, different types of cracks may have similar areas, but the morphological complexity difference is significant, and the basic parameters need to be converted into dimensionless morphological features.

[0061] S33, the contour complexity and the shape compactness are obtained by comparing and calculating the contour enclosed area and the contour boundary perimeter, and the contour bending degree and the circumscribed rectangle length-width ratio are determined according to the longest straight line distance and the minimum circumscribed rectangle.

[0062] S34, the contour complexity, the shape compactness, the contour bending degree and the circumscribed rectangle length-width ratio constitute the abnormal shape data set in the detail image.

[0063] It should be noted that the contour complexity reflects the irregularity of the contour, and the acquisition method is the ratio of the square of the contour boundary perimeter to the contour enclosed area. The greater the contour complexity, the more irregular the contour.

[0064] The shape compactness reflects the compactness of the contour, and the existing formula is: , represents the shape compactness, represents the circumference, represents the contour enclosed area, represents the contour boundary perimeter, the more irregular the contour, the smaller the shape compactness, for example, the shape compactness of the circular contour is the largest.

[0065] The contour bending degree reflects the bending degree of the contour, and the acquisition method is the ratio of the longest straight line distance to the corresponding long side of the minimum circumscribed rectangle. The closer the contour bending degree to 1, the straighter the contour, and the smaller the contour bending degree, the more curved the contour.

[0066] The circumscribed rectangle length-width ratio reflects the length-width difference of the contour, and the acquisition method is the ratio of the long side to the short side of the minimum circumscribed rectangle. The larger the circumscribed rectangle length-width ratio, the more elongated the contour, and the closer the circumscribed rectangle length-width ratio to 1, the closer the contour to the circular shape.

[0067] The present application performs pixel-level and region-level hierarchical feature extraction on global images, analyzes the definition score of the pixel-level scale image and the texture feature vector of each local block, combines the normal reference image set of the inspection road to filter the abnormal area, and then collects the detail image through the unmanned aerial vehicle, effectively eliminates the interference of blurred and distorted images, accurately extracts the abnormal shape data of the abnormal area, and significantly improves the accuracy of the later crack identification.

[0068] S4, the abnormal shape data set is matched with the reference shape data range of all preset crack type diseases, and whether the abnormal area is a crack disease is identified.

[0069] It should be noted that the road crack type disease includes but is not limited to transverse crack disease, longitudinal crack disease, network crack disease and crack disease, and each road crack type disease has obvious morphological feature difference, for example, the transverse crack is mostly straight and elongated, the network crack is in a staggered branch shape, and the longitudinal crack is distributed along the extension direction of the road.

[0070] In one embodiment of the present application, as shown in Figure 3 the identification of whether the abnormal area is a crack disease, the content is as follows: S41, extracting the reference value range of each morphological feature from the reference morphological data range of all crack type diseases.

[0071] S42, comparing the values of each morphological feature in the abnormal morphological data set with the reference value range of the corresponding morphological feature of all crack type diseases, if the values of each morphological feature are respectively within the reference value range of the corresponding morphological feature of a crack type disease, it is determined that the abnormal area is a crack disease, otherwise it is determined that the abnormal area is not a crack disease, the abnormal morphological data set is recorded and included in the detection report.

[0072] Considering that different morphological features are located in different crack type disease corresponding reference value range, so that the matching degree with different crack type diseases is different, therefore the matching degree is quantified by similarity, and the type with the highest similarity is selected as the final result.

[0073] S43, calculating the similarity of the abnormal morphological data set and the reference morphological data range of each crack type disease, and selecting the crack type disease with the highest similarity as the crack type disease of the abnormal area.

[0074] The present application adopts standardized Euclidean distance to calculate the similarity, and the smaller the Euclidean distance, the higher the similarity.

[0075] Considering that the historical data of each crack type corresponding morphological feature usually conforms to normal distribution, the statistical characteristics of the data are determined by probability distribution fitting, and the abnormal values exceeding the reasonable range are removed based on the standard deviation, to ensure the consistency of the remaining sample data.

[0076] Based on this, the reference morphological data range of all crack type diseases is preset as follows: first, extracting the morphological data set in the detail image of each historical record of each crack type disease from the historical database collected by the unmanned aerial vehicle.

[0077] Secondly, the probability distribution fitting is performed on each morphological feature in the morphological data set of each historical record to determine the mean and standard deviation of each morphological feature, and the historical records exceeding the set proportion of the standard deviation are removed.

[0078] In a specific embodiment of the present application, the set proportion of the standard deviation can be set to 3 times, which conforms to the 3σ principle of normal distribution. The implementer can also adjust it by himself, but the set proportion of the standard deviation should not be higher than 3.

[0079] Finally, based on all the remaining historical records corresponding to each crack type disease, the reference value range of each morphological feature corresponding to each crack type disease is constructed.

[0080] The application identifies whether the abnormal area is a crack disease and the crack type disease by matching the abnormal morphology data set in the detail image with the reference morphology data range of all preset crack type diseases, realizes accurate determination of the crack type, and reduces the misjudgment probability of non-crack diseases.

[0081] S5, when the abnormal area is a crack disease, the depth value and the extension length of the crack are obtained through the unmanned aerial vehicle depth detection, and a crack three-dimensional space model is constructed.

[0082] In an embodiment of the application, the crack three-dimensional space model is constructed in the following manner: S51, each crack is extracted from the detail image of the abnormal area, each crack is point-by-point scanned by the laser radar device carried by the unmanned aerial vehicle, and the depth value and the horizontal interval of each scanning point are obtained.

[0083] S52, the strike angle of each crack with respect to the horizontal ground is determined according to the depth value of each scanning point corresponding to each crack, and the change trend curve of the horizontal interval with respect to the crack length is established based on the horizontal interval of each scanning point, and the extension length of the corresponding crack is obtained therefrom.

[0084] S53, the crack three-dimensional space model is constructed according to the outline enclosed area, the outline boundary perimeter, the longest straight line distance and the smallest circumscribed rectangle in the detail image, in combination with the strike angle of each crack with respect to the horizontal ground and the extension length.

[0085] It should be noted that the strike angle is determined in the following manner: for each crack, a smooth curve is fitted on the reference surface according to the depth value of each scanning point and the position of the crack in the reference surface, taking the vertical horizontal ground as the reference surface, and the fitting straight line of the smooth curve is calculated by the least square method, and the angle between the fitting straight line and the horizontal ground is taken as the strike angle.

[0086] The extension length of the crack is determined in the following manner: taking the actual length of the crack at the scanning point as the abscissa and the horizontal interval of the scanning point as the ordinate, a coordinate system of the actual length of the crack and the horizontal interval is constructed, the horizontal interval and the actual length of the crack of each scanning point are substituted into the constructed coordinate system, the change trend curve of the horizontal interval with respect to the crack length is fitted, the crack length corresponding to the horizontal interval of 0 is read from the change trend curve, and the crack length is taken as the extension length of the crack.

[0087] The horizontal interval of each scanning point in the above represents the crack width perpendicular to the extension direction of the crack at the position of the scanning point.

[0088] S6, the actual coverage contour of the crack is analyzed based on the crack three-dimensional space model, and a road disease detection report is generated.

[0089] In one embodiment of the present application, the method for generating the road disease detection report is as follows: first, the three-dimensional coordinates of all points on each crack are obtained from the three-dimensional crack model, the three-dimensional coordinates of all points are mapped to the same plane to obtain all horizontal mapping points, and the minimum range contour containing all horizontal mapping points is circled according to the positions of all horizontal mapping points, which is taken as the actual coverage contour of the crack.

[0090] Then, the road disease detection report containing the position of the center point of the abnormal area, the crack type disease and the actual coverage contour is generated.

[0091] The present application obtains the crack depth value and the extension length through the depth detection of the unmanned aerial vehicle laser radar, constructs the three-dimensional crack model and analyzes the actual coverage contour, thereby providing comprehensive and accurate data support for the crack expansion trend prediction, which is helpful for formulating the targeted repair scheme and reducing the maintenance cost.

[0092] The above embodiments can be realized by software, hardware, firmware or any combination thereof, in whole or in part. When realized by software, the above embodiments can be realized in the form of a computer program product in whole or in part.

[0093] Those skilled in the art can realize that the modules and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solutions. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0094] In addition, the functional modules in each embodiment of the present application can be integrated in one processing module, or each module can exist physically alone, or two or more modules can be integrated in one module.

[0095] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0096] Finally, the above is merely a preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A road defect identification method based on AI and multi-scale fusion, characterized in that, include: The actual width of the road being inspected is obtained by a mobile inspection vehicle, and the image acquisition width of the platform mounted at the bottom is dynamically adjusted based on the actual width to acquire global images covering the road surface in real time. Hierarchical feature extraction is performed on the global image to obtain global image feature data. Combined with the normal reference image set of the inspected road, the quality of the global image is evaluated to determine whether it is up to standard, and abnormal areas with substandard quality are screened out. Based on the outline boundary of the abnormal area, a fixed-point acquisition command is sent to the drone carried by the mobile inspection vehicle to control the drone to hover above the center point of the abnormal area to acquire detailed images and extract the abnormal morphology dataset from the detailed images. The abnormal morphology dataset is matched with the preset reference morphology data range for all types of crack defects to identify whether the abnormal area is a crack defect. When the abnormal area is a crack, the depth and extension length of the crack are obtained by using a drone for depth detection, and a three-dimensional spatial model of the crack is constructed. Based on the analysis of the actual coverage contour of the crack using a three-dimensional spatial model of the crack, a road defect detection report is generated.

2. The road defect identification method based on AI and multi-scale fusion according to claim 1, characterized in that: The method for dynamically adjusting the image acquisition width of the bottom mounting platform is as follows: The spatial distance between the two sides of the current inspection road is collected by the distance sensor mounted on the mobile inspection vehicle and used as the actual width of the current inspection road. The fixed shooting angle range and acquisition overlap rate of the image acquisition device in the platform mounted on the bottom of the mobile inspection vehicle are retrieved, and the effective shooting width of the image acquisition device is determined based on the fixed shooting angle range and acquisition overlap rate. The ratio of the actual width of the current inspection road to the effective shooting width is calculated to obtain the required number of image acquisition devices in the bottom mounting platform. The image acquisition width of the bottom mounting platform is determined based on the required number of image acquisition devices, and then dynamically adjusted based on the current image acquisition width of the bottom mounting platform.

3. The road defect identification method based on AI and multi-scale fusion according to claim 1, characterized in that: The evaluation method for whether the global image quality is acceptable is as follows: The global image is scaled to obtain pixel-level and region-level images; The pixel-scale image is processed into grayscale. The grayscale image is then convolved using a gradient operator to obtain the horizontal and vertical gradient maps. The gradient magnitude of each pixel is calculated. Pixels with gradient magnitudes greater than a set gradient magnitude threshold are selected as valid pixels. The sum of the squares of the gradient magnitudes of all valid pixels is calculated to obtain the sharpness score of the pixel-scale image. The region-scale image is processed in grayscale, and the processed region-scale image is divided into several non-overlapping local blocks. Multiple directions containing distance and angle are selected in each local block, and the gray-level co-occurrence matrix of multiple directions is calculated to determine the texture feature vector of each local block in the region-scale image. Based on the sharpness score of pixel-level images and the texture feature vectors of each local block, the images are compared with the normal reference images of the inspected roads recorded in the road history database to determine whether the overall image quality is up to standard.

4. The road defect identification method based on AI and multi-scale fusion according to claim 3, characterized in that: The method for determining whether the global image quality is acceptable is as follows: Extract each reference image from the normal reference image set of the inspected road, obtain the reference sharpness score and reference texture feature vector of the corresponding image based on each reference image, and filter and statistically analyze the minimum reference sharpness score and the range of each reference texture feature; If the sharpness score of a pixel-level image is less than the minimum reference sharpness score, or if a texture feature value in the texture feature vector of a local block is outside the range of the corresponding reference texture feature, then the global image quality is deemed unqualified; otherwise, the global image quality is deemed qualified.

5. The road defect identification method based on AI and multi-scale fusion according to claim 3, characterized in that: The abnormal region filtering criteria are as follows: Filter all pixels whose gradient magnitude is less than the set gradient magnitude threshold, delineate the bounding polygon containing all pixels based on the position of all pixels, and record it as the suspected abnormal region corresponding to the pixel-level scale image. Local blocks whose texture feature values ​​in the texture feature vector are outside the range of the reference texture feature are denoted as candidate local blocks; The Euclidean distance between the texture features of all candidate local blocks and the mean range of the corresponding reference texture features is calculated. Candidate local blocks with Euclidean distance greater than the benchmark Euclidean distance are selected and their regions are merged to obtain the suspected abnormal regions corresponding to the region-scale image. The suspected anomalous regions in pixel-level and region-level images are compared by overlapping, and the overlapping regions are taken as the final anomalous regions.

6. The road defect identification method based on AI and multi-scale fusion according to claim 1, characterized in that: The dataset for extracting abnormal morphology from detailed images specifically includes: The acquired detail images are subjected to noise reduction and image enhancement processing, and abnormal edge contours of the processed detail images are extracted using an image edge algorithm. Read the bounding area, perimeter of the contour boundary, longest straight-line distance, and minimum bounding rectangle of the abnormal edge contour; The contour complexity and shape compactness are calculated by comparing the area enclosed by the contour boundary with the perimeter of the contour. The contour curvature and the aspect ratio of the circumscribed rectangle are determined based on the longest straight-line distance and the smallest circumscribed rectangle. Anomalies in detail images are constructed by considering contour complexity, shape compactness, contour curvature, and aspect ratio of the bounding rectangle.

7. The road defect identification method based on AI and multi-scale fusion according to claim 6, characterized in that: The method for identifying whether an abnormal area is a crack defect is as follows: Extract the reference value range of each morphological feature from the reference morphological data range of all preset crack types; The values ​​of each morphological feature in the abnormal morphological dataset are compared with the reference value range of the morphological features corresponding to all crack types. If the values ​​of each morphological feature are within the reference value range of the morphological features corresponding to a certain crack type, the abnormal area is determined to be a crack. Calculate the similarity between the abnormal morphology dataset and the reference morphology dataset for each crack type, and select the crack type with the highest similarity as the crack type of the abnormal region.

8. The road defect identification method based on AI and multi-scale fusion according to claim 7, characterized in that: The preset method for the reference morphological data range of all crack types is as follows: Detailed images of each type of crack disease in each historical record were extracted from the historical database collected by drones, and morphological datasets were extracted from the detailed images of each historical record. The probability distribution of each morphological feature in the morphological dataset of each historical record is fitted to determine the mean and standard deviation of each morphological feature, and historical records that exceed the set proportion of the standard deviation are removed. Based on all remaining historical records corresponding to each type of crack, a reference numerical range for each morphological characteristic corresponding to each type of crack is constructed.

9. The road defect identification method based on AI and multi-scale fusion according to claim 7, characterized in that: The method for constructing the three-dimensional spatial model of the crack is as follows: Each crack is extracted from the detailed image of the abnormal area, and each crack is scanned point by point using a lidar device mounted on a drone to obtain the depth value and horizontal spacing of each scan point. Based on the depth values ​​of each scan point corresponding to each crack, the orientation angle of each crack relative to the horizontal ground is determined, and based on the horizontal spacing of each scan point, a trend curve of the horizontal spacing as a function of crack length is established, from which the extension length of the corresponding crack is obtained. Based on the detailed images, the area enclosed by the contour, the perimeter of the contour boundary, the longest straight-line distance, and the minimum bounding rectangle are extracted. Combined with the direction angle and extension length of each crack relative to the horizontal ground, a three-dimensional spatial model of the crack is constructed.

10. A road defect identification method based on AI and multi-scale fusion according to claim 9, characterized in that: The method for generating road defect detection reports is as follows: The three-dimensional coordinates of all points on each crack are obtained from the three-dimensional spatial model of the crack. The three-dimensional coordinates of all points are mapped to the same plane to obtain all horizontal mapping points. Based on the position of all horizontal mapping points, the minimum range contour containing all horizontal mapping points is delineated and used as the actual coverage contour of the crack. Generate a road defect detection report that includes the location of the center point of the abnormal area, the type of crack, and the actual coverage outline.

Images