Road surface disease inspection method based on vehicle-mounted edge equipment

Abstract

The invention discloses a pavement disease inspection method based on a vehicle-mounted edge device, and relates to the technical field of pavement disease inspection, and the method comprises the steps: 1, obtaining image data, thermal imaging data and road geometric data of a pavement through an information obtaining module; 2, the information processing module processes the image data to obtain a disease area I; matching the image data with the thermal imaging data to obtain a disease area II; performing height change analysis on the geometric data of the road to obtain a disease area III; 3, the information processing module matches the disease area I, the disease area II and the disease area III with the position of the road surface, and when any position of the road surface is successfully matched with more than two disease areas I, disease areas II or disease areas III, the position of the road is marked as a disease part. According to the method, the multi-modal road surface data are adopted, and various light-weight means are adopted, so that the remarkable optimization effect is achieved in the aspects of performance, efficiency, resource utilization and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of road surface defect inspection, and in particular to a road surface defect inspection method based on vehicle-mounted edge equipment. Background Art

[0002] In order to accelerate the long-term mechanism of highway maintenance management and improve the level of highway maintenance management, the application ratio and detection accuracy of automatic detection equipment and artificial intelligence technologies in highway maintenance are constantly improving;

[0003] In road defect inspections, intelligent road defect detection equipment is mainly installed on vehicles, which collects road surface data during driving and detects and identifies the location and type of road defects based on the collected data.

[0004] After searching, a Chinese patent (publication number: CN116051561A) discloses a road disease inspection method based on a vehicle-mounted edge device. The patent includes continuously acquiring ROI area images within a set range; introducing an optimized network structure to build a lightweight inference engine; preprocessing the input ROI area image based on the lightweight inference engine; parsing the result data stream to generate predicted value data for disease classification of disease boundary data, and uploading it to the cloud server to end the inspection.

[0005] In the prior art, the optimization of lightweight algorithm is poor, and the image acquisition, algorithm reasoning, and result analysis process nodes are separated from each other, and the overall integration is low. Therefore, the present invention proposes a road disease inspection method based on vehicle-mounted edge devices. Summary of the invention

[0006] The purpose of the present invention is to provide a road surface disease inspection method based on a vehicle-mounted edge device to solve the problems mentioned in the above background technology.

[0007] The present invention can be implemented through the following technical solutions: a road surface disease inspection method based on a vehicle-mounted edge device, comprising an information acquisition module, an information processing module, and a cloud server;

[0008] The information acquisition module is a vehicle-mounted device used to obtain road information, including:

[0009] Industrial camera unit collects high-definition images in real time while the vehicle is driving, captures road surface details, and obtains image data;

[0010] The infrared camera unit collects infrared thermal images of the road surface while the vehicle is driving, obtains thermal imaging data, and identifies changes in road surface materials and crack areas through temperature distribution;

[0011] The laser radar unit detects the three-dimensional geometric changes of the road surface in combination with the height change data of the laser radar during the driving process of the vehicle, obtains the road geometry data, and is used to identify structural defects such as potholes and ridges, and provides height information;

[0012] The information processing module is an in-vehicle edge device, including:

[0013] The image processing unit performs histogram equalization on the image data collected by the industrial camera unit to enhance the contrast of the image and make the cracks and potholes clearer under different light conditions. The image data is then converted from the spatial domain to the frequency domain. The high-frequency information is retained to highlight the details of the diseased area (such as the crack edge) and suppress the low-frequency background information of the flat road surface. Finally, the image data processed in the frequency domain is subjected to Canny edge detection to further enhance the edge contour of the diseased area and eliminate the interference of the background road surface to obtain the diseased area I.

[0014] The fusion processing unit matches the image data with the thermal imaging data, corresponding to the temperature and optical characteristics of the same spatial area of ​​the road, and through the multi-spectral fusion algorithm, highlights the area with obvious temperature changes in the same spatial area of ​​the road, and forms fused image data;

[0015] Based on the fused image data, the fusion processing unit marks the area with abnormal temperature in the road surface, supplements the area with insufficient optical information, and obtains the disease area II;

[0016] The structural analysis unit performs a height change analysis on the road geometry data, reduces the data volume and improves the processing efficiency through point cloud preprocessing, including downsampling and noise removal, and then extracts three-dimensional features through wavelet transformation, including the depth of potholes and the height of ridges, to obtain the disease area III;

[0017] The AI ​​analysis unit matches the diseased area I, diseased area II and diseased area III with the road surface position. When two or more diseased areas I, diseased areas II or diseased areas III are successfully matched at any position on the road surface, the road position is marked as a diseased part.

[0018] The AI ​​analysis unit extracts the diseased parts through the U-Net segmentation network to reduce the scope of the diseased parts, extracts features from the reduced diseased parts through CNN, and classifies the disease types through the classifier to obtain disease classification results, including cracks, potholes, and bulges.

[0019] Finally, the AI ​​analysis unit transmits the diseased parts and their corresponding location information and disease classification results to the cloud server;

[0020] The inspection method includes the following steps:

[0021] Step 1: Obtain road surface image data, thermal imaging data and road geometry data through an information acquisition module;

[0022] Step 2: The information processing module processes the image data to obtain the diseased area I;

[0023] Match the image data with the thermal imaging data to obtain fused image data, and mark the area with abnormal temperature in the road surface to obtain the disease area II;

[0024] The road geometry data is analyzed for height variation, and three-dimensional features are extracted through point cloud preprocessing and wavelet transformation to obtain the damaged area III;

[0025] Step 3: The information processing module matches the damaged area I, the damaged area II and the damaged area III with the road surface position. When two or more damaged areas I, II or III are successfully matched at any position on the road surface, the road position is marked as a damaged area.

[0026] Step 4: The information processing module reduces the scope of the diseased part, and performs feature extraction and classification on the reduced diseased part to obtain a disease classification result;

[0027] Step 5: The cloud server receives the diseased part and its corresponding location information and the disease classification result.

[0028] A further technical improvement of the present invention is that it includes an inertial measurement unit for detecting changes in sensor posture caused by vehicle movement.

[0029] A further technical improvement of the present invention is that: when the industrial camera unit, the infrared camera unit and the laser radar unit are used, spatial calibration of multimodal data is performed, including alignment of multimodal data, posture compensation and spatiotemporal synchronization of data;

[0030] Its formula is T i (t) = H i (t)×P i (t)+R(t)+E i (t);

[0031] Among them, T i (t) is the data after calibration of the industrial camera unit, infrared camera unit and lidar unit at time t, ensuring that the outputs of different sensors are aligned at the same time and coordinate system;

[0032] H i(t) is the registration matrix of the industrial camera unit, infrared camera unit and laser radar unit at time t. The industrial camera unit, infrared camera unit and laser radar unit have their own registration matrices, which are used to align the data of different sensors into a unified coordinate system, that is, to align the image data, thermal imaging data and road geometry data;

[0033] P i (t) is the raw data collected by the industrial camera unit, infrared camera unit and lidar unit at time t, without any calibration;

[0034] R(t) is the vehicle's attitude transformation matrix, which is used to describe the sensor attitude changes caused by vehicle motion. It compensates for the sensor position and angle changes caused by vehicle motion based on the inertial measurement unit;

[0035] E i (t) is the attitude error term, which describes the small attitude error caused by vehicle motion and is used for compensation in the further calibration process.

[0036] A further technical improvement of the present invention is that the histogram equalization processing of the image processing unit adopts the CLAHE algorithm, whose formula is:

[0037] Among them, I c (x, y) is the pixel value of the enhanced image at position (x, y) after histogram equalization;

[0038] I v (x, y) is the input image, that is, the pixel value of the original image data at position (x, y);

[0039] max(I v ) and min(I v ) are the minimum and maximum pixel values ​​in the input image,

[0040] L is the gray level of the image.

[0041] A further technical improvement of the present invention is that after the image data is processed by the CLAHE algorithm, a further dynamic median filtering process is performed, and the formula is:

[0042] I f (x,y)=Median({I c (x+i, y+j)|i, j∈[-k,k]});

[0043] I f (x, y) is the pixel value at position (x, y) of the image after dynamic median filtering;

[0044] I c(x, y) is the pixel value at position (x, y) of the enhanced image after histogram equalization;

[0045] k refers to the radius of the filter window, that is, the window size is (2k+1)×(2k+1) pixel neighborhood;

[0046] Median is a median operation that calculates the median value of all pixel values ​​in the neighborhood of the pixel at the (x, y) position.

[0047] A further technical improvement of the present invention is that the formula used by the fusion processing unit to match the image data with the thermal imaging data is:

[0048] I f =αI v +(1-α)I r +γΔT;

[0049] I f It is a fused multimodal image, which is a comprehensive image of visible light image, infrared thermal image and temperature difference, containing optical, temperature and temperature difference information;

[0050] I v is the image data;

[0051] I r It is thermal imaging data;

[0052] ΔT is the temperature difference data, which is the local temperature difference of the pavement, such as the temperature difference between the crack area and the intact area. The temperature difference can further highlight the pavement disease area;

[0053] α and γ are weight coefficients, which respectively control the contribution of image data, thermal imaging data and temperature difference to the final fused image.

[0054] A further technical improvement of the present invention is that: the cloud server trains a three-dimensional structure analysis model based on the long-term collection of road geometry data by the laser radar unit, optimizes the reconstruction algorithm of the edge device, and feeds back to the on-board edge device to improve the detection accuracy;

[0055] After receiving the data after point cloud preprocessing, the cloud server analyzes and processes the three-dimensional structural analysis model and further improves the detection accuracy.

[0056] A further technical improvement of the present invention is that when the industrial camera unit is working continuously, there is a large area of ​​useless sky or distant scenery. At the same time, at a fixed capture distance, there is a fixed repeated area between adjacent captured pictures. The camera capture ROI area is set according to the repeated area range, and only the image of this area is generated, thereby reducing the size of the captured image data at the source without losing the image display accuracy.

[0057] A further technical improvement of the present invention is that the AI ​​analysis unit uses the TensorRT or OpenVINO lightweight deep learning framework to execute the U-Net model on the edge device to segment the diseased area;

[0058] The formula is:

[0059] L d is the Dice loss function, which is used to optimize the segmentation model;

[0060] S t is the temperature similarity;

[0061] S g is the geometric similarity;

[0062] a and b are weight coefficients, which control the influence of temperature similarity and geometric similarity on the final loss function.

[0063] A further technical improvement of the present invention is that the area of ​​the road surface captured by the infrared camera unit is the same as the ROI area of ​​the industrial camera unit, and the matching method thereof comprises the following steps:

[0064] S1. Perform spatial calibration between the infrared camera unit and the industrial camera unit;

[0065] The calibration method adopts the calibration plate method or the image registration method;

[0066] S2, coordinate transformation and ROI area mapping;

[0067] The coordinate point corresponding to the ROI area of ​​the industrial camera unit is (x o ,y o ), the coordinate point corresponding to the ROI area of ​​the infrared camera unit is (x t ,y t );

[0068] By formula The ROI area coordinates of the industrial camera unit are converted into the coordinate system of the infrared camera unit, so as to obtain an ROI area in the infrared camera unit that is consistent with the industrial camera unit.

[0069] A further technical improvement of the present invention is that the infrared camera unit adopts a reduced resolution or an adaptive resolution for the captured ROI area.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] The present invention achieves significant optimization effects in terms of performance, efficiency and resource utilization by using multimodal road surface data and a variety of lightweight means;

[0072] In the process of multimodal data fusion, lightweight design is used to ensure that data redundancy and processing volume are greatly reduced without sacrificing detection accuracy, so that the system can still operate efficiently in complex environments;

[0073] And by dynamically adjusting and reducing the amount of processed data, the system can maintain efficient real-time detection capabilities in complex road environments, greatly reducing delays and making disease detection faster and more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0075] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0076] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0077] See also Figure 1 As shown, the present invention provides a road surface disease inspection method based on a vehicle-mounted edge device, and the inspection method includes the following steps:

[0078] Step 1: Obtain road surface image data, thermal imaging data and road geometry data through an information acquisition module;

[0079] The information acquisition module is a vehicle-mounted device used to obtain road information, including:

[0080] Industrial camera unit collects high-definition images in real time while the vehicle is driving, captures road surface details, and obtains image data;

[0081] The infrared camera unit collects infrared thermal images of the road surface while the vehicle is driving, obtains thermal imaging data, and identifies changes in road surface materials and crack areas through temperature distribution;

[0082] The laser radar unit detects the three-dimensional geometric changes of the road surface in combination with the height change data of the laser radar during the driving process of the vehicle, obtains the road geometry data, and is used to identify structural defects such as potholes and ridges, and provide height information;

[0083] An inertial measurement unit to detect changes in sensor attitude caused by vehicle motion;

[0084] When the industrial camera unit is working continuously, there is a large area of ​​useless sky or distant scenery. At the same time, at a fixed capture distance, there is a fixed repeated area between adjacent captured pictures. The camera capture ROI area is set according to the repeated area range, and only the image of this area is generated, which reduces the size of the captured image data at the source without losing the image display accuracy.

[0085] The area of ​​the road surface captured by the infrared camera unit is the same as the ROI area of ​​the industrial camera unit, and the matching method includes the following steps:

[0086] S1. Perform spatial calibration between the infrared camera unit and the industrial camera unit;

[0087] In this embodiment, an image registration method is used, including:

[0088] a1. The industrial camera unit and the infrared camera unit capture the same scene, and use the Image Registration algorithm to align the images of the two. According to the image feature points, the spatial mapping relationship between the two images is found;

[0089] a2. The industrial camera unit provides the contour or edge information of the visible light image, and the infrared camera unit provides the temperature difference distribution image. The registration is performed by matching the edge or contour features of the two.

[0090] S2, coordinate transformation and ROI area mapping;

[0091] The coordinate point corresponding to the ROI area of ​​the industrial camera unit is (x o ,y o ), the coordinate point corresponding to the ROI area of ​​the infrared camera unit is (x t ,y t );

[0092] By formula The ROI area coordinates of the industrial camera unit are converted into the coordinate system of the infrared camera unit, so as to obtain the ROI area in the infrared camera unit that is consistent with the industrial camera unit;

[0093] In this embodiment, the infrared camera unit adopts adaptive resolution for the captured ROI area, that is, when an area with a large temperature difference (such as a crack or material peeling) is detected, the high resolution is maintained;

[0094] Use lower resolution in areas with smaller temperature differences to reduce the amount of data.

[0095] The industrial camera unit, infrared camera unit and laser radar unit perform spatial calibration of multimodal data when in use, including alignment of multimodal data, posture compensation and spatiotemporal synchronization of data;

[0096] Its formula is T i (t) = H i (t)×P i (t)+R(t)+E i (t);

[0097] Among them, T i (t) is the data after calibration of the industrial camera unit, infrared camera unit and lidar unit at time t, ensuring that the outputs of different sensors are aligned at the same time and coordinate system;

[0098] H i (t) is the registration matrix of the industrial camera unit, infrared camera unit and laser radar unit at time t. The industrial camera unit, infrared camera unit and laser radar unit have their own registration matrices, which are used to align the data of different sensors into a unified coordinate system, that is, to align the image data, thermal imaging data and road geometry data;

[0099] P i (t) is the raw data collected by the industrial camera unit, infrared camera unit and lidar unit at time t, without any calibration;

[0100] R(t) is the vehicle's attitude transformation matrix, which is used to describe the sensor attitude changes caused by vehicle motion. It compensates for the sensor position and angle changes caused by vehicle motion based on the inertial measurement unit;

[0101] E i (t) is the attitude error term, which describes the small attitude error caused by vehicle motion and is used for compensation in the further calibration process;

[0102] The H i The detailed expression of (t) is:

[0103]

[0104] Among them, R X , R y , R z The rotation matrix representing the direction of the coordinate axis rotates the sensor coordinates to align;

[0105] T X , T y , T z Represents the translation matrix, ensuring that the position deviation of the sensor can be compensated;

[0106] The forms of R(t) include:

[0107]

[0108] θ is the change in vehicle tilt angle.

[0109] Step 2: The information processing module processes the image data to obtain the diseased area I;

[0110] The process is as follows: the information processing module uses an image processing unit to perform histogram equalization on the image data collected by the industrial camera unit to enhance the contrast of the image and make cracks and potholes clearer under different light conditions. The image data is then converted from the spatial domain to the frequency domain, and the details of the diseased area are highlighted by retaining high-frequency information, while the low-frequency background information of the flat road surface is suppressed. Finally, the image data processed in the frequency domain is subjected to Canny edge detection to further enhance the edge contour of the diseased area and eliminate the interference of the background road surface, thereby obtaining the diseased area I.

[0111] The histogram equalization process of the image processing unit adopts the CLAHE algorithm, and its formula is:

[0112] Among them, I c (x, y) is the pixel value of the enhanced image at position (x, y) after histogram equalization;

[0113] I v (x, y) is the input image, that is, the pixel value of the original image data at position (x, y);

[0114] max(I v ) and min(I v ) are the minimum and maximum pixel values ​​in the input image,

[0115] L is the gray level of the image;

[0116] It evenly distributes the pixel values ​​in the original image over the entire grayscale range by linear stretching, normalizes the original image pixel values ​​so that their grayscale values ​​are distributed between 0 and 1, and then multiplies them by L-1 to expand them to the entire grayscale level range;

[0117] The image is divided into 8x8 grids by CLAHE, and histogram equalization is performed separately, and then these areas are stitched together to improve the visual effect of the image under different lighting conditions;

[0118] After the image data is processed by the CLAHE algorithm, it is further processed by dynamic median filtering, and the formula is:

[0119] I f (x,y)=Median({I c (x+i, y+j)|i, j∈[-k,k]});

[0120] I f(x, y) is the pixel value at position (x, y) of the image after dynamic median filtering;

[0121] I c (x, y) is the pixel value at position (x, y) of the enhanced image after histogram equalization;

[0122] k is the radius of the filter window, that is, the window size is (2k+1)×(2k+1) pixel neighborhood;

[0123] Median is a median operation, which calculates the median of all pixel values ​​in the neighborhood of the pixel at the (x, y) position;

[0124] In dynamic median filtering, a (2k+1)×(2k+1) sliding window is selected to process each pixel in the image. Within this window, the median of the sorted results of all pixel values ​​will be used as I f The output value of (x,y) is used to eliminate noise;

[0125] If k=1, the window size is 3×3, that is, the 8 pixel values ​​around each pixel are sorted and the middle value is taken as the output;

[0126] By performing dynamic median filtering on the image data, noise, such as salt noise or pepper noise in the image, can be effectively removed, while retaining the edge details of the defective areas such as cracks and potholes, making the outline of the defective area clearer;

[0127] Match the image data with the thermal imaging data to obtain fused image data, and mark the area with abnormal temperature in the road surface to obtain the disease area II;

[0128] The process is that the information processing module uses a fusion processing unit to match the image data with the thermal imaging data, corresponding to the temperature and optical characteristics of the same spatial area of ​​the road, and through the multi-spectral fusion algorithm, highlight the areas with obvious temperature changes in the same spatial area of ​​the road, and form fused image data;

[0129] Based on the fused image data, the fusion processing unit marks the area with abnormal temperature in the road surface, supplements the area with insufficient optical information, and obtains the disease area II;

[0130] The formula used by the fusion processing unit to match the image data with the thermal imaging data is:

[0131] I f =αI v +(1-α)I r +γΔT;

[0132] I fIt is a fused multimodal image, which is a comprehensive image of visible light image, infrared thermal image and temperature difference, containing optical, temperature and temperature difference information;

[0133] I v is the image data;

[0134] I r It is thermal imaging data;

[0135] ΔT is the temperature difference data, which is the local temperature difference of the pavement, such as the temperature difference between the crack area and the intact area. The temperature difference can further highlight the pavement disease area;

[0136] α and γ are weight coefficients, which respectively control the contribution of image data, thermal imaging data and temperature difference to the final fused image;

[0137] The road geometry data is analyzed for height variation, and three-dimensional features are extracted through point cloud preprocessing and wavelet transformation to obtain the damaged area III;

[0138] The process is that the information processing module uses the structural analysis unit to analyze the height change of the road geometry data, and reduces the data volume and improves the processing efficiency through point cloud preprocessing, including downsampling and noise removal, and then extracts three-dimensional features through wavelet transformation, including the depth of potholes and the height of ridges, to obtain the disease area III;

[0139] Step 3: The information processing module uses an AI analysis unit to match the diseased area I, diseased area II and diseased area III with the road surface position. When two or more diseased areas I, diseased areas II or diseased areas III are successfully matched at any position on the road surface, the road position is marked as a diseased area.

[0140] Step 4: The AI ​​analysis unit extracts the diseased parts through the U-Net segmentation network to reduce the scope of the diseased parts, extracts features from the reduced diseased parts through CNN, and classifies the disease types through the classifier to obtain the disease classification results, including cracks, potholes, and bulges;

[0141] The AI ​​analysis unit uses the TensorRT lightweight deep learning framework to execute the U-Net model on the edge device to segment the diseased area;

[0142] The formula is:

[0143] L d is the Dice loss function, which is used to optimize the segmentation model;

[0144] S t is the temperature similarity;

[0145] S gis the geometric similarity;

[0146] a and b are weight coefficients, which control the influence of temperature similarity and geometric similarity on the final loss function;

[0147] Step 5. Finally, the AI ​​analysis unit transmits the diseased part and its corresponding location information and the disease classification result to the cloud server, and the cloud server receives the diseased part and its corresponding location information and the disease classification result;

[0148] The cloud server trains a three-dimensional structure analysis model based on the long-term collection of road geometry data by the laser radar unit, optimizes the reconstruction algorithm of the edge device, and feeds back to the on-board edge device to improve detection accuracy;

[0149] After receiving the data after point cloud preprocessing, the cloud server analyzes and processes the three-dimensional structural analysis model and further improves the detection accuracy.

[0150] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A road surface disease inspection method based on a vehicle-mounted edge device, characterized in that: The inspection method comprises the following steps: Step 1: Obtain road surface image data, thermal imaging data and road geometry data through an information acquisition module; Step 2: The information processing module processes the image data to obtain the diseased area I; Match the image data with the thermal imaging data to obtain fused image data, and mark the area with abnormal temperature in the road surface to obtain the disease area II; The road geometry data is analyzed for height changes, and three-dimensional features are extracted through point cloud preprocessing and wavelet transformation to obtain the damaged area III; Step 3: The information processing module matches the damaged area I, the damaged area II and the damaged area III with the road surface position. When any two or more of the damaged areas I, II and III are successfully matched at any position on the road surface, the road position is marked as a damaged area. Step 4: The information processing module reduces the scope of the diseased part, and performs feature extraction and classification on the reduced diseased part to obtain a disease classification result; Step 5: The cloud server receives the diseased part and its corresponding location information and the disease classification result.

2. A road surface disease inspection method based on a vehicle-mounted edge device according to claim 1, characterized in that: The information acquisition module includes: Industrial camera unit collects high-definition images in real time while the vehicle is driving, captures road surface details, and obtains image data; The infrared camera unit collects infrared thermal images of the road surface and obtains thermal imaging data while the vehicle is driving; The laser radar unit detects the three-dimensional geometric changes of the road surface in combination with the height change data of the laser radar during the driving process of the vehicle, and obtains the road geometry data.

3. A road surface disease inspection method based on a vehicle-mounted edge device according to claim 2, characterized in that: The information processing module includes: The image processing unit performs histogram equalization processing on the image data collected by the industrial camera unit to enhance the contrast of the image, and then converts the image data from the spatial domain to the frequency domain, and suppresses the low-frequency background information of the flat road surface. Finally, the image data processed in the frequency domain is subjected to Canny edge detection to further enhance the edge contour of the diseased area, eliminate the interference of the background road surface, and obtain the diseased area I; The fusion processing unit matches the image data with the thermal imaging data, corresponding to the temperature and optical characteristics of the same spatial area of ​​the road, and through the multi-spectral fusion algorithm, highlights the area with obvious temperature changes in the road in the same spatial area, and forms fused image data; Based on the fused image data, the fusion processing unit marks the area with abnormal temperature in the road surface, supplements the area with insufficient optical information, and obtains the disease area II; The structural analysis unit performs a height change analysis on the road geometry data, reduces the data volume and improves the processing efficiency through point cloud preprocessing, and then extracts three-dimensional features through wavelet transformation to obtain the disease area III; The AI ​​analysis unit matches the defect area I, defect area II and defect area III with the road surface position, and extracts the defect part through the U-Net segmentation network to reduce the scope of the defect part, extracts features of the reduced defect part through CNN, and classifies the defect type through the classifier to obtain the defect classification result; Finally, the AI ​​analysis unit transmits the diseased parts, their corresponding location information and disease classification results to the cloud server.

4. A road surface disease inspection method based on a vehicle-mounted edge device according to claim 2, characterized in that: When in use, the industrial camera unit, infrared camera unit and laser radar unit perform spatial calibration of multimodal data, including alignment of multimodal data, posture compensation and time-space synchronization of data.

5. A road surface disease inspection method based on a vehicle-mounted edge device according to claim 4, characterized in that: The histogram equalization processing of the image processing unit adopts the CLAHE algorithm; After being processed by the CLAHE algorithm, the image data is further processed by dynamic median filtering.

6. A road surface disease inspection method based on a vehicle-mounted edge device according to claim 2, characterized in that: When the industrial camera unit works continuously, the camera capture ROI area is set according to the repeated area range.

7. A road surface disease inspection method based on a vehicle-mounted edge device according to claim 6, characterized in that: The area of ​​the road surface captured by the infrared camera unit is the same as the ROI area of ​​the industrial camera unit, and the matching method includes the following steps: S1. Perform spatial calibration between the infrared camera unit and the industrial camera unit; The calibration method adopts the calibration plate method or the image registration method; S2, coordinate transformation and ROI area mapping; The coordinate point corresponding to the ROI area of ​​the industrial camera unit is (x o ,y o ), the coordinate point corresponding to the ROI area of ​​the infrared camera unit is (x t ,y t ); By formula Convert the ROI area coordinates of the industrial camera unit to the coordinate system of the infrared camera unit.

8. A road surface disease inspection method based on a vehicle-mounted edge device according to claim 7, characterized in that: The infrared camera unit uses reduced resolution or adaptive resolution for the captured ROI area based on the degree of temperature difference.

Images