An automated rapid pavement detection method based on high-speed camera technology

Abstract

The present invention relates to the technical field of image analysis, and specifically to a method for automatic and rapid detection of road surfaces based on high-speed camera technology. First, the emergency layer in the dynamic photographing trigger mechanism triggers the high-speed camera to take pictures according to the information collected by the sensor; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to determine whether the emergency layer is mis-triggered; the statistics layer dynamically adjusts the trigger threshold; then, a pixel displacement constraint formula is constructed based on the vehicle speed data, a dynamic adjustment strategy for the shutter speed is constructed according to the pixel displacement constraint formula and pictures are taken, and a first set of road surface images is constructed by using the imaging clarity index; the motion blur removal model fuses the images, and removes the blur from the fused images according to the dynamic blur kernel to obtain clear images; the road surface detection model obtains the image features, vibration spectrum features and spectral features and processes them to obtain the detection results. Through the method proposed by the present invention, the efficiency of automatic road surface detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and particularly to an automatic and rapid pavement detection method based on high-speed camera technology. Background Art

[0002] With the continuous expansion of the scale of traffic infrastructure, the automatic detection technology for pavement health status has become a key requirement for improving road maintenance efficiency and reducing accident risks. Traditional detection methods mostly rely on manual inspections or low-frame-rate imaging devices, which have problems such as low efficiency, limited coverage, and poor adaptability to dynamic scenes. In recent years, high-speed camera technology, with its high frame rate and high-resolution characteristics, has provided a hardware basis for pavement image acquisition under high-speed movement conditions. However, the resulting massive image data poses a severe challenge to real-time processing and analysis capabilities.

[0003] In the field of image data processing, traditional methods usually use algorithms such as threshold segmentation, edge detection, or template matching for image analysis. However, such methods are significantly limited in high-speed dynamic scenes: firstly, the algorithms rely on manually designed features such as texture gradients or geometric shapes, and it is difficult to distinguish real defects from interference factors such as pavement stains and shadows, resulting in an increased false detection rate; secondly, the calculation process of traversing each pixel is difficult to meet the real-time requirements of high-speed data streams, and the fixed parameter settings cannot adapt to the changes in different pavement materials such as asphalt and concrete or environmental conditions, and the generalization ability is insufficient. Although deep learning models have shown advantages in static image defect recognition, high-precision network structures usually rely on high-performance computing devices, and when directly applied to high-speed scenarios, they face the contradiction between model complexity and computing resources and are difficult to be deployed on in-vehicle embedded platforms. In addition, the multi-spectral or high-dynamic-range imaging data supported by high-speed cameras have not been fully exploited. For example, the sensitivity of the near-infrared band to water seepage areas or the motion correlation between sequential frames. Existing algorithms, due to the lack of multi-modal fusion and time dimension modeling, are prone to ignoring small defects or misjudging instantaneous noise.

[0004] Therefore, an automatic and rapid pavement detection method based on high-speed camera technology is proposed. Summary of the Invention

[0005] The object of the present invention is to provide a pavement automatic rapid detection method based on high-speed camera technology to improve the efficiency of pavement automatic detection. First, the emergency layer in the dynamic photo-taking trigger mechanism triggers the high-speed camera to take pictures according to the information collected by the sensor; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to judge whether the emergency layer is mis-triggered; the statistics layer dynamically adjusts the trigger threshold; then, a pixel displacement constraint formula is constructed according to the vehicle speed data, a shutter speed dynamic adjustment strategy is constructed according to the pixel displacement constraint formula and pictures are taken, and a first pavement image set is constructed by using the imaging clarity index; the motion blur removal model fuses the images, and removes the blur from the fused images according to the dynamic blur kernel to obtain clear images; the pavement detection model obtains image features, vibration spectrum features and spectral features and processes them to obtain detection results.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A pavement automatic rapid detection method based on high-speed camera technology, comprising:

[0008] The dynamic photo-taking trigger mechanism includes an emergency layer, an analysis layer and a statistics layer; wherein the emergency layer is used to construct a trigger condition according to the information collected by the sensor and trigger the high-speed camera to take pictures; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to judge whether the emergency layer is mis-triggered; the statistics layer dynamically adjusts the trigger threshold based on historical data and data in the most recent time window; a pixel displacement constraint formula is constructed according to the vehicle speed data, and a shutter speed dynamic adjustment strategy is constructed according to the pixel displacement constraint formula;

[0009] Further, the formula for the trigger condition of the emergency layer is:

[0010] ;

[0011] Wherein, represents the trigger condition, represents the vertical acceleration measured by the triaxial accelerometer, represents the basic acceleration threshold, represents the vehicle speed influence factor, represents the current vehicle speed, represents the obstacle distance detected by the millimeter wave radar, represents the safety distance threshold.

[0012] Further, the analysis layer includes: constructing a joint feature vector according to the vibration spectrum and the spectral reflectance and the image ROI texture entropy ;

[0013] If the result of the improved SVM decision function processing the joint feature vector is within the dynamic confidence region, it is not a false trigger. The formula for dynamic confidence is:

[0014] ;

[0015] Among them, represents the dynamic confidence, represents the accumulated value of the vibration feature frequency band energy, represents the spectral reflectance and the absolute difference with the road surface reference value ; represents the exponential function, , and represent the weight coefficients.

[0016] Furthermore, the statistical layer updates the acceleration threshold using the exponentially weighted moving average. The formula is:

[0017] ;

[0018] Among them, represents the basic acceleration threshold at time represents the historical data decay factor, represents the basic acceleration threshold at time represents the corresponding average acceleration verified by the analysis layer within the

[0019] Furthermore, the shutter speed dynamic adjustment strategy is constructed according to the pixel displacement constraint formula. The pixel displacement constraint formula is:

[0020] ;

[0021] Among them, represents the maximum allowable exposure time, represents the pixel size of the camera sensor, represents the current vehicle speed, represents the optical magnification, represents the angle between the camera optical axis and the road surface normal;

[0022] Take pictures according to the shutter speed dynamic adjustment strategy to obtain a set of road surface images;

[0023] Sort the images in the collected set of road surface images according to the imaging clarity index, and construct the first set of road surface images based on the images whose imaging clarity index is greater than the threshold.

[0024] Construct an imaging clarity index based on the normalized gradient energy, signal-to-noise ratio, and ambiguity based on frequency domain analysis, and construct a first set of road surface images using the imaging clarity index;

[0025] The motion blur removal model fuses the first set of road surface images, and removes the blur from the fused image according to the dynamic blur kernel to obtain a clear image; the clear image is uploaded to the cloud, and the road surface detection model obtains the image features, vibration spectrum features, and spectral features and processes them to obtain the detection result.

[0026] Furthermore, the motion blur removal model includes an image fusion layer, a fused image input layer, a multi-scale encoder-decoder layer, and a clear image output layer; among them, the image fusion layer obtains the images in the first set of road surface images and fuses them to obtain a fused image;

[0027] The multi-scale encoder-decoder layer includes an encoder and a decoder. The encoder contains 4 levels of downsampling, each level contains a dynamic convolution block and a motion blur attention block, and the decoder contains 4 levels of upsampling, each level contains a cross-scale skip link block and a non-local attention block;

[0028] A dynamic blur kernel is constructed according to the vehicle speed data in the motion blur attention block, and the formula is:

[0029] ;

[0030] Among them, represents the dynamic blur kernel at the position in the fused image, represents the exposure time, represents the two-dimensional Dirac function, and represent the components of the vehicle speed on the axis and the axis, and represent the components of the acceleration of the vehicle motion on the axis and the axis, represents the integration variable;

[0031] Construct a loss function according to the blur kernel matching loss and the gradient perception loss.

[0032] Furthermore, the road surface detection model includes a multi-modal input encoding layer, a cross-modal fusion layer, and a multi-task output layer. The multi-modal input encoding layer receives data and extracts image features, vibration spectrum features, and spectral features; the cross-modal fusion layer includes a cross-attention layer and a dynamic gating fusion layer for fusing the features; the multi-task output layer includes a segmentation head for outputting the segmentation mask of the object to be detected, a classification head for outputting the category probability of the object to be detected, and a regression head for outputting the size and depth of the object to be detected;

[0033] Construct a joint loss function based on the segmentation loss, classification loss, and regression loss.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] 1. The emergency layer in the dynamic photo-triggering mechanism can achieve fast triggering and low-latency response; the analysis layer conducts analysis by constructing a joint feature vector, significantly reducing the false triggering rate; the statistics layer adapts to the real-time changes in road conditions by updating the acceleration threshold; the dynamic photo-triggering mechanism realizes the automatic triggering of high-speed cameras to take pictures through a logical closed-loop of response-validation-feedback adjustment, improving the accuracy of triggering to take pictures.

[0036] 2. Construct a pixel displacement constraint formula based on vehicle speed data and dynamically adjust the shutter speed, which can effectively eliminate motion blur in high-speed moving scenarios and ensure clear details of road surface images captured at different vehicle speeds; at the same time, combined with the imaging clarity index constructed by normalizing gradient energy, signal-to-noise ratio, and frequency domain blur, it can quantitatively evaluate each frame of image in the multi-exposure image set and adaptively screen out the optimal image subset, thereby significantly improving the quality of input data.

[0037] 3. The motion blur removal model realizes high-quality image fusion and blur removal through weighted fusion, pixel weight coefficients, and dynamic blur kernels, providing clear images for road surface detection; the road surface detection model obtains clear road surface images and conducts multi-modal analysis in combination with data of other modalities, ultimately realizing the accurate detection of road surface conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of a road surface automatic rapid detection method based on high-speed camera technology provided by an embodiment of the present invention;

[0039] Figure 2 It is a structural schematic diagram of a motion blur removal model provided by an embodiment of the present invention;

[0040] Figure 3 It is a structural schematic diagram of a road surface detection model provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0042] Embodiment 1

[0043] When a company conducts defect detection on the newly built highway pavement, in order to improve the detection accuracy and shorten the detection time, it introduces a pavement automatic rapid detection method based on high-speed camera technology provided by the present invention. The method flow is as follows: Figure 1 As shown below, the specific implementation is as follows:

[0044] The dynamic photo-taking trigger mechanism includes an emergency layer, an analysis layer, and a statistics layer; among them, the emergency layer is used to construct a trigger condition based on the information collected by the sensor to trigger the high-speed camera to take pictures; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to judge whether the emergency layer is mis-triggered; the statistics layer dynamically adjusts the trigger threshold based on historical data and data in the most recent time window;

[0045] Furthermore, the formula for the trigger condition of the emergency layer is:

[0046] ;

[0047] Among them, represents the trigger condition, represents the vertical acceleration measured by the triaxial accelerometer, represents the basic acceleration threshold, represents the vehicle speed influence factor, represents the current vehicle speed, represents the obstacle distance detected by the millimeter wave radar, represents the safety distance threshold, which is dynamically calculated according to the braking distance formula. The formula is: where represents the friction coefficient, represents the gravitational acceleration.

[0048] By setting the photo-taking trigger condition with the dual constraints of vertical acceleration and dynamic safety distance, it not only considers the vehicle body vibration characteristics in the high-speed scenario, but also monitors the obstacle distance in real time through the millimeter wave radar to ensure accurate triggering of photo-taking within the vehicle braking limit range, effectively improving the photo-taking efficiency and safety.

[0049] Furthermore, the analysis layer includes: constructing a joint feature vector according to the vibration spectrum , the spectral reflectance and the image ROI texture entropy ;

[0050] Furthermore, the vibration spectrum is extracted from the vehicle vibration data, the spectral reflectance is extracted from the captured image, and the image ROI texture entropy describes the degree of chaos of the pixel gray distribution;

[0051] Furthermore, the formula for the improved SVM decision function is:

[0052] ;

[0053] Among them, represents the decision function, which is used to judge which category the input sample belongs to. represents the class label. represents the Gaussian kernel function. represents the exponential function. represents controlling the attenuation rate of feature similarity. represents the th sample. represents the square of the Euclidean distance. represents the th Lagrange multiplier of the sample, which is optimized by the SMO algorithm. represents the th class label of the sample. represents the bias term, which is determined in the training phase. The training set contains 1000 groups of positive samples and 2000 groups of negative samples.

[0054] Furthermore, it is judged whether there is an obstacle actually according to the dynamic confidence. If the result of the improved SVM decision function processing the joint feature vector is within the dynamic confidence region, it is not a false trigger. The formula for the dynamic confidence is:

[0055] ;

[0056] Among them, represents the dynamic confidence. represents the accumulated value of the vibration feature frequency band energy. represents the spectral reflectance and the absolute difference from the road surface reference value . represents the exponential function. , and represent the weight coefficients.

[0057] By constructing the joint feature vector, the information provided by different sensors and image features can be fully utilized to improve the triggering accuracy; using the Gaussian kernel function can effectively process non-linear data, enabling the SVM model to better fit the complex data distribution, thereby optimizing the judgment accuracy.

[0058] Furthermore, the statistical layer updates the acceleration threshold by using the exponentially weighted moving average. The formula is:

[0059] ;

[0060] Among them, Represents The basic acceleration threshold at a moment, Represents the historical data decay factor, Represents The basic acceleration threshold at a moment, Represents the Corresponding average acceleration verified by the analysis layer within the moment.

[0061] By adopting the exponentially weighted moving average to update the acceleration threshold, dynamic adaptive threshold optimization can be achieved, adapting to environmental changes in real time, retaining historical threshold information through the historical decay factor, smoothing the influence of instantaneous noise, enhancing robustness, and reducing the false trigger rate.

[0062] Construct a pixel displacement constraint formula according to the vehicle speed data, and construct a shutter speed dynamic adjustment strategy according to the pixel displacement constraint formula; construct an image selection model according to the normalized gradient energy, signal-to-noise ratio, and ambiguity based on frequency domain analysis, and use the image selection model to select the first road surface image set;

[0063] Furthermore, the pixel displacement constraint formula is:

[0064] ;

[0065] Among them, Represents the maximum allowable exposure time to ensure that the pixel displacement is within the error tolerance range, Represents the pixel size of the camera sensor, Represents the current vehicle speed, Represents the optical magnification, Represents the angle between the camera optical axis and the road surface normal;

[0066] Furthermore, construct a shutter speed dynamic adjustment strategy according to the pixel displacement constraint formula, and the formula is:

[0067] ;

[0068] Among them Represents the shutter start time, Represents the minimum function, Represents the frame period, Represents the current vehicle speed.

[0069] Furthermore, the calculation formula of the imaging clarity index is:

[0070] ;

[0071] Among them, Represents the imaging clarity index of the image, Represents the normalized gradient energy The weight coefficient of Represents the image signal-to-noise ratio The weight coefficient of Represents the ambiguity based on frequency domain analysis The weight coefficient of

[0072] Furthermore, the normalized gradient energy refers to the mean of the gradients of all pixels, which is used to evaluate the overall sharpness. The blurriness based on frequency domain analysis refers to the proportion of low-frequency energy, and the blur intensity is evaluated by analyzing the energy distribution characteristics of the image in the frequency domain, such as after Fourier transform.

[0073] Further, taking photos according to the shutter speed dynamic adjustment strategy to obtain a road surface image set;

[0074] Furthermore, the images in the collected road surface image set are sorted according to the imaging clarity index, and a first road surface image set is constructed according to the images whose imaging clarity index is greater than a threshold.

[0075] The dynamic shutter speed adjustment strategy can accurately control the shutter time at different vehicle speeds, further suppress motion blur, and ensure the clarity of image details; through the imaging clarity index, a high-quality first road image set is screened out, significantly improving the subsequent input data quality; through shutter speed adjustment and image optimization, data redundancy and transmission costs are reduced, achieving coordinated optimization of image acquisition efficiency and accuracy in high-speed scenarios.

[0076] The structure of the motion blur removal model is as follows Figure 2 As shown, the first road surface image is first fused, and the fused image is deblurred according to the dynamic fuzzy core to obtain a clear image; the clear image is uploaded to the cloud, and the road surface detection model obtains image features, vibration spectrum features and spectral features and processes them to obtain the detection result.

[0077] Furthermore, the motion blur removal model includes an image fusion layer, a fused image input layer, a multi-scale codec layer and a clear image output layer; wherein the image fusion layer obtains and fuses the images in the first road image set to obtain a fused image, and the fusion formula is:

[0078] ;

[0079] in, Indicated in The fused pixel value at the position, Indicates that the main image is The pixel value at the position, Indicates the number of auxiliary images, Indicates Auxiliary images in The pixel value at the position, represents the pixel weight coefficient, and the calculation formula of the pixel weight coefficient is:

[0080] ;

[0081] Among them, represents the pixel weight function at the position, represents the natural base, represents the weight transition steepness coefficient, with a default value of 10, which controls the weight change rate between the edge area and the non-edge area, represents the local edge intensity of the main image at the position, calculated by the Sobel operator, represents the edge intensity threshold, calculated from the mean and standard deviation.

[0082] Furthermore, the multi-scale codec layer includes an encoder and a decoder. The encoder contains 4 levels of downsampling, each level containing a dynamic convolution block and a motion blur attention block. The decoder contains 4 levels of upsampling, each level containing a cross-scale skip link block and a non-local attention block;

[0083] Furthermore, in the motion blur attention block, a dynamic blur kernel is constructed according to the vehicle speed data, and the formula is:

[0084] ;

[0085] Among them, represents the dynamic blur kernel at the position in the fused image, represents the exposure time, represents the two-dimensional Dirac function, indicating the contribution of the instantaneous displacement to the blur kernel, represents the integration variable, and represent the components of the acceleration of the vehicle movement in the axis and axis; and represent the components of the vehicle speed in the axis and axis, calculated from the vehicle speed and camera parameters, and the formula is:

[0086] ;

[0087] Among them, represents the angle between the vehicle movement direction and the horizontal axis of the image, represents the pixel size of the camera sensor, represents the current vehicle speed, represents the camera focal length;

[0088] Furthermore, the coordinate axes are established according to the plane where the fused image is located, and the vehicle motion speed vector is also within this plane.

[0089] Furthermore, the clear image output layer receives the results output by the multi-scale codec and outputs a clear image.

[0090] Furthermore, a loss function is constructed based on the blur kernel matching loss and the gradient perception loss.

[0091] Table 1. Comparison of Key Metrics

[0092] Model Name PSNR SSIM Inference Speed (ms) Number of Parameters (M) Scene Applicability Model 1 29.1 0.918 22 60.9 Low-light Scenario Model 2 32.7 0.953 35 15.1 General Scenario Model 3 33.2 0.958 25 12.8 High Dynamic Range Scenario Motion Blur Removal Model 33.5 0.961 18 3.2 Full Scenario

[0093] As shown in Table 1, the key metrics of the motion blur removal model and other models are compared. PSNR and SSIM are used to measure the image restoration quality. First, the motion blur removal model enhances the image details through image fusion to obtain a clear fused image. Then, the motion blur removal model uses a multi-scale codec and a dynamic blur kernel to achieve image deblurring, which can effectively eliminate the non-uniform blur caused by vehicle speed and body jolts in high-speed scenarios, providing high-quality image data for road surface detection.

[0094] The structure of the road surface detection model is as Figure 3 shown, including a multi-modal input encoding layer, a cross-modal fusion layer, and a multi-task output layer. The multi-modal input encoding layer receives data and extracts image features, vibration spectrum features, and spectral features. The cross-modal fusion layer includes a cross-attention layer and a dynamic gating fusion layer for feature fusion. The multi-task output layer includes a segmentation head for outputting the segmentation mask of the object to be detected, a classification head for outputting the class probability of the object to be detected, and a regression head for outputting the size and depth of the object to be detected.

[0095] A joint loss function is constructed based on the segmentation loss, classification loss, and regression loss.

[0096] Table 2. Comparison of Key Metrics

[0097] Model Name F1-Score Multi-modal Support Multi-task Output Number of Parameters (M) Scene Applicability Model S1 0.85 Not Supported Only Detection 7.3 Lighting Stable Scenario Model S2 0.92 Not Supported Detection + Segmentation 44.3 Full Scenario Model S3 0.88 Not Supported Only Detection 6.8 High-speed Scenario Road Surface Detection Model 0.96 Not Supported Detection + Classification + Regression 3.2 Full Scenario

[0098] As shown in Table 2, the key metrics of the road surface detection model and other models are compared, and it can be seen that it is superior to other models. The road surface detection model first extracts features through the multi-modal input encoding layer to achieve efficient data processing. Then, it realizes the fusion and complementarity of multi-source data through the cross-modal fusion layer, reducing the misjudgment rate. Finally, through the multi-task collaborative output of the segmentation head, classification head, and regression head, it realizes accurate recognition, providing a high-precision and low-latency road surface recognition model for intelligent highway maintenance.

[0099] Through the coordination of the dynamic photo-triggering mechanism and the dynamic shutter speed adjustment strategy, the real-time performance and clarity of image acquisition in high-speed scenarios are ensured; high-quality images are screened based on the imaging clarity index, and the motion blur removal model fuses and removes the blur from the images, achieving the restoration of image details; finally, the road surface detection model in the cloud extracts, fuses, and analyzes the features of multi-modal data, realizing accurate classification and evaluation, and improving the efficiency of road detection.

[0100] Embodiment 2

[0101] A certain company introduced a road surface automatic rapid detection method based on high-speed camera technology provided by the present invention during the road surface defect detection to improve the detection efficiency. The specific implementation method is as follows:

[0102] The dynamic photo-triggering mechanism includes an emergency layer, an analysis layer, and a statistics layer; among them, the emergency layer is used to construct a triggering condition based on the information collected by the sensor to trigger the high-speed camera to take a photo; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to determine whether the emergency layer is mis-triggered; the statistics layer dynamically adjusts the triggering threshold based on historical data and data in the most recent time window; a pixel displacement constraint formula is constructed according to the vehicle speed data, and a dynamic shutter speed adjustment strategy is constructed based on the pixel displacement constraint formula;

[0103] Further, the formula for the triggering condition of the emergency layer is:

[0104] ;

[0105] Wherein, represents the triggering condition, represents the vertical acceleration measured by the triaxial accelerometer, represents the basic acceleration threshold, represents the vehicle speed influence factor, represents the current vehicle speed, represents the obstacle distance detected by the millimeter-wave radar, represents the safety distance threshold.

[0106] Further, the analysis layer includes: constructing a joint feature vector according to the vibration spectrum , the spectral reflectance and the image ROI texture entropy ;

[0107] If the result of processing the joint feature vector by the improved SVM decision function is within the dynamic confidence region, it is not a mis-trigger. The formula for the dynamic confidence is:

[0108] ;

[0109] Wherein, represents the dynamic confidence level, represents the cumulative value of the energy in the vibration characteristic frequency band, represents the spectral reflectance and the absolute difference from the road surface reference value is represents the exponential function, , and represents the weight coefficient.

[0110] Furthermore, the statistical layer updates the acceleration threshold using the exponentially weighted moving average, and the formula is:

[0111] ;

[0112] where represents the basic acceleration threshold at time represents the historical data decay factor, represents the basic acceleration threshold at time represents the corresponding average acceleration verified by the analysis layer within the

[0113] Table 3 shows some of the data for triggering the camera, and it can be seen that the high-speed camera can be triggered quickly.

[0114] Detection Time Whether to Trigger Photographing Trigger Time 2024-10-05 14:22:15 Yes 10 2024-10-05 14:22:17 No - 2024-10-05 14:22:19 Yes 20 2024-10-05 14:22:21 Yes 35

[0115] Furthermore, a dynamic shutter speed adjustment strategy is constructed according to the pixel displacement constraint formula, and the pixel displacement constraint formula is:

[0116] ;

[0117] where represents the maximum allowable exposure time to ensure that the pixel displacement is within the error tolerance range, represents the pixel size of the camera sensor, represents the current vehicle speed, represents the optical magnification, represents the angle between the camera optical axis and the road surface normal;

[0118] Taking pictures according to the dynamic shutter speed adjustment strategy, a set of road surface images is obtained;

[0119] Construct an imaging clarity index based on the normalized gradient energy, signal-to-noise ratio, and blur degree based on frequency domain analysis, and use the imaging clarity index to construct the first set of road surface images;

[0120] Further, sort the images in the collected road surface image set according to the imaging clarity index, and construct a first road surface image set based on the images with the imaging clarity index greater than the threshold.

[0121] The motion blur removal model fuses the first road surface images, and removes the blur from the fused images according to the dynamic blur kernel to obtain clear images; upload the clear images to the cloud, and the road surface detection model obtains image features, vibration spectrum features and spectral features and processes them to obtain detection results.

[0122] Further, the motion blur removal model includes an image fusion layer, a fused image input layer, a multi-scale encoder-decoder layer and a clear image output layer; wherein the image fusion layer obtains the images in the first road surface image set and fuses them to obtain fused images;

[0123] The multi-scale encoder-decoder layer includes an encoder and a decoder. Among them, the encoder contains 4 levels of downsampling, each level contains a dynamic convolution block and a motion blur attention block, and the decoder contains 4 levels of upsampling, each level contains a cross-scale skip link block and a non-local attention block;

[0124] A dynamic blur kernel is constructed according to the vehicle speed data in the motion blur attention block, and the formula is:

[0125] ;

[0126] Among them, represents the dynamic blur kernel at the position in the fused image, represents the exposure time, represents the two-dimensional Dirac function, indicating the contribution of the instantaneous displacement to the blur kernel, and represent the components of the vehicle speed in the axis and the axis, and represent the components of the acceleration of the vehicle motion in the axis and the axis, represents the integration variable;

[0127] Construct a loss function according to the blur kernel matching loss and the gradient perception loss.

[0128] Furthermore, the road surface detection model includes a multi-modal input encoding layer, a cross-modal fusion layer, and a multi-task output layer. The multi-modal input encoding layer receives data and extracts image features, vibration spectrum features, and spectral features. The cross-modal fusion layer includes a cross-attention layer and a dynamic gating fusion layer for fusing the features. The multi-task output layer includes a segmentation head for outputting the segmentation mask of the object to be detected, a classification head for outputting the class probability of the object to be detected, and a regression head for outputting the size and depth of the object to be detected.

[0129] A joint loss function is constructed based on the segmentation loss, classification loss, and regression loss.

[0130] The method provided by the present invention realizes rapid road surface detection, improves the detection efficiency, and shortens the detection time.

[0131] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for rapid automatic road surface detection based on high-speed camera technology, characterized in that: include: Determine whether to take a photo according to the dynamic photo trigger mechanism, which includes an emergency layer, an analysis layer, and a statistical layer; The emergency layer is used to construct trigger conditions based on the information collected by the sensor to trigger the high-speed camera to take pictures; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to determine whether the emergency layer is falsely triggered; the statistical layer dynamically adjusts the trigger threshold based on historical data and data in the most recent time window; after triggering the photo, a pixel displacement constraint formula is constructed based on the vehicle speed data, and a shutter speed dynamic adjustment strategy is constructed based on the pixel displacement constraint formula to take pictures, and a road image set is obtained. The pixel displacement constraint formula is: ； in, Indicates the maximum allowed exposure time, Indicates the camera sensor pixel size, Indicates the current vehicle speed. Indicates the optical magnification, Indicates the angle between the camera optical axis and the road surface normal; constructing an imaging clarity index according to normalized gradient energy, signal-to-noise ratio, and ambiguity based on frequency domain analysis, sorting images in the collected road image set according to the imaging clarity index, and constructing a first road image set according to images whose imaging clarity index is greater than a threshold; The motion blur removal model fuses the first road surface image and removes blur from the fused image according to the dynamic blur kernel to obtain a clear image; the clear image is uploaded to the cloud, and the road surface detection model obtains and processes the image features, vibration spectrum features and spectral features to obtain the detection results.

2. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The formula for the triggering condition of the emergency layer is: ； in, Indicates the trigger condition. represents the vertical acceleration measured by the triaxial accelerometer, Indicates the basic acceleration threshold, represents the vehicle speed influence factor, Indicates the current vehicle speed. Indicates the distance of obstacles detected by the millimeter-wave radar. Indicates the safety distance threshold.

3. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The analysis layer includes: according to the vibration spectrum , spectral reflectance And image ROI texture entropy Constructing joint feature vector ; If the result of the improved SVM decision function processing the joint feature vector is within the dynamic confidence region, it is not a false trigger. The formula for dynamic confidence is: ； in, represents the dynamic confidence, Indicates the energy accumulation value of the vibration characteristic frequency band. Represents spectral reflectance Road surface reference value The absolute difference represents the exponential function, , and Represents the weight coefficient.

4. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The statistical layer uses exponentially weighted moving average to update the acceleration threshold, and the formula is: ； in, express The basic acceleration threshold at the moment, represents the historical data attenuation factor, express The basic acceleration threshold at the moment, Indicates The corresponding acceleration mean value verified by the analysis layer within the time.

5. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The motion blur removal model includes an image fusion layer, a fused image input layer, a multi-scale codec layer and a clear image output layer; wherein the image fusion layer obtains and fuses images in the first road image set to obtain a fused image; The multi-scale codec layer includes an encoder and a decoder, where the encoder contains 4 levels of downsampling, each level contains a dynamic convolution block and a motion blur attention block, and the decoder contains 4 levels of upsampling, each level contains a cross-scale skip link block and a non-local attention block; In the motion blur attention block, the dynamic blur kernel is constructed according to the vehicle speed data. The formula is: ； in, Indicates that in the fused image The dynamic blur kernel on the position, Indicates the exposure time, represents the two-dimensional Dirac function, and Indicates vehicle speed Axis and The weight of the axis, and The acceleration of the vehicle is Axis and The weight of the axis, represents the integral variable; The loss function is constructed based on blur kernel matching loss and gradient-aware loss.

6. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The road surface detection model includes a multimodal input encoding layer, a cross-modal fusion layer, and a multi-task output layer, wherein the multimodal input encoding layer receives data and extracts image features, vibration spectrum features, and spectral features; The cross-modal fusion layer includes a cross-attention layer and a dynamic gated fusion layer for fusing features; the multi-task output layer includes a segmentation head for outputting the segmentation mask of the object to be detected, a classification head for outputting the category probability of the object to be detected, and a regression head for outputting the size and depth of the object to be detected; A joint loss function is constructed based on segmentation loss, classification loss and regression loss.

Images