Automatic rapid road surface detection method based on high-speed camera technology

Abstract

The invention relates to the technical field of image analysis, in particular to an automatic rapid road surface detection method based on a high-speed camera technology, which comprises the following steps: firstly, an emergency layer in a dynamic photographing triggering mechanism triggers a high-speed camera to photograph according to information acquired by a sensor; the analysis layer constructs a joint feature vector and judges whether the emergency layer is mistakenly triggered by using an improved SVM decision function; the statistical layer dynamically adjusts a trigger threshold value; constructing a pixel displacement constraint formula according to the vehicle speed data, constructing a shutter speed dynamic adjustment strategy according to the pixel displacement constraint formula, performing photographing, and constructing a first pavement image set by using an imaging clearness index; fusing the images by the motion blur removal model, and removing blur of the fused image according to a dynamic blur kernel to obtain a clear image; and the road surface detection model obtains and processes the image features, the vibration spectrum features and the spectrum features to obtain a detection result. Through the method provided by the invention, the road surface automatic detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to an automatic rapid road surface detection method based on high-speed camera technology. Background Art

[0002] As the scale of transportation infrastructure continues to expand, automated detection technology for road health has become a key requirement for improving road maintenance efficiency and reducing accident risks. Traditional detection methods mostly rely on manual inspections or low-frame-rate camera equipment, which have problems such as low efficiency, limited coverage, and poor adaptability to dynamic scenes. In recent years, high-speed camera technology has provided a hardware foundation for road image acquisition under high-speed moving conditions with its high frame rate and high resolution characteristics, but the resulting massive image data poses a severe challenge to real-time processing and analysis capabilities.

[0003] In the field of image data processing, traditional methods usually use algorithms such as threshold segmentation, edge detection or template matching for image analysis. However, such methods are significantly limited in high-speed dynamic scenes: first, the algorithm relies on artificially designed features such as texture gradients or geometric shapes, and it is difficult to distinguish between real defects and interference factors such as road stains and shadows, resulting in an increase in the false detection rate; second, the pixel-by-pixel traversal calculation process is difficult to meet the real-time requirements of high-speed data streams, and the fixed parameter setting cannot adapt to different road materials such as asphalt and concrete or changes in environmental conditions, and the generalization ability is insufficient. Although deep learning models have shown advantages in static image defect recognition, high-precision network structures usually rely on high-performance computing devices. When they are directly applied to high-speed scenes, they face the contradiction between model complexity and computing resources, and are difficult to deploy on vehicle-mounted embedded platforms. In addition, the multispectral or high dynamic range imaging data supported by high-speed cameras have not been fully explored, such as the sensitivity of the near-infrared band to the seepage area or the motion correlation between time series frames. The existing algorithms are prone to ignoring small defects or misjudging instantaneous noise due to the lack of multimodal fusion and time dimension modeling.

[0004] Therefore, an automatic and rapid road surface detection method based on high-speed camera technology is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide a method for rapid automatic road surface detection based on high-speed camera technology to improve the efficiency of automatic road surface detection. First, the emergency layer in the dynamic photo triggering mechanism triggers the high-speed camera to take photos according to the information collected by the sensor; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to determine whether the emergency layer is falsely triggered; the statistical layer dynamically adjusts the trigger threshold; then, a pixel displacement constraint formula is constructed according to the vehicle speed data, a shutter speed dynamic adjustment strategy is constructed according to the pixel displacement constraint formula and photos are taken, and then the first road surface image set is constructed using the imaging clarity index; the motion blur removal model fuses the images, and the fused images are deblurred according to the dynamic blur check to obtain a clear image; the road surface detection model obtains image features, vibration spectrum features and spectrum features and processes them to obtain detection results.

[0006] To achieve the above object, the present invention provides the following technical solutions: A road surface automatic rapid detection method based on high-speed camera technology, comprising: The dynamic photo triggering mechanism includes an emergency layer, an analysis layer and a statistical layer; the emergency layer is used to construct trigger conditions based on the information collected by the sensor to trigger the high-speed camera to take photos; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to determine whether the emergency layer is falsely triggered; the statistical layer dynamically adjusts the trigger threshold based on historical data and data in the most recent time window; a pixel displacement constraint formula is constructed based on vehicle speed data, and a shutter speed dynamic adjustment strategy is constructed based on the pixel displacement constraint formula; Furthermore, the formula for the triggering condition of the emergency layer is: ; in, Indicates the trigger condition. represents the vertical acceleration measured by the triaxial accelerometer, Indicates the basic acceleration threshold, represents the vehicle speed impact factor, Indicates the current vehicle speed. Indicates the obstacle distance detected by the millimeter-wave radar. Indicates the safety distance threshold.

[0007] Furthermore, the analysis layer includes: according to the vibration spectrum , spectral reflectance And image ROI texture entropy Constructing joint feature vector ; If the result of the improved SVM decision function processing the joint feature vector is within the dynamic confidence region, it is not a false trigger. The formula for dynamic confidence is: ; in, represents the dynamic confidence, Indicates the energy accumulation value of the vibration characteristic frequency band. Represents spectral reflectance Road surface reference value The absolute difference represents the exponential function, , and Represents the weight coefficient.

[0008] Furthermore, the statistical layer uses exponentially weighted moving average to update the acceleration threshold, and the formula is: ; in, express The basic acceleration threshold at the moment, represents the historical data attenuation factor, express The basic acceleration threshold at the moment, Indicates The corresponding acceleration mean value verified by the analysis layer within the time.

[0009] Furthermore, the shutter speed dynamic adjustment strategy is constructed according to the pixel displacement constraint formula, and the pixel displacement constraint formula is: ; in, Indicates the maximum allowed exposure time, Indicates the camera sensor pixel size, Indicates the current vehicle speed. Indicates the optical magnification, Indicates the angle between the camera optical axis and the road surface normal; Taking photos according to the shutter speed dynamic adjustment strategy to obtain a road surface image set; The images in the collected road surface image set are sorted according to the imaging clarity index, and a first road surface image set is constructed according to the images whose imaging clarity index is greater than a threshold.

[0010] constructing an imaging clarity index according to normalized gradient energy, a signal-to-noise ratio, and ambiguity based on frequency domain analysis, and constructing a first road surface image set using the imaging clarity index; The motion blur removal model fuses the first road surface image and removes blur from the fused image according to the dynamic blur kernel to obtain a clear image; the clear image is uploaded to the cloud, and the road surface detection model obtains and processes the image features, vibration spectrum features and spectral features to obtain the detection results.

[0011] Furthermore, the motion blur removal model includes an image fusion layer, a fused image input layer, a multi-scale codec layer and a clear image output layer; wherein the image fusion layer obtains and fuses images in the first road surface image set to obtain a fused image; The multi-scale codec layer includes an encoder and a decoder, where the encoder contains 4 levels of downsampling, each level contains a dynamic convolution block and a motion blur attention block, and the decoder contains 4 levels of upsampling, each level contains a cross-scale skip link block and a non-local attention block; In the motion blur attention block, the dynamic blur kernel is constructed according to the vehicle speed data. The formula is: ; in, Indicates that in the fused image The dynamic blur kernel on the position, Indicates the exposure time, represents the two-dimensional Dirac function, and Indicates vehicle speed Axis and The weight of the axis, and The acceleration of the vehicle is Axis and The weight of the axis, represents the integral variable; The loss function is constructed based on blur kernel matching loss and gradient-aware loss.

[0012] Furthermore, the road surface detection model includes a multimodal input encoding layer, a cross-modal fusion layer and a multi-task output layer, wherein the multimodal input encoding layer receives data and extracts image features, vibration spectrum features and spectral features; the cross-modal fusion layer includes a cross-attention layer and a dynamic gated fusion layer for fusing features; the multi-task output layer includes a segmentation head for outputting a segmentation mask of an object to be detected, a classification head for outputting a category probability of an object to be detected, and a regression head for outputting a size and depth of an object to be detected; A joint loss function is constructed based on segmentation loss, classification loss and regression loss.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. The emergency layer in the dynamic photo trigger mechanism can achieve fast triggering and low-latency response; the analysis layer constructs a joint feature vector for analysis, which greatly reduces the false trigger rate; the statistical layer adapts to real-time changes in road conditions by updating the acceleration threshold; the dynamic photo trigger mechanism realizes automatic triggering of high-speed cameras through a logical closed loop of response-verification-feedback adjustment, thereby improving the accuracy of triggering photos.

[0014] 2. Constructing a pixel displacement constraint formula based on vehicle speed data and dynamically adjusting the shutter speed can effectively eliminate motion blur in high-speed moving scenarios and ensure clear details of road images captured at different vehicle speeds. At the same time, the imaging clarity index constructed by combining normalized gradient energy, signal-to-noise ratio and frequency domain blur can quantitatively evaluate each frame image in a multi-exposure image set and adaptively select the optimal image subset, thereby significantly improving the quality of input data.

[0015] 3. The motion blur removal model achieves high-quality image fusion and blur removal through weighted fusion, pixel weight coefficient and dynamic blur kernel, providing clear images for road surface detection; the road surface detection model obtains clear road surface images and combines data from other modes for multimodal analysis, ultimately achieving accurate detection of road surface conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a method for automatic rapid detection of a road surface based on high-speed camera technology provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a motion blur removal model provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a road surface detection model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Embodiment 1 When a company was conducting defect inspection on a newly built highway pavement, in order to improve the inspection accuracy and shorten the inspection time, it introduced a road surface automatic rapid inspection method based on high-speed camera technology provided by the present invention. The method flow is as follows: Figure 1 As shown, the specific implementation is as follows: The dynamic photo triggering mechanism includes an emergency layer, an analysis layer and a statistical layer; wherein the emergency layer is used to construct trigger conditions based on the information collected by the sensor to trigger the high-speed camera to take photos; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to determine whether the emergency layer is falsely triggered; the statistical layer dynamically adjusts the trigger threshold based on historical data and data in the most recent time window; Furthermore, the formula for the triggering condition of the emergency layer is: ; in, Indicates the trigger condition. represents the vertical acceleration measured by the triaxial accelerometer, Indicates the basic acceleration threshold, represents the vehicle speed impact factor, Indicates the current vehicle speed. Indicates the obstacle distance detected by the millimeter-wave radar. Indicates the safety distance threshold, which is dynamically calculated based on the braking distance formula: ,in represents the friction coefficient, Represents the acceleration due to gravity.

[0019] The photo triggering conditions are set through the dual constraints of vertical acceleration and dynamic safety distance, which not only takes into account the vibration characteristics of the vehicle body in high-speed scenarios, but also monitors the distance to obstacles in real time through millimeter-wave radar to ensure accurate triggering of photography within the vehicle's braking limit, effectively improving photography efficiency and safety.

[0020] Furthermore, the analysis layer includes: according to the vibration spectrum , spectral reflectance And image ROI texture entropy Constructing joint feature vector ; Furthermore, the vibration spectrum is extracted from the vehicle vibration data, the spectral reflectance is extracted from the captured image, and the image ROI texture entropy describes the degree of disorder of the pixel grayscale distribution; Furthermore, the formula of the improved SVM decision function is: ; in, Represents a decision function, used to judge the input sample Which category does it belong to? represents the category label, represents the Gaussian kernel function, represents the exponential function, Indicates the control feature similarity decay rate, Indicates samples, represents the square of the Euclidean distance, Indicates The Lagrange multiplier of samples is obtained by optimizing the SMO algorithm. Indicates The class labels of samples, Represents the bias term, which is determined by the training phase, and the training set Contains 1000 groups of positive samples and 2000 groups of negative samples; Furthermore, the dynamic confidence is used to determine whether an obstacle actually exists. If the result of the improved SVM decision function processing the joint feature vector is within the dynamic confidence region, it is not a false trigger. The formula for the dynamic confidence is: ; in, represents the dynamic confidence, Indicates the energy accumulation value of the vibration characteristic frequency band. Represents spectral reflectance Road surface reference value The absolute difference represents the exponential function, , and Represents the weight coefficient.

[0021] By constructing a joint feature vector, we can make full use of the information provided by different sensors and image features to improve the accuracy of triggering; using the Gaussian kernel function can effectively process nonlinear data, so that the SVM model can better fit the complex data distribution, thereby optimizing the accuracy of judgment.

[0022] Furthermore, the statistical layer uses exponentially weighted moving average to update the acceleration threshold, and the formula is: ; in, express The basic acceleration threshold at the moment, represents the historical data attenuation factor, express The basic acceleration threshold at the moment, Indicates The corresponding acceleration mean value verified by the analysis layer within the time.

[0023] By using exponentially weighted moving average to update the acceleration threshold, dynamic adaptive threshold optimization can be achieved, which can adapt to environmental changes in real time, retain historical threshold information through historical attenuation factors, smooth the impact of instantaneous noise, improve robustness, and reduce false triggering rate.

[0024] A pixel displacement constraint formula is constructed according to the vehicle speed data, and a shutter speed dynamic adjustment strategy is constructed according to the pixel displacement constraint formula; an image selection model is constructed according to normalized gradient energy, signal-to-noise ratio, and blur based on frequency domain analysis, and a first road surface image set is selected using the image selection model; Furthermore, the pixel displacement constraint formula is: ; in, Indicates the maximum allowable exposure time to ensure that the pixel displacement is within the allowable error range. Indicates the camera sensor pixel size, Indicates the current vehicle speed. Indicates the optical magnification, Indicates the angle between the camera optical axis and the road surface normal; Furthermore, a shutter speed dynamic adjustment strategy is constructed based on the pixel displacement constraint formula, which is: ; in Indicates the shutter start time. It means taking a small function. represents the frame period, Indicates the current vehicle speed.

[0025] Furthermore, the calculation formula of the imaging clarity index is: ; in, Indicates the image clarity index. represents the normalized gradient energy The weight coefficient of Represents the image signal-to-noise ratio The weight coefficient of Represents the ambiguity based on frequency domain analysis The weight coefficient of Furthermore, the normalized gradient energy refers to the mean of the gradients of all pixels, which is used to evaluate the overall sharpness. The blurriness based on frequency domain analysis refers to the proportion of low-frequency energy, and the blur intensity is evaluated by analyzing the energy distribution characteristics of the image in the frequency domain, such as after Fourier transform.

[0026] Further, taking photos according to the shutter speed dynamic adjustment strategy to obtain a road surface image set; Furthermore, the images in the collected road surface image set are sorted according to the imaging clarity index, and a first road surface image set is constructed according to the images whose imaging clarity index is greater than a threshold.

[0027] The dynamic shutter speed adjustment strategy can accurately control the shutter time at different vehicle speeds, further suppress motion blur, and ensure the clarity of image details; through the imaging clarity index, a high-quality first road image set is screened out, significantly improving the subsequent input data quality; through shutter speed adjustment and image optimization, data redundancy and transmission costs are reduced, achieving coordinated optimization of image acquisition efficiency and accuracy in high-speed scenarios.

[0028] The structure of the motion blur removal model is as follows Figure 2As shown, the first road surface image is first fused, and the fused image is deblurred according to the dynamic fuzzy core to obtain a clear image; the clear image is uploaded to the cloud, and the road surface detection model obtains image features, vibration spectrum features and spectral features and processes them to obtain the detection result.

[0029] Furthermore, the motion blur removal model includes an image fusion layer, a fused image input layer, a multi-scale codec layer and a clear image output layer; wherein the image fusion layer obtains and fuses the images in the first road image set to obtain a fused image, and the fusion formula is: ; in, Indicated in The fused pixel value at the position, Indicates that the main image is The pixel value at the position, Indicates the number of auxiliary images, Indicates Auxiliary images in The pixel value at the position, Represents the pixel weight coefficient, and the calculation formula of the pixel weight coefficient is: ; in, Indicated in The pixel weight function at position, represents the natural base, Represents the weight transition steepness coefficient, the default value is 10, which controls the weight change rate of the edge area and non-edge area. Indicates that the main image is The local edge strength at the position is calculated by the Sobel operator. Represents the edge strength threshold, calculated by the mean and standard deviation.

[0030] Furthermore, the multi-scale codec layer includes an encoder and a decoder, wherein the encoder includes 4 levels of downsampling, each level includes a dynamic convolution block and a motion blur attention block, and the decoder includes 4 levels of upsampling, each level includes a cross-scale skip link block and a non-local attention block; Furthermore, the motion blur attention block constructs a dynamic blur kernel based on the vehicle speed data, and the formula is: ; in, Indicates that in the fused image The dynamic blur kernel on the position, Indicates the exposure time, represents the two-dimensional Dirac function, which represents the contribution of instantaneous displacement to the blur kernel, represents the integral variable, and The acceleration of the vehicle is Axis and The weight of the axis; and Indicates vehicle speed Axis and The axis components are calculated from the vehicle speed and camera parameters using the formula: ; in, Represents the angle between the vehicle's moving direction and the image's horizontal axis, Indicates the camera sensor pixel size, Indicates the current vehicle speed. Indicates the focal length of the camera; Furthermore, the coordinate axis is established based on the plane where the fused image is located, and the vehicle motion velocity vector is also in this plane.

[0031] Furthermore, the clear image output layer receives the result output by the multi-scale codec and outputs a clear image; Furthermore, a loss function is constructed based on blur kernel matching loss and gradient-aware loss.

[0032] Table 1. Comparison of key indicators Model Name PSNR SSIM Inference speed (ms) Parameter quantity (M) Scenario Applicability Model 1 29.1 0.918 22 60.9 Low-light scenes Model 2 32.7 0.953 35 15.1 General scenarios Model 3 33.2 0.958 25 12.8 High dynamic scene Motion blur removal model 33.5 0.961 18 3.2 Full scene Table 1 shows the key indicator comparison between the motion blur removal model and other models. PSNR and SSIM are used to measure the quality of image restoration. First, the motion blur removal model enhances image details through image fusion to obtain a clear fused image. Then, the motion blur removal model uses a multi-scale codec and a dynamic blur kernel to achieve image deblurring, which can effectively eliminate the non-uniform blur caused by vehicle speed and vehicle body bumps in high-speed scenes, and provide high-quality image data for road surface detection.

[0033] The structure of the road surface detection model is as follows: Figure 3 As shown, it includes a multimodal input encoding layer, a cross-modal fusion layer and a multi-task output layer, wherein the multimodal input encoding layer receives data and extracts image features, vibration spectrum features and spectral features; the cross-modal fusion layer includes a cross-attention layer and a dynamic gated fusion layer for fusing features; the multi-task output layer includes a segmentation head for outputting a segmentation mask of an object to be detected, a classification head for outputting a category probability of an object to be detected, and a regression head for outputting a size and depth of an object to be detected; A joint loss function is constructed based on segmentation loss, classification loss and regression loss.

[0034] Table 2. Comparison of key indicators Model Name F1-Score Multimodal support Multi-task output Parameter quantity (M) Scenario Applicability Model S1 0.85 Not supported Detection only 7.3 Lighting stability scene Model S2 0.92 Not supported Detection + Segmentation 44.3 Full scene Model S3 0.88 Not supported Detection only 6.8 High-speed scene Road surface detection model 0.96 Not supported Detection + Classification + Regression 3.2 Full scene Table 2 shows the key indicators of the road surface detection model compared with other models. It can be seen that the road surface detection model is superior to other models. The road surface detection model first extracts features through the multimodal input encoding layer to achieve efficient data processing; then it realizes multi-source data fusion and complementation through the cross-modal fusion layer to reduce the misjudgment rate; finally, the multi-task collaborative output of the segmentation head, classification head and regression head is used to achieve accurate recognition, providing a high-precision and low-latency road surface recognition model for intelligent highway maintenance.

[0035] Through the coordination of dynamic photo triggering mechanism and dynamic shutter speed adjustment strategy, the real-time and clarity of image acquisition in high-speed scenarios are ensured; the imaging clarity index selects high-quality images, and the motion blur removal model fuses and deblurs the images to achieve image detail recovery; finally, the cloud-based road surface detection model achieves accurate classification and evaluation through feature extraction, fusion and analysis of multimodal data, thereby improving the efficiency of highway detection.

[0036] Embodiment 2

[0037] A company introduced a road surface automatic rapid detection method based on high-speed camera technology provided by the present invention when conducting road surface defect detection on highways to improve detection efficiency. The specific implementation method is as follows: The dynamic photo triggering mechanism includes an emergency layer, an analysis layer and a statistical layer; the emergency layer is used to construct trigger conditions based on the information collected by the sensor to trigger the high-speed camera to take photos; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to determine whether the emergency layer is falsely triggered; the statistical layer dynamically adjusts the trigger threshold based on historical data and data in the most recent time window; a pixel displacement constraint formula is constructed based on vehicle speed data, and a shutter speed dynamic adjustment strategy is constructed based on the pixel displacement constraint formula; Furthermore, the formula for the triggering condition of the emergency layer is: ; in, Indicates the trigger condition. represents the vertical acceleration measured by the triaxial accelerometer, Indicates the basic acceleration threshold, represents the vehicle speed impact factor, Indicates the current vehicle speed. Indicates the obstacle distance detected by the millimeter-wave radar. Indicates the safety distance threshold.

[0038] Furthermore, the analysis layer includes: according to the vibration spectrum , spectral reflectance And image ROI texture entropy Constructing joint feature vector ; If the result of the improved SVM decision function processing the joint feature vector is within the dynamic confidence region, it is not a false trigger. The formula for dynamic confidence is: ; in, represents the dynamic confidence, Indicates the energy accumulation value of the vibration characteristic frequency band. Represents spectral reflectance Road surface reference value The absolute difference represents the exponential function, , and Represents the weight coefficient.

[0039] Furthermore, the statistical layer uses exponentially weighted moving average to update the acceleration threshold, and the formula is: ; in, express The basic acceleration threshold at the moment, represents the historical data attenuation factor, express The basic acceleration threshold at the moment, Indicates The corresponding acceleration mean value verified by the analysis layer within the time. Table 3 shows the data of partial triggering of photography, which shows that the high-speed camera can be quickly triggered to take photos.

[0040] Table 3. Some triggered photo data Detection time Whether to trigger photo taking Trigger time 2024-10-05 14:22:15 yes 10 2024-10-05 14:22:17 no - 2024-10-05 14:22:19 yes 20 2024-10-05 14:22:21 yes 35 Furthermore, a shutter speed dynamic adjustment strategy is constructed based on the pixel displacement constraint formula. The pixel displacement constraint formula is: ; in, Indicates the maximum allowable exposure time to ensure that the pixel displacement is within the allowable error range. Indicates the camera sensor pixel size, Indicates the current vehicle speed. Indicates the optical magnification, Indicates the angle between the camera optical axis and the road surface normal; Taking photos according to the shutter speed dynamic adjustment strategy to obtain a road surface image set; constructing an imaging clarity index according to normalized gradient energy, a signal-to-noise ratio, and ambiguity based on frequency domain analysis, and constructing a first road surface image set using the imaging clarity index; Furthermore, the images in the collected road surface image set are sorted according to the imaging clarity index, and a first road surface image set is constructed according to the images whose imaging clarity index is greater than a threshold.

[0041] The motion blur removal model fuses the first road surface image and removes blur from the fused image according to the dynamic blur kernel to obtain a clear image; the clear image is uploaded to the cloud, and the road surface detection model obtains and processes the image features, vibration spectrum features and spectral features to obtain the detection results.

[0042] Furthermore, the motion blur removal model includes an image fusion layer, a fused image input layer, a multi-scale codec layer and a clear image output layer; wherein the image fusion layer obtains and fuses the images in the first road image set to obtain a fused image; The multi-scale codec layer includes an encoder and a decoder, where the encoder contains 4 levels of downsampling, each level contains a dynamic convolution block and a motion blur attention block, and the decoder contains 4 levels of upsampling, each level contains a cross-scale skip link block and a non-local attention block; In the motion blur attention block, the dynamic blur kernel is constructed according to the vehicle speed data. The formula is: ; in, Indicates that in the fused image The dynamic blur kernel on the position, Indicates the exposure time, represents the two-dimensional Dirac function, which represents the contribution of instantaneous displacement to the blur kernel, and Indicates vehicle speed Axis and The weight of the axis, and The acceleration of the vehicle is Axis and The weight of the axis, represents the integral variable; The loss function is constructed based on blur kernel matching loss and gradient-aware loss.

[0043] Furthermore, the road surface detection model includes a multimodal input encoding layer, a cross-modal fusion layer and a multi-task output layer, wherein the multimodal input encoding layer receives data and extracts image features, vibration spectrum features and spectral features; the cross-modal fusion layer includes a cross-attention layer and a dynamic gated fusion layer for fusing features; the multi-task output layer includes a segmentation head for outputting a segmentation mask of an object to be detected, a classification head for outputting a category probability of an object to be detected, and a regression head for outputting a size and depth of an object to be detected; A joint loss function is constructed based on segmentation loss, classification loss and regression loss.

[0044] The method provided by the present invention realizes rapid road surface detection, improves detection efficiency, and shortens detection time.

[0045] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for rapid automatic road surface detection based on high-speed camera technology, characterized in that: include: Determine whether to take a photo according to the dynamic photo trigger mechanism, which includes an emergency layer, an analysis layer, and a statistical layer; The emergency layer is used to construct trigger conditions based on the information collected by the sensor to trigger the high-speed camera to take pictures; the analysis layer constructs a joint feature vector and uses an improved SVM decision function to determine whether the emergency layer is falsely triggered; the statistical layer dynamically adjusts the trigger threshold based on historical data and data in the most recent time window; after triggering the photo, a pixel displacement constraint formula is constructed based on the vehicle speed data, and a shutter speed dynamic adjustment strategy is constructed based on the pixel displacement constraint formula to take pictures, thereby obtaining a road surface image set; constructing an imaging clarity index according to normalized gradient energy, a signal-to-noise ratio, and ambiguity based on frequency domain analysis, and processing a road surface image set using the imaging clarity index to obtain a first road surface image set; The motion blur removal model fuses the first road surface image and removes blur from the fused image according to the dynamic blur kernel to obtain a clear image; the clear image is uploaded to the cloud, and the road surface detection model obtains and processes the image features, vibration spectrum features and spectral features to obtain the detection results.

2. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The formula for the triggering condition of the emergency layer is: ； in, Indicates the trigger condition. represents the vertical acceleration measured by the triaxial accelerometer, Indicates the basic acceleration threshold, represents the vehicle speed impact factor, Indicates the current vehicle speed. Indicates the distance of obstacles detected by the millimeter-wave radar. Indicates the safety distance threshold.

3. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The analysis layer includes: according to the vibration spectrum , spectral reflectance And image ROI texture entropy Constructing joint feature vector ; If the result of the improved SVM decision function processing the joint feature vector is within the dynamic confidence region, it is not a false trigger. The formula for dynamic confidence is: ； in, represents the dynamic confidence, Indicates the energy accumulation value of the vibration characteristic frequency band. Represents spectral reflectance Road surface reference value The absolute difference represents the exponential function, , and Represents the weight coefficient.

4. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The statistical layer uses exponentially weighted moving average to update the acceleration threshold, and the formula is: ； in, express The basic acceleration threshold at the moment, represents the historical data attenuation factor, express The basic acceleration threshold at the moment, Indicates The corresponding acceleration mean value verified by the analysis layer within the time.

5. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The shutter speed dynamic adjustment strategy is constructed according to the pixel displacement constraint formula, and the pixel displacement constraint formula is: ； in, Indicates the maximum allowed exposure time, Indicates the camera sensor pixel size, Indicates the current vehicle speed. Indicates the optical magnification, Indicates the angle between the camera optical axis and the road surface normal; Taking photos according to the shutter speed dynamic adjustment strategy to obtain a road surface image set; The images in the collected road surface image set are sorted according to the imaging clarity index, and a first road surface image set is constructed according to the images whose imaging clarity index is greater than a threshold.

6. The method for rapid automatic road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The motion blur removal model includes an image fusion layer, a fused image input layer, a multi-scale codec layer and a clear image output layer; wherein the image fusion layer obtains and fuses images in the first road image set to obtain a fused image; The multi-scale codec layer includes an encoder and a decoder, where the encoder contains 4 levels of downsampling, each level contains a dynamic convolution block and a motion blur attention block, and the decoder contains 4 levels of upsampling, each level contains a cross-scale skip link block and a non-local attention block; In the motion blur attention block, the dynamic blur kernel is constructed according to the vehicle speed data. The formula is: ； in, Indicates that in the fused image The dynamic blur kernel on the position, Indicates the exposure time, represents the two-dimensional Dirac function, and Indicates vehicle speed Axis and The weight of the axis, and The acceleration of the vehicle is Axis and The weight of the axis, represents the integral variable; The loss function is constructed based on blur kernel matching loss and gradient-aware loss.

7. The method for automatic rapid road surface detection based on high-speed camera technology according to claim 1 is characterized in that: The road surface detection model includes a multimodal input encoding layer, a cross-modal fusion layer, and a multi-task output layer, wherein the multimodal input encoding layer receives data and extracts image features, vibration spectrum features, and spectral features; The cross-modal fusion layer includes a cross-attention layer and a dynamic gated fusion layer for fusing features; the multi-task output layer includes a segmentation head for outputting the segmentation mask of the object to be detected, a classification head for outputting the category probability of the object to be detected, and a regression head for outputting the size and depth of the object to be detected; A joint loss function is constructed based on segmentation loss, classification loss and regression loss.

Images