A method and system for simulating jitter blur of road defect detection cameras

Abstract

The present disclosure provides a method and system for simulating jitter and blur of a road defect detection camera, which relates to the field of computer vision technology. The method comprises: acquiring multi-source sensor data of a road defect detection camera platform, including color images, camera imaging geometric parameters, acceleration and angular velocity data collected by an inertial measurement unit, and depth information collected by a laser ranging module; reconstructing a continuous displacement trajectory of the camera in three-dimensional space based on the multi-source sensor data; mapping the reconstructed continuous displacement trajectory to an image plane to form a non-uniform point spread function; and performing fuzzy convolution on the acquired color image using the non-uniform point spread function to generate a simulated jitter and blur image. The present disclosure, through the combined assistance of a laser ranging module and an inertial measurement unit, realistically restores the complex jitter process of the road detection platform, provides high-fidelity image samples for the training and evaluation of a defect detection algorithm, and significantly improves the robustness and adaptability of the algorithm under non-ideal conditions.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a method and system for simulating jitter and blur of a road defect detection camera. Background Art

[0002] With the widespread application of vision-based road defect detection technology in intelligent transportation and road maintenance, the quality of on-site images has become a key factor affecting the accuracy and robustness of detection algorithms. Road defect detection cameras are often mounted on mobile platforms or roadside brackets, making them susceptible to the combined effects of vehicle vibration, wind loads, and environmental disturbances. This can cause jitter blur, characterized by non-stationary depth of field, spatial non-uniformity, and temporal non-stationarity. This can lead to a significant loss of edge detail in defect features (such as cracks, potholes, and subsidence), compromising the detection model's ability to locate and classify minor damage.

[0003] Currently, most image-based motion blur simulation methods are limited to assuming that the camera moves at a constant speed or acceleration within a fixed depth of field plane or in a single direction. The constructed point spread functions (PSFs) are mostly one-dimensional or two-dimensional uniform linear motion kernels with fixed intervals, simple trajectories, and no consideration of the effects of depth of field and gravity. Although such methods can improve the model's adaptability to motion blur to a certain extent during convolutional neural network training, they are still insufficient in reproducing complex motion patterns due to the lack of true three-dimensional motion paths and depth information. This makes it difficult to accurately fit the actual distribution of multi-degree-of-freedom, small-amplitude, and high-frequency motion in road defect detection scenarios.

[0004] Inertial measurement unit (IMU) technology has been widely used in the field of attitude estimation and motion tracking on mobile platforms. A three-axis accelerometer and gyroscope can be used to collect real-time acceleration and angular velocity data from the camera-mounted platform. After zero-bias calibration and gravity separation, the linear acceleration and angular velocity signals of the camera in the machine system are obtained. Using interpolation and integration operations, the attitude and displacement trajectory can be reconstructed. However, due to the accumulated error (drift) and noise interference of IMU data, the positioning accuracy of trajectory reconstruction based solely on IMUs will be significantly reduced in long-term or highly dynamic scenes. In addition, IMUs cannot provide depth information between the camera and scene objects, making it difficult to accurately map camera motion to image coordinates at the pixel plane.

[0005] Therefore, existing methods that use IMUs to reconstruct trajectories and generate blur kernels often ignore the correction of the geocentric gravity component and depth mapping, resulting in deviations between the generated PSF and the actual jitter in terms of spatial distribution and energy normalization. At the same time, blur simulations that rely solely on IMUs cannot consider the impact of the difference in depth of field on the width and direction of motion blur.

[0006] In summary, among existing road defect detection technologies, traditional image blur simulation methods cannot truly restore the problem of non-uniform camera jitter blur in complex dynamic environments, which ultimately affects the robustness and generalization ability of actual road detection. Summary of the Invention

[0007] To address the aforementioned issues, this paper proposes a camera shake and blur simulation method and system for road defect detection. By combining the assistance of a laser ranging module (LRM) and an inertial measurement unit (IMU), this method effectively integrates spatial motion patterns and depth information, realistically reproducing the complex shake process of the road inspection platform. This provides high-fidelity image samples for the training and evaluation of defect detection algorithms, significantly improving the robustness and adaptability of the algorithms under non-ideal conditions.

[0008] According to some embodiments, the present disclosure adopts the following technical solutions:

[0009] A method for simulating the shaking and blurring of a road defect detection camera, comprising:

[0010] Acquire multi-source sensor data from the road defect detection camera platform and perform data preprocessing, wherein the multi-source sensor data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by the inertial measurement unit, and depth information collected by the laser ranging module;

[0011] Reconstruct the continuous displacement trajectory of the camera in three-dimensional space based on multi-source sensor data;

[0012] The reconstructed continuous displacement trajectory is mapped to the image plane to form a non-uniform point spread function;

[0013] The acquired color image is fuzzy convolved by a non-uniform point spread function to generate a simulated jittered blurred image. According to some embodiments, the present disclosure adopts the following technical solutions:

[0014] A jitter and blur simulation system for a road defect detection camera, comprising:

[0015] a data acquisition module configured to acquire multi-source sensor data from the road defect detection camera platform and perform data preprocessing, wherein the multi-source sensor data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by the inertial measurement unit, and depth information collected by the laser ranging module;

[0016] The trajectory reconstruction module is configured to: reconstruct the continuous displacement trajectory of the camera in three-dimensional space based on multi-source sensor data;

[0017] The function forming module is configured to: map the reconstructed continuous displacement trajectory to the image plane to form a non-uniform point spread function;

[0018] The blur simulation module is configured to: perform blur convolution on the acquired color image through a non-uniform point spread function to generate a simulated dithered blur image.

[0019] According to some embodiments, the present disclosure adopts the following technical solutions:

[0020] A computer program product includes a computer program. When the computer program is executed by a processor, the computer program implements the method for simulating shaking and blurring of a road defect detection camera.

[0021] According to some embodiments, the present disclosure adopts the following technical solutions:

[0022] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method for simulating jitter and blur of a road defect detection camera is implemented.

[0023] According to some embodiments, the present disclosure adopts the following technical solutions:

[0024] An electronic device includes: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the jitter blur simulation method for a road defect detection camera.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. This paper utilizes LRM ranging to suppress IMU integral drift errors, while simultaneously providing continuous trajectories through IMU degravity and interpolation reconstruction. The two complement each other to improve the accuracy of trajectories and blur kernels, ensuring the stability and reliability of simulation results in long-term, high-dynamic scenarios. This solves the problem that traditional image blur simulation methods cannot accurately reproduce the non-uniform camera jitter blur in complex dynamic environments.

[0027] 2. The present disclosure can realistically reproduce the non-uniform jitter blur characteristics of pixel-level road images under complex vibration disturbances, providing reliable simulation data and reference basis for subsequent image deblurring, disease identification and system calibration.

[0028] 3. The large-scale, multi-modal jittered blurred images generated by this disclosure can be used as online / offline data enhancement samples for deep learning networks, enhancing the model's sensitivity and recognition accuracy to the edges and details of defects under complex vibration interference, effectively improving the robustness and generalization ability of actual road detection systems.

[0029] 4. This disclosure supports dynamic configuration of key parameters such as PSF size, sampling frequency, and pixel scale, and can calculate convolution in real time on the GPU, which is convenient for integration and deployment in online detection systems, and realizes real-time jitter blur simulation, detection algorithm verification and automatic calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which constitute a part of the present disclosure, are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation to the present disclosure.

[0031] Figure 1 This is a flow chart of the method of Example 1.

[0032] Figure 2 Schematic diagram of the three-axis acceleration zero bias calibration vector and the three-axis angular velocity zero bias calibration vector of Example 1, wherein (a), (b), and (c) are respectively a comparison diagram of the three-axis acceleration vector before and after calibration, a comparison diagram of the three-axis angular velocity vector before and after calibration, and a comparison diagram of the timestamp distribution;

[0033] Figure 3 Schematic diagram of the three-dimensional displacement trajectory of the camera platform in the coordinate system of Example 1;

[0034] Figure 4 Schematic diagram of three-dimensional plane trajectory mapping of Example 1, where (a), (b), and (c) are trajectory mapping diagrams of the XY plane, XZ plane, and YZ plane, respectively;

[0035] Figure 5 Schematic diagram of the PSF grid of Example 1;

[0036] Figure 6 Schematic diagram comparing the original road damage image and the simulated motion blurred road damage image in Example 1. DETAILED DESCRIPTION

[0037] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0038] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs.

[0039] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "comprising" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0040] A Laser Ranging Module (LRM) can acquire the distance (i.e., depth information) between the platform and the road ahead or reference objects through time-of-flight (ToF) or triangulation. This true depth measurement can not only be used to scale the mapping of motion to image pixels at different distances, but also assist in compensating for the integral and cumulative errors of IMU data. Traditional image blur simulations mostly ignore depth information and only construct blur kernels within a pre-set pixel scale. This fails to reflect the differential effects of object distance and camera motion on the blur effect. Therefore, the combined application of IMU and LRM in the shake blur simulation of road defect detection cameras offers significant advantages.

[0041] On the one hand, the IMU provides a rich sequence of three-axis acceleration and angular velocity. After zero-bias calibration, interpolation synchronization, and gravity vector removal, it can reconstruct the camera's continuous 3D displacement trajectory in the world coordinate system. On the other hand, the real-time laser ranging results provided by the LRM can be fused with the trajectory data. The physical motion is mapped to the image plane using pixel scale (pixel_scale), generating a PSF that conforms to the depth variation and non-uniform jitter distribution. This PSF is normalized and combined with fast Fourier transform convolution (FFT convolution) and applied to each channel of the color image, simulating realistic jitter blur effects at the pixel level.

[0042] While existing research uses IMUs to reconstruct trajectories and generate blur kernels, these methods often neglect correction for the geocentric gravity component and depth mapping, resulting in deviations in the spatial distribution and energy normalization of the generated PSF from actual motion jitter. Furthermore, blur simulations relying solely on IMUs fail to account for the impact of foreground-to-background depth differences on the width and direction of motion blur. Therefore, leveraging the complementary strengths of LRMs and IMUs, fusing multi-source information on 3D motion trajectories, and applying the resulting non-uniform PSF to image convolution is an effective approach to addressing the practical needs of road defect detection. Furthermore, as deep learning algorithms for road defect detection increasingly demand sample diversity and feature integrity, blurred image data based on realistic physical motion simulations can be used not only for data augmentation during training but also for robustness evaluation and algorithm validation of online detection systems. However, a camera-side simulation methodology for non-uniform motion blur caused by camera vibration in road inspection applications is currently lacking, hindering the dual demands of algorithm development and system integration. In summary, there is an urgent need for a method and system for simulating the non-uniform jitter and blur of road defect detection cameras based on the joint assistance of IMU and LRM. This method performs zero-bias calibration and gravity separation on the IMU data, and synchronously reconstructs it through trajectory interpolation. It then combines the LRM ranging result mapping to generate a deep adaptive PSF, and applies it to image FFT convolution. This method can truly reproduce the jitter and blur characteristics of road surface inspection cameras under vibration disturbances at the pixel level, providing reliable data support for the subsequent development and evaluation of image deblurring and detection algorithms.

[0043] Therefore, the present disclosure proposes a method and system for simulating non-uniform jitter and blur of road defect detection cameras based on the joint assistance of LRM and IMU. This method accurately reconstructs the camera's continuous displacement trajectory in three-dimensional space by fusing multi-source sensor information from an inertial measurement unit (IMU) and a laser ranging module (LRM). This information is then mapped onto the image plane to form a non-uniform point spread function (PSF). This method then performs blur convolution on the sharp image, simulating realistic pixel-level jitter blur. Specifically, the acceleration and angular velocity data acquired by the IMU are loaded and calibrated, the zero bias is removed, and the system gravity is estimated. Simultaneously, the LRM measures the depth between the camera and the object in front in real time, and the pixel scale (meters per pixel) is calculated. Secondly, the calibration data is interpolated in the synchronous time domain, and the attitude is recovered using quaternion integration. A continuous three-dimensional trajectory is constructed by combining gravity separation and integration operations. The three-dimensional trajectory plane is then projected and normalized onto a two-dimensional PSF matrix, forming a trajectory energy distribution that is then normalized to form a convolution kernel. Subsequently, the PSF is applied to the three image channels using a fast Fourier transform (FFT) to simulate realistic blurring. Finally, parameter modeling is used to rationally configure the PSF size, trajectory sampling number, and zero bias estimation range to ensure simulation accuracy and computational efficiency.

[0044] Example 1

[0045] In one embodiment of the present disclosure, a method for simulating jitter blur of a road defect detection camera is provided, comprising:

[0046] Step 1: Acquire multi-source sensor data from the road defect detection camera platform and perform data preprocessing. The multi-source sensor data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by the inertial measurement unit, and depth information collected by the laser ranging module.

[0047] Step 2: Reconstruct the continuous displacement trajectory of the camera in three-dimensional space based on multi-source sensor data;

[0048] Step 3: Map the reconstructed continuous displacement trajectory to the image plane to form a non-uniform point spread function;

[0049] Step 4: Perform blur convolution on the acquired color image through the non-uniform point spread function to generate a simulated dithered blurred image.

[0050] As an embodiment, the present disclosure discloses a method for simulating the vibration and blur of a road defect detection camera. By combining the assistance of a laser ranging module (LRM) and an inertial measurement unit (IMU), the method effectively integrates spatial motion patterns and depth information, realistically reproducing the complex vibration process of the road detection platform. This provides high-fidelity image samples for the training and evaluation of the defect detection algorithm, significantly improving the algorithm's robustness and adaptability under non-ideal conditions. Figure 1 The specific implementation process is as follows:

[0051] S1. Obtaining color image information of road damage captured by the camera platform SHARP_IMAGE , camera imaging geometric parameters and timestamp information of the captured image.

[0052] Specifically, in view of the fact that the vehicle-mounted camera is prone to vibration and shaking with the vehicle during road disease detection, which causes the camera image to be blurred, the color image information of the road disease captured by the camera is read. SHARP_IMAGE , and record the camera focal length and camera optical center parameters and the complete timestamp information of the generated image .

[0053] S2. Obtain distance sequence information of road damage from the Laser Ranging Module (LRM) located on the camera platform , specifically:

[0054] Step S2-1: Hardware Physical Association. Secure the LRM and camera to a rigid bracket to form a camera platform, ensuring a stable relative position between them. Also, adjust the LRM's measurement direction to be as parallel as possible to the camera's optical axis and record the angular deviation between the two.

[0055] Step S2-2: Associate the generated image with the distance data. Match the timestamp information of the image generated in step S1 , the distance sequence corresponding to the timestamp .

[0056] S3. Get the acceleration vector of the IMU on the camera platform and the angular velocity vector , specifically:

[0057] Step S3-1: Hardware physical connection: Using the camera platform from step S2-1, fix the IMU on a rigid bracket to keep the relative position of the two stable.

[0058] Step S3-2: Read the raw sampled data ACCEL_PATH and GYRO_PATH from the accelerometer and gyroscope files of the IMU and extract the accelerometer timestamp , gyroscope timestamp , three-axis acceleration vector and the three-axis angular velocity vector .

[0059] S4, performing zero bias calibration on the three-axis acceleration vector and the three-axis angular velocity vector obtained in step S3, specifically:

[0060] Step S4-1: Record the total number of still moments in front of the camera platform as , then the calibration vectors of the three-axis acceleration and three-axis angular velocity are:

[0061] (1)

[0062] (2)

[0063] Where, For the The three-axis acceleration vector at the moment , For the The three-axis angular velocity vector at the moment .

[0064] Step S4-2: Three-axis acceleration vector and the three-axis angular velocity vector Perform zero bias calibration.

[0065] (3)

[0066] (4)

[0067] The calibration results are as follows Figure 2As shown, Figure 2 (a), (b), and (c) are the comparison diagrams before and after the three-axis acceleration vector calibration, the comparison diagrams before and after the three-axis angular velocity vector calibration, and the comparison diagrams of the timestamp distribution, respectively.

[0068] S5. Estimate the gravity of the camera platform using the three-axis acceleration vector obtained in step S3 to obtain the gravity vector .

[0069] Specifically, the three-axis acceleration zero bias calibration vector is obtained by step S4-2. , calculate the gravity vector under the camera platform system :

[0070] (5)

[0071] S6, based on the camera imaging geometric parameters obtained in step S1 and the LRM distance sequence obtained in step S2 Dynamic modeling pixel scale, specifically:

[0072] Step S6-1: Three-axis acceleration bias calibration vector based on step S4-2 And the three-axis angular velocity bias calibration vector , linear interpolation functions are established for the distance sequence of the timestamp in step S2-2 and the calibrated acceleration and angular velocity:

[0073] (6)

[0074] (7)

[0075] (8)

[0076] Where, 、 and They are the original time stamps of the accelerometer, gyroscope and laser ranging module respectively. for The distance sequence at the moment, Represents a linear interpolation function.

[0077] Step S6-2: Establish image plane coordinate increments based on the pinhole imaging model With physical space increment relation:

[0078] (9)

[0079] Where, is the physical space distance information. Therefore, the dynamic pixel scale is defined as:

[0080] (10)

[0081] When the distance sequence of LRM When the fluctuation is less than 1%, take the time domain average:

[0082] (11)

[0083] S7, based on the calibrated acceleration and angular velocity of step S4 and the gravity vector of step S5, reconstructing the three-dimensional displacement trajectory of the camera platform in the world coordinate system, specifically including:

[0084] Step S7-1: Select acceleration and angular velocity The overlapping interval of , ], and perform equal-interval sampling, then:

[0085] (12)

[0086] (13)

[0087] exist[ ] interval uniformly generates P sampling moments :

[0088] (14)

[0089] Therefore, each sampling moment for:

[0090] (15)

[0091] Step S7-2: Calculate each sampling moment Linear acceleration (m / s²), angular velocity (rad / s) and distance (m), specifically:

[0092] The distance obtained in step S4 and the calibrated acceleration and angular velocity linear interpolation function are used to calculate the velocity at each sampling moment. Linear acceleration (m / s²), angular velocity (rad / s) and distance (m):

[0093] (16)

[0094] (17)

[0095] (18)

[0096] Step S7-3: Update the posture of the camera platform system relative to the world coordinate system. By introducing quaternions Represents the initial posture of the camera system relative to the world coordinate system, then:

[0097] (19)

[0098] Rotate a vector by a small angle ( ) Update the posture of the world coordinate system, then:

[0099] (20)

[0100] In the formula, ” indicates quaternion multiplication.

[0101] Step S7-4: Remove the gravity vector from the IMU acceleration and update the camera platform velocity and displacement. Let the world system velocity be , is the total time, is the total time recorded by IMU, and the displacement is , initial velocity and displacement, then:

[0102] (twenty one)

[0103] (twenty two)

[0104] At the moment When , the gravity vector after removing the IMU acceleration is:

[0105] (twenty three)

[0106] Where, The initial pose of the camera system relative to the world coordinate system The corresponding 3×3 rotation matrix.

[0107] Step S7-5: Update at time The camera platform speed and displacement at time , then:

[0108] (twenty four)

[0109] (25)

[0110] Step S7-6: Calculate the camera platform trajectory matrix and Combine them in order to get a trajectory matrix of shape P×3, such as Figure 3 As shown, the trajectory output is achieved:

[0111] (26)

[0112] Where, is the continuous three-dimensional displacement trajectory of the camera platform in the world coordinate system.

[0113] S8. Map the three-dimensional displacement trajectory of step S7 to the image plane and construct a discrete PSF grid, specifically:

[0114] Step S8-1: For each sampling point For plane projection,

[0115] (27)

[0116] Where, Indicates the World system displacement vector at this moment , and Respectively represent The x-axis and y-axis coordinates of the pixel on the image plane at the moment are formed as follows Figure 4 The three-dimensional plane trajectory mapping diagram shown in FIG, wherein, Figure 4 (a), (b), and (c) are the trajectory mapping diagrams of the XY plane, XZ plane, and YZ plane, respectively. The pixel coordinate extreme values of all trajectory sampling points P are calculated:

[0117] , (28)

[0118] , (29)

[0119] Step S8-2: Initialize the PSF grid, let the PSF grid matrix .

[0120] Step S8-3: For each point ( , ) Calculate its pixel index on the PSF matrix:

[0121] , (30)

[0122] Where, and They represent the x-axis and y-axis coordinates of the discrete PSF point diffusion matrix grid, respectively.

[0123] After accumulating the count, if ( , ) falls within the range [0,N−1]×[0,N−1], then:

[0124] (31)

[0125] Step S8-4: Construct a PSF convolution kernel with a total energy of 1, and perform the grid matrix Normalize it, then

[0126] (32)

[0127] The final normalized grid matrix , which is the non-uniform point spread function PSF.

[0128] S9, applying the PSF of step S8 to the road damage color image of step S1 , through the fast Fourier transform to accelerate the convolution operation, generate a simulated jitter blur image, specifically:

[0129] Step S9-1: Colorful sharp images ,but:

[0130] (33)

[0131] Where, is the image reading function, .

[0132] Convert and normalize the color space of the sharp color image, then:

[0133] (34)

[0134] (35)

[0135] Where, Indicates the original sharp image in pixels ,aisle The intensity value is normalized to [0,1], Represent the width and height of the input image respectively, , Represents the color channel index, , Represents the colors R, G, and B respectively.

[0136] Step S9-2: Construct the spatial domain and frequency domain convolution model of the jitter-blurred image, and define the spatial domain convolution function for each channel c. Then:

[0137] (36)

[0138] Where, is the intensity value of the simulated jitter blurred image at pixel (x, y), channel c, is the PSF size (square side length, unit: pixel). Fill with zeros.

[0139] For each channel c and Perform a two-dimensional fast Fourier transform, then:

[0140] (37)

[0141] (38)

[0142] Where, Represents a two-dimensional Fast Fourier Transform (FFT) operation.

[0143] At this time, the jittered blurred image under channel c is:

[0144] (39)

[0145] (40)

[0146] Where, represents the two-dimensional inverse FFT operation, is the convolution operator in the spatial domain.

[0147] Step S9-3: Perform range cropping on the jitter-blurred image under channel c, then:

[0148] (41)

[0149] Quantize the cropped dithered blurred image to 8-bit data:

[0150] (42)

[0151] Get a jittery blurred image.

[0152] S10, the jittered blurred image of step S9 After generation, to ensure optimal performance in terms of blur realism, trajectory restoration, and application adaptability, the key parameters introduced in this embodiment must be modeled and configured, and systematically evaluated from perspectives such as trajectory geometry, imaging mapping, system accuracy, numerical stability, and perceptual performance. This is then used for the subsequent batch generation of jitter-blurred images. This is divided into five sub-steps:

[0153] Step S10-1: PSF size To ensure that the PSF can completely cover the trajectory projection on the image plane and avoid the motion energy being clipped, its side length is set to meet the following requirements:

[0154] (43)

[0155] in, is the maximum trajectory offset radius, and the calculation formula is:

[0156] (44)

[0157] (45)

[0158] (46)

[0159] in, represents the side length of the PSF convolution kernel (pixels); 、 For the The pixel coordinates of the trajectory points on the image plane; 、 represents the center of gravity of the trajectory; is the number of trajectory sampling points.

[0160] Step S10-2: Number of trajectory sampling points To ensure that the temporal resolution of the trajectory can accurately reflect the high-frequency motion of the camera, the trajectory sampling interval must meet the Nyquist sampling condition:

[0161] (47)

[0162] (48)

[0163] in, is the trajectory sampling time step (seconds); is the maximum estimated jitter frequency of the platform (Hz); and are the start and end boundaries of the trajectory time respectively; is the number of trajectory sampling points. , avoiding the PSF fragmentation and discontinuity problems caused by trajectory discretization.

[0164] Step S10-3: Modeling the bias accuracy of the IMU static sample number static_n. The IMU bias estimation accuracy directly affects the stability of the subsequent trajectory. According to statistical principles, its error variance is:

[0165] (49)

[0166] (50)

[0167] in, , are the zero bias estimates of acceleration and angular velocity respectively; , are the noise variances of IMU acceleration and angular velocity respectively; represents the number of samples used for stationary estimation.

[0168] Step S10-4: Pixel scale Dynamic modeling and depth mapping. The pixel scale maps the physical trajectory to the image space, which is defined as:

[0169] (51)

[0170] (52)

[0171] in, is the pixel / meter ratio at time t; is the camera equivalent focal length (pixels); Indicates the LRM real-time distance value (meters); is the temporal average pixel scale. Use dynamic It can enhance the front and back depth adaptation capability of PSF. It is more suitable for scenes where the image depth does not change drastically.

[0172] Step S10-5: PSF normalization regularization term To prevent the numerical instability of PSF when the trajectory energy is sparse or single-point distributed, a regularization term is introduced in the normalization:

[0173] (53)

[0174] Among them, the trajectory energy accumulation matrix, the regularization term, and the normalized point spread function value are included. This term effectively prevents the PSF energy from being normalized to zero or over-concentrated, ensuring the numerical stability of the convolution process.

[0175] Step S10-6: Comparative experiment evaluation and parameter optimization feedback.

[0176] After completing the above parameter configuration, it is necessary to conduct comparative experiments based on the performance indicators in Table 1 to provide feedback on the effectiveness of the parameter settings:

[0177] Table 1 Performance Index

[0178]

[0179] Through subjective visual analysis and objective indicator evaluation of images generated under different parameter combinations, the fuzzy simulation process of this embodiment can be verified, optimized, and iterated.

[0180] In the specific implementation process, through Figure 5 The non-uniform PSF grid shown in can intuitively show that this embodiment uses the convolution kernel generated by fusing the IMU trajectory with the LRM depth information to accurately depict the vibration amplitude and direction; Figure 6 The comparison between the original road damage image and the simulated motion blurred image further verified that the PSF can truly reproduce the blurred edges and detail loss of damage caused by camera shake at the pixel level, thereby providing high-fidelity training and evaluation samples for subsequent deblurring processing and damage detection algorithms, significantly improving the detection accuracy and robustness of the system in complex dynamic environments.

[0181] Example 2

[0182] In one embodiment of the present disclosure, a system for simulating jitter and blur of a road defect detection camera is provided, comprising:

[0183] a data acquisition module configured to acquire multi-source sensor data from the road defect detection camera platform and perform data preprocessing, wherein the multi-source sensor data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by the inertial measurement unit, and depth information collected by the laser ranging module;

[0184] The trajectory reconstruction module is configured to: reconstruct the continuous displacement trajectory of the camera in three-dimensional space based on multi-source sensor data;

[0185] The function forming module is configured to: map the reconstructed continuous displacement trajectory to the image plane to form a non-uniform point spread function;

[0186] The blur simulation module is configured to: perform blur convolution on the acquired color image through a non-uniform point spread function to generate a simulated dithered blur image.

[0187] Example 3

[0188] In one embodiment of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the computer program implements the method for simulating shaking and blurring of a road defect detection camera.

[0189] Example 4

[0190] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method for simulating jitter blur of a road defect detection camera is implemented.

[0191] Example 5

[0192] In one embodiment of the present disclosure, an electronic device is provided, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device implements the jitter blur simulation method for a road defect detection camera.

[0193] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0195] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Those skilled in the art should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A method for simulating camera shake and blur for road damage detection, characterized in that: include: Acquire multi-source sensor data from the road defect detection camera platform and perform data preprocessing, wherein the multi-source sensor data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by the inertial measurement unit, and depth information collected by the laser ranging module; Reconstruct the continuous displacement trajectory of the camera in three-dimensional space based on multi-source sensor data; The reconstructed continuous displacement trajectory is mapped to the image plane to form a non-uniform point spread function; The acquired color image is blurred and convolved by a non-uniform point spread function to generate a simulated dithered blurred image; The camera platform includes an inertial measurement unit, a laser ranging module, and a road damage detection camera fixed on a rigid bracket, and the rigid bracket ensures the relative stability of the three. The color image is a color image of road damage taken by the camera platform; the acceleration and angular velocity are the acceleration and angular velocity of the inertial measurement unit located on the camera platform; the depth information is the distance sequence information of the road damage measured by the laser ranging module located on the camera platform; The pretreatment includes: Load and calibrate the acceleration and angular velocity data collected by the inertial measurement unit, remove the zero bias, and estimate the gravity vector under the camera platform system; Dynamically model pixel scale based on camera imaging geometric parameters and depth information collected by the laser ranging module; The reconstruction of the continuous displacement trajectory of the camera in the three-dimensional space is based on the gravity vector and the pre-processed acceleration and angular velocity data, and is carried out by numerical integration in the world coordinate system to reconstruct the continuous three-dimensional displacement trajectory of the camera platform; The non-uniform point spread function is formed by plane-projecting the continuous displacement trajectory based on the pixel scale and normalizing and mapping it into a two-dimensional point spread function matrix to form a trajectory energy distribution and normalize it into a convolution kernel to obtain a non-uniform point spread function.

2. The method for simulating the shaking blur of a road defect detection camera according to claim 1, wherein: The fuzzy convolution performed on the acquired color image is to use fast Fourier transform to apply the non-uniform point spread function to the three channels of the color image respectively, so as to obtain a simulated jittered blurred image.

3. The method for simulating camera shake and blur of road damage detection according to claim 1, characterized in that: It also includes optimizing and verifying the PSF size, number of trajectory sampling points, number of IMU static samples, pixel scale and regularization parameters based on the generated jittered blurred images, for subsequent batch generation of jittered blurred images.

4. A camera shake and blur simulation system for road damage detection, characterized in that: include: a data acquisition module configured to acquire multi-source sensor data from the road defect detection camera platform and perform data preprocessing, wherein the multi-source sensor data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by the inertial measurement unit, and depth information collected by the laser ranging module; The trajectory reconstruction module is configured to: reconstruct the continuous displacement trajectory of the camera in three-dimensional space based on multi-source sensor data; The function forming module is configured to: map the reconstructed continuous displacement trajectory to the image plane to form a non-uniform point spread function; The blur simulation module is configured to: perform blur convolution on the acquired color image through a non-uniform point spread function to generate a simulated dithered blurred image; The camera platform includes an inertial measurement unit, a laser ranging module, and a road damage detection camera fixed on a rigid bracket, and the rigid bracket ensures the relative position stability of the three. The color image is a color image of road damage taken by a camera platform; the acceleration and angular velocity are the acceleration and angular velocity of the IMU located on the camera platform; the depth information is the distance sequence information of the road damage by the laser ranging module located on the camera platform; the preprocessing includes: Load and calibrate the acceleration and angular velocity data collected by the inertial measurement unit, remove the zero bias, and estimate the gravity vector under the camera platform system; Dynamically model pixel scale based on camera imaging geometric parameters and depth information collected by the laser ranging module; The reconstruction of the continuous displacement trajectory of the camera in the three-dimensional space is based on the gravity vector and the pre-processed acceleration and angular velocity data, and is carried out by numerical integration in the world coordinate system to reconstruct the continuous three-dimensional displacement trajectory of the camera platform; The non-uniform point spread function is formed by plane-projecting the continuous displacement trajectory based on the pixel scale and normalizing and mapping it into a two-dimensional point spread function matrix to form a trajectory energy distribution and normalize it into a convolution kernel to obtain a non-uniform point spread function.

5. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the method for simulating jitter and blur of a road defect detection camera according to any one of claims 1 to 3 is implemented.

6. An electronic device, characterized in that: include: A processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the jitter blur simulation method for a road defect detection camera as described in any one of claims 1 to 3.

Images