Jitter fuzzy simulation method and system for road disease detection camera

Abstract

The invention provides a jitter fuzzy simulation method and system for a road disease detection camera, and relates to the technical field of computer vision, and the method comprises the steps: obtaining multi-source sensing data of a road disease detection camera platform, comprising color images, camera imaging geometric parameters, acceleration and angular velocity data acquired by an inertial measurement unit, and depth information acquired by a laser ranging module; based on the multi-source sensing data, reconstructing a continuous displacement track of the camera in a three-dimensional space; mapping the reconstructed continuous displacement track to an image plane to form a non-uniform point spread function; performing fuzzy convolution on the obtained color image through a non-uniform point spread function to generate a simulated jitter fuzzy image; through combined assistance of the laser ranging module and the inertial measurement unit, the complex jitter process of the road detection platform is truly restored, a high-fidelity image sample is provided for training and evaluation of a disease detection algorithm, and the robustness and adaptive capacity of the algorithm under non-ideal conditions are remarkably improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and particularly to a method and system for simulating jitter blur of a road disease detection camera. Background Art

[0002] With the wide application of vision-based road disease detection technology in the fields of intelligent transportation and road maintenance, the quality of on-site images has become one of the key factors affecting the accuracy and robustness of detection algorithms. Road disease detection cameras are often installed on mobile platforms or roadside brackets, and are easily affected by the combined effects of factors such as vehicle vibration, wind load, and environmental disturbances, resulting in non-fixed depth of field, non-uniformity in space, and non-stationarity in time jitter blur, causing serious loss of edge details of disease features (such as cracks, potholes, subsidence, etc.), and affecting the positioning and classification ability of the detection model for minor damages.

[0003] Currently, most image-based jitter blur simulation methods are limited to assuming that the camera moves at a constant speed / constant acceleration in a fixed depth of field plane or in a single direction. The constructed Point Spread Function (PSF) is mostly a one-dimensional or two-dimensional uniform linear motion kernel, with fixed intervals, simple trajectories, and no consideration of depth of field and gravity effects. Although such methods can improve the model's adaptability to motion blur to a certain extent in the training of convolutional neural networks, due to the lack of real three-dimensional motion paths and depth information, the restoration of complex jitter patterns is still insufficient, and it is difficult to accurately fit the actual distribution of multi-degree-of-freedom, small amplitude, and high-frequency jitter in the road disease detection scenario.

[0004] Inertial Measurement Unit (IMU) technology has been widely applied in the fields of mobile platform attitude estimation and motion tracking. Through triaxial accelerometers and gyroscopes, the acceleration and angular velocity data of the camera installation platform can be collected in real time. After zero-bias calibration and gravity separation, the linear acceleration and angular velocity signals of the camera in the body coordinate system can be obtained. Using interpolation and integration operations, the attitude and displacement trajectories can be reconstructed. However, due to the cumulative error (drift) and noise interference of IMU data, the positioning accuracy of pure IMU-based trajectory reconstruction will significantly decrease in long-term or high-dynamic scenarios. In addition, IMU cannot provide the depth information between the camera and the scene objects, and it is difficult to accurately map the camera motion to the image coordinates in the pixel plane.

[0005] Therefore, existing methods that use IMU to reconstruct trajectories and generate blur kernels often ignore the correction of the geocentric gravity component and depth mapping, resulting in deviations between the generated PSF and the real jitter situation in terms of spatial distribution and energy normalization. At the same time, single IMU-based blur simulation cannot consider the influence of the front and rear depth of field differences on the motion blur width and blur direction.

[0006] In summary, in the existing road disease detection technology, the traditional image blur simulation method cannot truly restore the problem of non-uniform jitter blur of the camera in a complex dynamic environment, ultimately affecting the robustness and generalization ability of actual road detection. Summary of the Invention

[0007] To solve the above problems, the present disclosure proposes a method and system for simulating jitter blur of a road disease detection camera. Through the joint assistance of a laser ranging module (LRM) and an inertial measurement unit (IMU), the spatial motion law and depth information are effectively fused to truly restore the complex jitter process of the road detection platform, providing high-fidelity image samples for the training and evaluation of disease detection algorithms, and significantly improving the robustness and adaptability of the algorithm under non-ideal conditions.

[0008] According to some embodiments, the present disclosure adopts the following technical solutions: A method for simulating jitter blur of a road disease detection camera, comprising: Obtaining multi-source sensing data of a road disease detection camera platform and performing data preprocessing, wherein the multi-source sensing data includes a color image, camera imaging geometric parameters, acceleration and angular velocity data collected by an inertial measurement unit, and depth information collected by a laser ranging module; Reconstructing the continuous displacement trajectory of the camera in three-dimensional space based on the multi-source sensing data; Mapping the reconstructed continuous displacement trajectory onto the image plane to form a non-uniform point spread function; Performing blur convolution on the obtained color image through the non-uniform point spread function to generate a simulated jitter blur image. According to some embodiments, the present disclosure adopts the following technical solutions: A system for simulating jitter blur of a road disease detection camera, comprising: A data acquisition module configured to: obtain multi-source sensing data of a road disease detection camera platform and perform data preprocessing, wherein the multi-source sensing data includes a color image, camera imaging geometric parameters, acceleration and angular velocity data collected by an inertial measurement unit, and depth information collected by a laser ranging module; A trajectory reconstruction module configured to: reconstruct the continuous displacement trajectory of the camera in three-dimensional space based on the multi-source sensing data; A function formation module configured to: map the reconstructed continuous displacement trajectory onto the image plane to form a non-uniform point spread function; A blur simulation module configured to: perform blur convolution on the obtained color image through the non-uniform point spread function to generate a simulated jitter blur image.

[0009] According to some embodiments, the present disclosure adopts the following technical solutions: A computer program product includes a computer program which, when executed by a processor, implements the method for simulating jitter blur of a road disease detection camera as described above.

[0010] According to some embodiments, the present disclosure adopts the following technical solutions: A non-transitory computer-readable storage medium is used to store computer instructions which, when executed by a processor, implement the method for simulating jitter blur of a road disease detection camera as described above.

[0011] According to some embodiments, the present disclosure adopts the following technical solutions: An electronic device includes: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes and implements the method for simulating jitter blur of a road disease detection camera as described above.

[0012] Compared with the prior art, the beneficial effects of the present disclosure are as follows: 1. The present disclosure uses LRM ranging to suppress the IMU integration drift error, and at the same time provides a continuous trajectory through IMU de-gravity and interpolation reconstruction. The two complement each other to improve the accuracy of the trajectory and the blur kernel, ensuring the stable reliability of the simulation results in long-term and high-dynamic scenarios, and solving the problem that traditional image blur simulation methods cannot truly restore the non-uniform jitter blur of the camera in complex dynamic environments.

[0013] 2. The present disclosure can truly reproduce the non-uniform jitter blur characteristics of pixel-level road images under complex vibration disturbances, providing reliable simulation data and reference basis for subsequent image de-blurring, disease recognition, and system calibration.

[0014] 3. The large-scale and multi-mode jitter blur images generated by the present disclosure can be used as online / offline data augmentation samples for deep learning networks, enhancing the sensitivity and recognition accuracy of the model to disease edges and details under complex vibration disturbances, and effectively improving the robustness and generalization ability of the actual road detection system.

[0015] 4. The present disclosure supports dynamic configuration of key parameters such as PSF size, sampling frequency, pixel scale, etc., and can perform real-time convolution calculation on the GPU, facilitating integration and deployment in an online detection system to achieve real-time jitter blur simulation, detection algorithm verification, and automatic calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings of the specification, which form a part of the present disclosure, are used to provide a further understanding of the present disclosure. The schematic embodiments and descriptions thereof of the present disclosure are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure.

[0017] Figure 1 It is a flowchart of the method for Embodiment 1.

[0018] Figure 2 It is a schematic diagram of the zero-bias calibration vector of the three-axis acceleration and the zero-bias calibration vector of the three-axis angular velocity for Embodiment 1. Among them, (a), (b), and (c) are respectively the comparison diagrams of the three-axis acceleration vector before and after calibration, the comparison diagrams of the three-axis angular velocity vector before and after calibration, and the comparison diagram of the timestamp distribution. Figure 3 It is a schematic diagram of the three-dimensional displacement trajectory of the camera platform in the coordinate system of Embodiment 1. Figure 4 It is a schematic diagram of the three-dimensional plane trajectory mapping for Embodiment 1. Among them, (a), (b), and (c) are respectively the trajectory mapping diagrams of the XY plane, the XZ plane, and the YZ plane. Figure 5 It is a schematic diagram of the PSF grid for Embodiment 1. Figure 6 It is a schematic diagram of the comparison between the original road disease image and the simulated motion-blurred disease image for Embodiment 1. Detailed implementation manners

[0019] The present disclosure will be further described below in conjunction with the accompanying drawings and embodiments.

[0020] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs.

[0021] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "comprise" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0022] The Laser Ranging Module (LRM) can obtain the distance (i.e., depth information) between the platform and the road or reference object ahead through the Time-of-Flight (ToF) or triangulation principle. Real-depth measurement can not only be used to scale the mapping ratio of motion to image pixels at different distances, but also assist in compensating for the integration cumulative error of IMU data. Most traditional image blur simulations ignore depth information and only construct blur kernels within a preset pixel scale, unable to reflect the differential effects of object distance and camera movement on the blur effect. Therefore, jointly applying IMU and LRM to the jitter blur simulation of a road disease detection camera has significant advantages.

[0023] On the one hand, the IMU provides rich sequences of three-axis acceleration and angular velocity. After zero-bias calibration, interpolation synchronization, and removal of the gravity vector, the continuous three-dimensional displacement trajectory of the camera in the world coordinate system can be reconstructed. On the other hand, the real-time laser ranging results provided by the LRM can be fused with the trajectory data, and the physical motion is mapped to the image plane through the pixel_scale to generate a PSF that conforms to the depth change and non-uniform jitter distribution law. After normalization, this PSF is applied to each channel of the color image through fast Fourier transform convolution, thereby simulating the real jitter blur effect at the pixel level.

[0024] Although existing studies have methods for reconstructing trajectories and generating blur kernels using IMUs, they often neglect the correction of the geocentric gravity component and depth mapping, resulting in deviations between the generated PSFs and the actual jitter in terms of spatial distribution and energy normalization. At the same time, the blur simulation relying solely on IMUs cannot consider the effects of foreground and background depth differences on the motion blur width and direction. Therefore, giving full play to the complementary advantages of LRM and IMU, fusing multi-source information for three-dimensional motion trajectories, and applying the generated non-uniform PSF to image convolution is the effective path that is closer to the actual requirements of road disease detection. In addition, as the requirements of deep learning algorithms for road disease detection for sample diversity and feature integrity continue to increase, the blurred image data based on real physical motion simulation can not only be used for data augmentation in the training stage, but also for the robustness evaluation and algorithm verification of online detection systems. However, there is still a lack of a non-uniform jitter blur camera-side simulation methodology for road detection cameras caused by vibration, which cannot meet the dual needs of algorithm development and system integration. To sum up, there is an urgent need for a non-uniform jitter blur simulation method and system for road disease detection cameras based on the joint assistance of IMU and LRM; this method can perform zero-bias calibration and gravity separation on IMU data, interpolate and synchronously reconstruct trajectories, then combine the LRM ranging results to map and generate depth-adaptive PSFs, and apply them to image FFT convolution, capable of truly reproducing the jitter blur characteristics of pavement detection cameras under vibration disturbances at the pixel level, providing reliable data support for the subsequent research and evaluation of image deblurring and detection algorithms.

[0025] Therefore, the present disclosure proposes a non-uniform jitter blur simulation method and system for road disease detection cameras based on the joint assistance of LRM and IMU. This method fuses multi-source sensing information from an inertial measurement unit (IMU) and a laser ranging module (LRM), precisely reconstructs the continuous displacement trajectory of the camera in three-dimensional space, and maps it to the image plane to form a non-uniform point spread function (PSF), and then performs blur convolution on the sharp image to achieve the simulation of true jitter blur at the pixel level; specifically: First, load and calibrate the acceleration and angular velocity data obtained by the IMU, remove the zero bias and estimate the body-frame gravity, and at the same time measure the depth between the camera and the object in front in real time by the LRM to calculate the pixel scale (meters / pixel); Second, interpolate the calibrated data in the synchronous time domain, use quaternion integration to restore the attitude, and combine gravity separation and integration operations to construct a continuous three-dimensional trajectory; Then, project the three-dimensional trajectory plane and normalize it and map it into a two-dimensional PSF matrix to form a trajectory energy distribution and normalize it into a convolution kernel; Subsequently, use the fast Fourier transform to apply the PSF to the three channels of the image respectively to achieve the simulation of the true blur effect; Finally, reasonably configure the PSF size, trajectory sampling number, and zero-bias estimation range through parameter modeling to ensure the simulation accuracy and calculation efficiency.

[0026] Example 1 In an embodiment of the present disclosure, a method for simulating jitter blur of a road disease detection camera is provided, including: Step 1: Obtain multi-source sensing data of the road disease detection camera platform and perform data preprocessing. Among them, the multi-source sensing data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by an inertial measurement unit, and depth information collected by a laser ranging module; Step 2: Based on the multi-source sensing data, reconstruct the continuous displacement trajectory of the camera in three-dimensional space; Step 3: Map the reconstructed continuous displacement trajectory to the image plane to form a non-uniform point spread function; Step 4: Perform blurred convolution on the obtained color image through the non-uniform point spread function to generate a simulated jitter blurred image.

[0027] As an embodiment, the method for simulating jitter blur of a road disease detection camera in the present disclosure effectively fuses the spatial motion law and depth information through the joint assistance of a laser ranging module (LRM) and an inertial measurement unit (IMU), truly restores the complex jitter process of the road detection platform, provides high-fidelity image samples for the training and evaluation of disease detection algorithms, and significantly improves the robustness and adaptability of the algorithm under non-ideal conditions, such as Figure 1 shown, the specific implementation process is as follows: S1. Obtain the road disease color image information captured by the camera platform SHARP_IMAGE , camera imaging geometric parameters and the timestamp information of the captured image.

[0028] Specifically, aiming at the characteristic that the on-vehicle camera is prone to vibrate and jitter following the vehicle during road disease detection, which may cause camera imaging blur, read the road disease color image information captured by the camera SHARP_IMAGE , and record the camera focal length, camera optical center parameters and the complete timestamp information for generating this image .

[0029] S2. Obtain the distance sequence information of the road disease of the laser ranging module (LRM) located on the camera platform , specifically: Step S2-1: Hardware physical association. Fix the LRM and the camera on a rigid bracket to form a camera platform to ensure the relative position stability of the two. At the same time, adjust the measurement direction of the LRM to make it as parallel as possible to the camera optical axis and record the angle deviation between the two.

[0030] Step S2-2: Associate and generate image and distance data. Match the timestamp information of the image generated in corresponding step S1 , and form the distance sequence corresponding to this timestamp 。

[0031] S3. Obtain the acceleration vector and angular velocity vector of the IMU located on the camera platform, specifically as follows: and the angular velocity vector , specifically: Step S3-1: Hardware physical association. Fix the IMU on a rigid bracket through the camera platform in step S2-1 to keep the relative positions of the two stable.

[0032] Step S3-2: Read the original sampling data ACCEL_PATH and GYRO_PATH from the accelerometer and gyroscope files of the IMU, and extract the timestamps of the accelerometer , the timestamps of the gyroscope , the three-axis acceleration vector and the three-axis angular velocity vector .

[0033] S4. Perform zero-bias calibration on the three-axis acceleration vector and three-axis angular velocity vector obtained in step S3, specifically as follows: Step S4-1: Denote the total number of stationary moments before the camera platform as , then the calibration vectors of the three-axis acceleration and three-axis angular velocity are: (1) (2) In the formula, is the three-axis acceleration vector at the th moment , is the three-axis angular velocity vector at the th moment .

[0034] Step S4-2: Perform zero-bias calibration on the three-axis acceleration vector and the three-axis angular velocity vector , then (3) (4) The calibration results are as shown in Figure 2 , where Figure 2 in (a), (b), and (c) are the comparison diagrams of the three-axis acceleration vector before and after calibration, the comparison diagram of the three-axis angular velocity vector before and after calibration, and the comparison diagram of the timestamp distribution, respectively.

[0035] S5. Estimate the gravity of the camera platform through the three-axis acceleration vector obtained in step S3 to obtain the gravity vector .

[0036] Specifically, through the triaxial acceleration zero-bias calibration vector in step S4-2 , calculate the gravity vector in the camera platform system : (5) S6. Based on the camera imaging geometric parameters obtained in step S1 and the LRM distance sequence obtained in step S2 dynamically model the pixel scale, specifically: Step S6-1: Based on the triaxial acceleration zero-bias calibration vector in step S4-2 and the triaxial angular velocity zero-bias calibration vector , establish linear interpolation functions for the distance sequence of the time stamps in step S2-2 and the calibrated acceleration and angular velocity respectively: (6) (7) (8) In the formula, , and are the original time stamps of the accelerometer, gyroscope, and laser ranging module respectively, is the distance sequence at the moment of, represents the linear interpolation function.

[0037] Step S6-2: According to the pinhole imaging model, establish the relationship between the image plane coordinate increment and the physical space increment : (9) In the formula, is the physical space distance information. Thus, the dynamic pixel scale is defined as: (10) When the distance sequence of the LRM fluctuates less than 1%, take its time-domain average: (11) S7. Based on the calibrated acceleration and angular velocity in step S4 and the gravity vector in step S5, reconstruct the three-dimensional displacement trajectory of the camera platform in the world coordinate system, specifically including: Step S7-1: Select the overlapping interval of the acceleration and the angular velocity , , and perform equally spaced sampling, then: (12) (13) In interval, uniformly generate P sampling times : (14) Therefore, each sampling time is: (15) Step S7-2: Calculate the linear acceleration (m / s²), angular velocity (rad / s) and distance (m) at each sampling time Specifically: Using the distance obtained in step S4 and the linear interpolation functions of the calibrated acceleration and angular velocity, calculate the linear acceleration (m / s²), angular velocity (rad / s) and distance (m) at each sampling time : (16) (17) (18) Step S7-3: Update the attitude of the camera platform system relative to the world coordinate system. By introducing a quaternion to represent the initial attitude of the camera system relative to the world coordinate system, then: (19) Use a small-angle rotation vector ( ) to update the attitude of the world coordinate system, then: (20) In the formula, " " represents quaternion multiplication.

[0038] Step S7-4: Remove the gravity vector of the IMU acceleration and update the camera platform velocity and displacement. Let the velocity in the world coordinate system be , is the total number of time instants, which is the total number of time instants recorded by the IMU, and the displacement is , the initial velocity and displacement, then: (21) (22) Then at time , the gravity vector removed from the IMU acceleration is: (23) In the formula, is the initial pose of the camera system relative to the world coordinate system corresponding 3×3 rotation matrix.

[0039] Step S7-5: Update the camera platform speed and displacement at time , then: (24) (25) Step S7-6: Calculate the camera platform trajectory matrix, combine all in order to obtain a trajectory matrix of shape P×3, as shown in Figure 3 to achieve trajectory output: (26) wherein, is the continuous three-dimensional displacement trajectory of the camera platform in the world coordinate system.

[0040] S8. Map the three-dimensional displacement trajectory in step S7 to the image plane and construct a discrete PSF grid, specifically: Step S8-1: Perform planar projection on each sampling point , then (27) wherein, represents the world coordinate system displacement vector at the th moment , and respectively represent the pixel x-axis and y-axis coordinates in the image plane at the th moment, forming a three-dimensional plane trajectory mapping diagram as shown in Figure 4 . Among them, Figure 4 in (a), (b), and (c) are the trajectory mapping diagrams of the XY plane, XZ plane, and YZ plane respectively. Calculate the pixel coordinate extrema of all trajectory sampling points P: , (28) , (29) Step S8-2: Initialize the PSF grid, and let the PSF grid matrix .

[0041] Step S8-3: Calculate the pixel index of each point ([[]] , ) on the PSF matrix: , (30) wherein, and respectively represent the x-axis and y-axis coordinates of the discrete PSF point spread matrix grid.

[0042] After the cumulative count, if ( , ) falls within the range of [0, N−1]×[0, N−1], then: (31) Step S8-4: Construct a PSF convolution kernel with a total energy of 1 and normalize the grid matrix , then (32) The finally obtained normalized grid matrix is the non-uniform point spread function PSF.

[0043] S9. Apply the PSF of step S8 to the road disease color image of step S1 , and accelerate the convolution operation through the fast Fourier transform to generate a simulated jitter blurred image, specifically: Step S9-1: Color sharp image , then: (33) In the formula, is the image reading function, .

[0044] Convert and normalize the color space of the color sharp image, then: (34) (35) In the formula, represents the intensity value of the original sharp image at pixel , channel , normalized to [0, 1], respectively represent the width and height of the input image, , represents the color channel index, , respectively represent the colors R, G, B.

[0045] Step S9-2: Construct a jitter blurred image spatial domain and frequency domain convolution model, and define a spatial domain convolution function for each channel c, then: (36) In the formula, is the intensity value of the simulated jitter-blurred image at pixel (x, y) and channel c. is the PSF size (side length of the square, unit: pixel). When there is an out-of-boundary it is padded with zeros.

[0046] For each channel c and perform a two-dimensional fast Fourier transform, then: (37) (38) In the formula, represents the two-dimensional fast Fourier transform (FFT) operation.

[0047] At this time, the jitter-blurred image under channel c is: (39) (40) In the formula, represents the two-dimensional inverse FFT operation, is the spatial domain convolution operator.

[0048] Step S9-3: Perform value range clipping on the jitter-blurred image under channel c, then: (41) Quantize the clipped jitter-blurred image into 8-bit data, then: (42) Obtain the jitter-blurred image.

[0049] S10. After the jitter-blurred image in step S9 is generated, to ensure its optimal performance in terms of blur authenticity, trajectory reducibility, and application adaptability, it is necessary to perform model configuration and system evaluation on the key parameters introduced in this embodiment from the perspectives of trajectory geometry, imaging mapping, system accuracy, numerical stability, and perception performance for subsequent batch generation of jitter-blurred images, which are divided into five sub-steps, specifically: Step S10-1: Modeling and optimization of the PSF size To ensure that the PSF can fully cover the trajectory projection on the image plane and avoid cropping of the motion energy, set its side length to satisfy: (43) where, is the maximum trajectory offset radius, and the calculation formula is: (44) (45) (46) Among them, represents the side length (pixels) of the PSF convolution kernel; and are the pixel coordinates of the th trajectory point on the image plane; and represent the centroid of the trajectory; is the number of trajectory sampling points.

[0050] Step S10-2: Modeling the time resolution of the number of trajectory sampling points . In order to ensure that the time resolution of the trajectory can accurately reflect the high-frequency motion of the camera, the trajectory sampling interval needs to meet the Nyquist sampling condition: (47) (48) Among them, is the trajectory sampling time step (seconds); is the maximum estimated jitter frequency of the platform (Hz); and are the start and end boundaries of the trajectory time respectively; is the number of trajectory sampling points. By setting , the problems of PSF fragmentation and discontinuity caused by trajectory discretization are avoided.

[0051] Step S10-3: Modeling the zero-bias accuracy of the number of static samples static_n of the IMU. The zero-bias estimation accuracy of the IMU directly affects the stability of the subsequent trajectory. According to the statistical principle, its error variance is: (49) (50) Among them, , are the zero-bias estimations of acceleration and angular velocity respectively; , are the noise variances of the IMU acceleration and angular velocity respectively; represents the number of samples used for static estimation.

[0052] Step S10-4: Dynamic modeling and depth mapping of the pixel scale . The pixel scale maps the physical trajectory to the image space and is defined as: (51) (52) Among them, is the pixel / meter ratio at time t; is the equivalent focal length of the camera (pixels); represents the LRM real-time ranging value (meters); is the time-averaged pixel scale. Using dynamic can enhance the front and rear depth-of-field adaptability of the PSF, while is more suitable for scenes where the image depth changes slightly.

[0053] Step S10-5: Numerical stability modeling of the PSF normalization regularization term To prevent numerical instability of the PSF when the trajectory energy is sparse or distributed at a single point, a regularization term is introduced in the normalization: (53) where, the trajectory energy accumulation matrix; the regularization term; the value of the point spread function after normalization. This term effectively avoids the PSF energy from being normalized to zero or overly concentrated, ensuring numerical stability during the convolution process.

[0054] Step S10-6: Comparative experiment evaluation and parameter optimization feedback.

[0055] After completing the above parameter configuration, a comparative experiment needs to be carried out for the performance indicators in Table 1 to feedback the effectiveness of the parameter setting: Table 1 Performance Indicator Table

[0056] Through the subjective visual analysis and objective index evaluation of the generated images under different parameter combinations, the verification, optimization, and iteration of the blur simulation process of this embodiment can be realized.

[0057] In the specific implementation process, through Figure 5 the non-uniform PSF grid shown in Figure 6 it can be intuitively seen that the convolution kernel generated by fusing the IMU trajectory and LRM depth information in this embodiment accurately depicts the vibration amplitude and direction; while

[0058] Embodiment 2 In an embodiment of the present disclosure, a jitter blur simulation system for a road disease detection camera is provided, including: A data acquisition module, configured to: acquire multi-source sensing data of a road disease detection camera platform and perform data preprocessing, wherein the multi-source sensing data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by an inertial measurement unit, and depth information collected by a laser ranging module; A trajectory reconstruction module, configured to: reconstruct the continuous displacement trajectory of the camera in three-dimensional space based on the multi-source sensing data; A function formation module, configured to: map the reconstructed continuous displacement trajectory to the image plane to form a non-uniform point spread function; A blur simulation module, configured to: perform blur convolution on the acquired color image through the non-uniform point spread function to generate a simulated jitter blur image.

[0059] Embodiment 3 In an embodiment of the present disclosure, there is provided a computer program product, including a computer program, where when the computer program is executed by a processor, it implements the jitter blur simulation method of a road disease detection camera as described above.

[0060] Embodiment 4 In an embodiment of the present disclosure, there is provided a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, it implements the jitter blur simulation method of a road disease detection camera as described above.

[0061] Embodiment 5 In an embodiment of the present disclosure, there is provided an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes and implements the jitter blur simulation method of a road disease detection camera as described above.

[0062] The present disclosure is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so as to cause a series of operational steps to be performed on the computer or other programmable apparatus to generate a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for realizing the functions specified in one process or a plurality of processes and / or boxes. Figure 1 one process or a plurality of processes and / or boxes Figure 1 steps for realizing the functions specified in one box or a plurality of boxes.

[0064] Although the specific embodiments of the present disclosure have been described above in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that, based on the technical solutions of the present disclosure, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present disclosure.

Claims

1. A method for simulating the jitter blur of a road disease detection camera, characterized in that, Including: Obtain multi-source sensing data of a road disease detection camera platform and perform data preprocessing, where the multi-source sensing data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by an inertial measurement unit, and depth information collected by a laser ranging module; Based on the multi-source sensing data, reconstruct the continuous displacement trajectory of the camera in three-dimensional space; Map the reconstructed continuous displacement trajectory to the image plane to form a non-uniform point spread function; Through the non-uniform point spread function, perform blurred convolution on the obtained color image to generate a simulated jitter-blurred image.

2. The jitter blur simulation method of a road disease detection camera according to claim 1, characterized in that, The camera platform includes an inertial measurement unit, a laser ranging module, and a road disease detection camera fixed on a rigid bracket, and the relative positions of the three are ensured to be stable through the rigid bracket.

3. The jitter blur simulation method of a road disease detection camera according to claim 1, characterized in that, The preprocessing includes: Load and calibrate the acceleration and angular velocity data collected by the inertial measurement unit, remove the zero bias and estimate the gravity vector under the camera platform system; Based on the camera imaging geometric parameters and the depth information collected by the laser ranging module, dynamically model the pixel scale.

4. A method for simulating jitter blur of a road disease detection camera according to claim 3, characterized in that, The reconstruction of the continuous displacement trajectory of the camera in three-dimensional space is to reconstruct the continuous three-dimensional displacement trajectory of the camera platform through numerical integration in the world coordinate system based on the gravity vector and the preprocessed acceleration and angular velocity data.

5. The jitter blur simulation method of a road disease detection camera according to claim 3, characterized in that The formation of the non-uniform point spread function is to perform planar projection on the continuous displacement trajectory based on the pixel scale, and normalize and map it into a two-dimensional point spread function matrix to form a trajectory energy distribution and normalize it into a convolution kernel to obtain the non-uniform point spread function.

6. A method for simulating jitter blur of a road disease detection camera according to claim 1, characterized in that, The blurred convolution of the obtained color image is to use the fast Fourier transform to apply the non-uniform point spread function to the three channels of the color image respectively to obtain a simulated jitter-blurred image.

7. A method for simulating jitter blur of a road disease detection camera according to claim 1, characterized in that, It also includes optimizing and verifying the PSF size, trajectory sampling points, IMU stationary sample quantity, pixel scale, and regularization parameter based on the generated jitter-blurred image for subsequent batch generation of jitter-blurred images.

8. A jitter blur simulation system for a road disease detection camera, characterized in that, Including: A data acquisition module configured to: obtain multi-source sensing data of a road disease detection camera platform and perform data preprocessing, where the multi-source sensing data includes color images, camera imaging geometric parameters, acceleration and angular velocity data collected by an inertial measurement unit, and depth information collected by a laser ranging module; A trajectory reconstruction module configured to: reconstruct the continuous displacement trajectory of the camera in three-dimensional space based on the multi-source sensing data; A function formation module configured to: map the reconstructed continuous displacement trajectory to the image plane to form a non-uniform point spread function; A blur simulation module configured to: perform blurred convolution on the obtained color image through the non-uniform point spread function to generate a simulated jitter-blurred image.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, a jitter blur simulation method of a road disease detection camera as described in any one of claims 1-7 is implemented.

10. An electronic device, characterized in that, Including: A processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory so that the electronic device executes a method for simulating jitter blur of a road disease detection camera as described in any one of claims 1-7.

Images