Depth compensation system for motion blurred image

Abstract

The invention relates to the technical field of image processing, and discloses a depth compensation system for a motion blurred image. The depth compensation system for the motion blurred image not only can accurately calculate the motion speed and direction of an object in the image, but also can fully utilize time information to capture dynamic characteristics in a complex motion scene, and the method overcomes the limitation caused by fixed hypothesis and improves the accuracy of the motion blurred image. The system can still realize efficient fuzzy compensation under the condition of processing non-uniform motion or multi-direction motion, ensures that the compensated image is clearer and rich in details, and can further optimize the image quality, remove noise, enhance the contrast and correct the color by integrating the image post-processing unit. By means of the series of improvements, the visual effect of the image is improved, a more reliable basis is provided for subsequent image analysis and computer vision tasks, and the method has wide application prospects and practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and specifically to a depth compensation system for motion-blurred images. Background Art

[0002] Image compensation technology is an image processing method used to improve image quality. It aims to repair and enhance blurred, distorted, or noisy images to restore image clarity and details. This technology is of great significance in many application scenarios. For example, in the fields of motion image capture, remote sensing image processing, medical imaging, and surveillance video analysis, through effective image compensation means, the usability of visual information can be significantly improved, helping users to more clearly identify and analyze objects and scenes in the image. In addition, image compensation technology can also provide higher-quality inputs for subsequent computer vision tasks (such as object detection, image segmentation, etc.), improving the overall performance of the system.

[0003] Traditional motion-blurred image compensation methods often rely on fixed assumptions, such as assuming that the motion is uniform or a certain specific type of blur, which limits their applicability. In complex motion scenarios, the motion speed and direction of objects may change, resulting in unsatisfactory compensation effects. In addition, many existing technologies fail to fully utilize temporal information when processing high-frame-rate image sequences, resulting in the inability to accurately capture the motion characteristics of objects. In this case, the compensated image may still have obvious blur or distortion, reducing the visual effect and the accuracy of subsequent analysis. Summary of the Invention

[0004] (1) Technical Problems to be Solved In view of the deficiencies of the prior art, the present invention provides a depth compensation system for motion-blurred images, which can not only accurately calculate the motion speed and direction of objects in the image, but also fully utilize temporal information to capture the dynamic characteristics in complex motion scenarios. This method overcomes the limitations brought by fixed assumptions, enabling it to still achieve efficient blur compensation in the case of non-uniform motion or multi-directional motion, ensuring that the compensated image is clearer and richer in details. In addition, by integrating an image post-processing unit, the system can further optimize the image quality, remove noise, enhance contrast, and perform color correction, finally outputting high-quality clear motion images, thus solving the above problems.

[0005] (2) Technical Solutions To achieve the above object, the present invention provides the following technical solution: A depth compensation system for motion-blurred images, comprising a motion image acquisition unit, a motion image analysis unit, an image compensation unit, an image post-processing unit, and a clear image output unit; The motion image acquisition unit obtains an image sequence in a motion scene through a high-frame-rate camera, and calculates the time interval between each frame of images according to the timestamps of the acquired image sequence; The motion image analysis unit analyzes the image sequence in the acquired motion scene, calculates the motion blur intensity, the motion speed of the object in the image, and the motion direction of the moving object in the image, and transmits them to the image compensation unit; The image compensation unit uses the inverse motion blur algorithm to perform compensation processing on the blurred image according to the motion blur intensity, the motion speed of the object in the image, and the motion direction of the moving object in the image, and transfers the compensated image to the image post-processing unit; The image post-processing unit performs denoising, contrast enhancement, color correction, and sharpening processing on the compensated image and then sends it to the clear image output unit; The clear image output unit outputs the compensated clear motion image to the display screen.

[0006] Preferably, the formula for calculating the time interval between each frame of images is as follows:

[0007] In the formula, represents the time interval between each frame of images, represents the timestamp of the th frame of the image, represents the

[0008] th frame of the image.

[0009] In the formula, represents the motion blur intensity, represents the total number of pixels in the image, represents the square of the gradient value of the th pixel in the horizontal direction, represents the th pixel in the vertical direction, square of the gradient value of the represents the gradient magnitude of the

[0010] th pixel, and

[0011] In the formula, Represents the motion speed of an object in the image, Represents the position coordinates of the object at the nth frame, Represents the position coordinates of the object at the mth frame, Represents the time interval between each frame of the image.

[0012] Preferably, the calculation formula for the motion direction of the moving object in the image is as follows:

[0013] In the formula, Represents the motion direction of the moving object in the image, expressed by an angle, Represents the velocity component of the object in the horizontal direction, Represents the velocity component of the object in the vertical direction, Represents the inverse function used to obtain the motion direction of the moving object in the image.

[0014] Preferably, the formula for compensating the blurred image is as follows:

[0015] In the formula, Represents the compensated image, Represents the blurred image, Represents the motion blur kernel, Represents the Fourier transform, Represents the inverse Fourier transform.

[0016] Preferably, the formula for denoising the compensated image is as follows:

[0017] In the formula, Represents the pixel value of the denoised compensated image at position x, Represents the pixel value of the clear image after compensation in the neighborhood, Represents the Gaussian kernel function, Represents the normalization constant, , Represents the radius of the Gaussian kernel, , Represents the offset of the filter, Represents the double summation, which accumulates over all positions of the filter and then divides by the normalization constant to calculate the new pixel value.

[0018] Preferably, the formula for enhancing the contrast of the compensated image is as follows:

[0019] In the formula, represents the pixel value of the enhanced compensated image at position , represents the cumulative distribution function, which is used to map the pixel value to a new contrast range, represents the maximum range value of the pixel value.

[0020] Preferably, the formula for color correction of the compensated image is as follows:

[0021] In the formula, represents the pixel value of the color-corrected image at position , represents the white balance factor, which is used for image color correction.

[0022] Preferably, the calculation formula for sharpening processing of the compensated image is as follows:

[0023] In the formula, represents the pixel value of the sharpened compensated image at position , represents the sharpening intensity, represents the Laplacian operator, which is used to highlight the edges of the compensated image.

[0024] Compared with the prior art, the present invention provides a depth compensation system for motion-blurred images, having the following beneficial effects: The present invention can not only accurately calculate the motion speed and direction of objects in the image, but also make full use of time information to capture the dynamic features in complex motion scenes. This method overcomes the limitations brought by fixed assumptions, enabling it to still achieve efficient blur compensation in the case of non-uniform motion or multi-directional motion, ensuring that the compensated image is clearer and richer in details. In addition, by integrating an image post-processing unit, the system can further optimize the image quality, remove noise, enhance contrast, and perform color correction, and finally output a high-quality clear motion image. These series of improvements not only enhance the visual effect of the image, but also provide a more reliable basis for subsequent image analysis and computer vision tasks, having broad application prospects and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a schematic diagram of the system flow of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0026] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0027] Aiming at the problem that traditional motion blur image compensation methods often rely on fixed assumptions, resulting in unsatisfactory image compensation effects in complex motion scenes. At the same time, when processing high-frame-rate image sequences, the time information is not fully utilized, resulting in the inability to accurately capture the motion characteristics of objects, reducing the visual effect and the accuracy of subsequent analysis. For this reason, a deep compensation system for motion blur images is proposed. Please refer to Figure 1 , the system includes a motion image acquisition unit, a motion image analysis unit, an image compensation unit, an image post-processing unit, and a clear image output unit; The motion image acquisition unit uses a 120fps high-frame-rate camera to quickly capture image sequences in the motion scene. Through the precise timestamp recording function, this unit can assign a corresponding time mark to each frame of the image, thereby realizing precise control of the image acquisition process. After obtaining the image sequence, the system will analyze these timestamps and calculate the time interval between adjacent frames. This process usually relies on high-precision clock synchronization technology to ensure data consistency and accuracy. Through the analysis of these time intervals, the motion analysis algorithm can better understand the dynamic changes of moving objects, and then realize the precise recognition and tracking of motion behaviors. The calculation formula of the time interval between each frame of the image is as follows:

[0028] By calculating the time interval between each frame of the image, the dynamic characteristics of the moving object can be accurately analyzed. The smaller the time interval, the clearer the details of the action, which is helpful for subsequent motion analysis. In the formula, represents the time interval between each frame of the image, represents the timestamp of the image of the th frame, represents the timestamp of the image of the The motion image analysis unit calculates the motion blur intensity of each frame of the image using gradient-based edge detection and frequency domain analysis. This process involves performing a Fourier transform on the image to identify the attenuation of frequency components in the blurred image, thereby quantifying the degree of blur. Subsequently, the motion speed and direction of the object in the image are calculated. By analyzing the pixel displacement between adjacent frames of the image, the speed changes and motion trajectories of the object during motion can be accurately captured. After completing these calculations, the system transmits the extracted information such as the motion blur intensity, object motion speed, and motion direction to the image compensation unit through a high-bandwidth data channel for subsequent targeted blur compensation, denoising processing, and image enhancement, where: The calculation formula for the motion blur intensity is as follows:

[0029] The calculation of the motion blur intensity provides a quantitative index of the degree of blur, enabling subsequent compensation processing to be adjusted according to the specific blur intensity. In the formula, represents the motion blur intensity, represents the total number of pixels in the image, represents the square of the gradient value of the th pixel in the horizontal direction, represents the th pixel in the vertical direction, square of the gradient value of the represents the gradient magnitude of the th pixel.

[0030] represents the motion speed of the object in the image, represents the position coordinates of the object at the th frame, represents the position coordinates of the object at the th frame, represents the time interval between each frame of the image. By analyzing the motion speed, intelligent understanding of the behavior of objects in the scene can be achieved, such as distinguishing normal speed from abnormal speed, and thus different processing can be carried out; The calculation formula for the motion direction of the moving object in the image is as follows:

[0031] In complex scenarios, clarifying the direction of motion can help distinguish different objects and improve the accuracy of multi-object tracking. By calculating the direction of motion, directional compensation for motion blur can be carried out specifically, making the compensation result more accurate. In the formula, represents the direction of motion of the moving object in the image, expressed by an angle. represents the velocity component of the object in the horizontal direction. represents the velocity component of the object in the vertical direction. represents the inverse function used to obtain the direction of motion of the moving object in the image. The analysis of the direction of motion helps to identify the motion pattern of the object, providing important information for subsequent intelligent analysis and behavior recognition. The formula for compensating the blurred image is as follows:

[0032] Compensating the blurred image can significantly improve the clarity of the image. By applying the motion blur compensation algorithm, the details in the image can be effectively restored, making the originally blurred parts sharper. This improvement in clarity makes the important information in the image easier to identify, facilitating analysis and judgment by the observer. In the formula, represents the compensated image. represents the blurred image. represents the motion blur kernel. represents the Fourier transform. represents the inverse Fourier transform. The compensated image lays a good foundation for subsequent image analysis and processing. Whether in object recognition, tracking, or motion analysis, a clear image can improve the accuracy and reliability of the algorithm. The image quality directly affects the effect of computer vision tasks, and compensation processing can significantly increase the success rate of these tasks. The image post-processing unit performs denoising, contrast enhancement, color correction, and sharpening on the compensated image, where: The formula for denoising the compensated image is as follows:

[0033] Denoising processing can effectively remove the random noise in the image, improve the overall quality and clarity of the image, and make the details more obvious. In the formula, represents the pixel value of the denoised compensated image at position . represents the pixel value of the clear compensated image in the neighborhood. represents the Gaussian kernel function. represents the normalization constant. , represents the radius of the Gaussian kernel, , represents the offset of the filter, represents double summation, accumulating over all positions of the filter and then dividing by the normalization constant to calculate the new pixel value. After denoising, the edges and features of the objects in the image become clearer, improving the user's visual experience and making the image more natural and easy to understand; The formula for compensating image enhancement contrast is as follows:

[0034] Enhancing contrast can make the light and dark contrast of the image more obvious, thus improving the visual effect and making important features more prominent. In the formula, represents the pixel value of the compensated image after enhancement at position , represents the cumulative distribution function, which is used to map the pixel value to a new contrast range, represents the maximum range value of the pixel value. After improving the contrast, the details of the dark and bright parts can be better presented, facilitating observation and analysis; The formula for compensating image color correction is as follows:

[0035] Color correction can restore the true color of the image, making the image present a more natural color effect. Through color correction, the color consistency of multiple frames of images can be ensured, improving the stability of subsequent image processing. In the formula, represents the pixel value of the image after color correction at position , represents the white balance factor, which is used for image color correction; The calculation formula for compensating image sharpening is as follows:

[0036] Sharpening can enhance the details of the image, improve the edge sharpness, and make the contours of the objects in the image more distinct. By sharpening, the recognizability of the key information in the image is improved, making the compensated image clearer and easier to recognize. In the formula, represents the pixel value of the compensated image after sharpening at position , represents the sharpening intensity, represents the Laplacian operator, which is used to highlight the edges of the compensated image; The clear image output unit is responsible for outputting the compensated and post-processed clear motion image to the display screen in high quality for the user to observe and analyze in real time. Using a high-performance image processing interface, this unit is seamlessly connected to the image post-processing unit to ensure high-speed and stable data transmission. With the high-bandwidth video transmission protocol HDMI, the output unit can support high-resolution image display, ensuring the integrity of image details and the accuracy of colors.

[0037] Through the comprehensive application of the above system, not only can the motion speed and direction of objects in the image be accurately calculated, but also the time information is fully utilized to capture the dynamic features in complex motion scenes, overcoming the limitations brought by fixed assumptions. This enables efficient blur compensation even when dealing with non-uniform motion or multi-directional motion, ensuring that the compensated image is clearer and more detailed. At the same time, it can further optimize the image quality, remove noise, enhance contrast, and perform color correction, finally outputting a high-quality clear motion image.

[0038] Example 1: In this experiment, the success of the depth compensation system for motion-blurred images was verified. The following data was collected by the motion image acquisition unit. The timestamp of the first frame image is 0.000 seconds, the timestamp of the second frame image is 0.033 seconds, and the timestamp of the third frame image is 0.066 seconds. The object position in the image , , , and the blurred images of different frames of the segment are as follows:

[0039]

[0040]

[0041] Calculate the time interval between each frame of the image: , ; Since the blur evaluation varies with the mean square error of the image, the blur degree of each image is obtained as: , , , that is, the blur intensity ; Next, calculate the motion speed and direction of the object in the image. , , since the large change in the final motion direction indicates moving to the right; Using a motion blur kernel of 3*3 matrix, through the reverse motion blur algorithm, since the image moves too fast, the compensated and restored image is:

[0042] Due to the failure of the algorithm, the image cannot be repaired, resulting in serious distortion. Therefore, even after noise removal, the image still shows unreachable blurring and loss:

[0043] According to the above results, due to the loss of motion information caused by the rapid movement of the object, the compensation theoretically cannot restore the image, resulting in significant distortion and the inability to restore the information. At this time, re - compensation is required; Example two: In this experiment, the compensation success of the depth compensation system for motion - blurred images was verified. The following data was collected by the motion image acquisition unit. The timestamp of the first - frame image is 0.000 seconds, the timestamp of the second - frame image is 0.033 seconds, and the timestamp of the third - frame image is 0.066 seconds. The object position in the image , , , and the blurred images of different frames of the segment are as follows:

[0044]

[0045]

[0046] Calculate the time interval of each frame of the image: , ; Since the blur evaluation changes with the image, the blur degree of each image is obtained as: , , , that is, the blur intensity ; Next, calculate the motion speed and direction of the object in the image, , , since the motion direction only moves on the x - axis in the image, so ; Using a motion blur kernel of 3*3 matrix, through the reverse motion blur algorithm, the restored image is:

[0047] Through subsequent noise and contrast enhancement, the final image after denoising and enhancement processing is obtained:

[0048] According to the above results, it shows that there is only a small amount of blurring effect after compensation, so the compensation is successful.

[0049] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A depth compensation system for motion blurred images, characterized in that: It includes a motion image acquisition unit, a motion image analysis unit, an image compensation unit, an image post-processing unit and a clear image output unit; The motion image acquisition unit acquires an image sequence in a motion scene through a high frame rate camera, and calculates the time interval between each frame of the image according to the timestamp of the acquired image sequence; The motion image analysis unit analyzes the image sequence in the captured motion scene, calculates the motion blur intensity, the motion speed of the object in the image, and the motion direction of the moving object in the image, and transmits them to the image compensation unit; The image compensation unit uses an inverse motion blur algorithm to compensate the blurred image according to the motion blur intensity, the motion speed of the object in the image, and the motion direction of the moving object in the image, and transmits the compensated image to the image post-processing unit; The image post-processing unit performs denoising, contrast enhancement, color correction and sharpening on the compensated image and then sends it to the clear image output unit; The clear image output unit outputs the compensated clear motion image onto a display screen.

2. The depth compensation system for motion blurred images according to claim 1, characterized in that: The time interval calculation formula between each frame image is as follows: In the formula, Represents the time interval between each frame of image. Indicates The timestamp of the frame's image, Indicates The timestamp of the frame's image.

3. The depth compensation system for motion blurred images according to claim 2, characterized in that: The calculation formula of the motion blur intensity is as follows: In the formula, Indicates the intensity of motion blur, represents the total number of pixels in the image, Indicates The square of the gradient value of the pixel in the horizontal direction, Indicates The square of the gradient value of the pixel in the vertical direction, Indicates The gradient magnitude of pixels, It means that the gradient amplitude of the pixels is accumulated and then divided by the total number of pixels in the image to obtain the motion blur intensity.

4. The depth compensation system for motion blurred images according to claim 3, characterized in that: The calculation formula for the moving speed of the object in the image is as follows: In the formula, Indicates the speed of the object in the image. Indicates that the object is The position coordinates of the frame, Indicates that the object is The position coordinates of the frame, Indicates the time interval between each frame.

5. The depth compensation system for motion blurred images according to claim 4, characterized in that: The calculation formula for the moving direction of the moving object in the image is as follows: In the formula, Indicates the direction of motion of a moving object in an image, expressed as an angle. represents the velocity component of the object in the horizontal direction, represents the velocity component of the object in the vertical direction, It indicates that the inverse function is used to obtain the moving direction of the moving object in the image.

6. The depth compensation system for motion blurred images according to claim 5, characterized in that: The formula for compensating the blurred image is as follows: In the formula, represents the compensated image, represents a blurred image, represents the motion blur kernel, represents the Fourier transform, represents the inverse Fourier transform.

7. The depth compensation system for motion blurred images according to claim 6, characterized in that: The compensation image denoising formula is as follows: In the formula, Indicates the compensated image after denoising at position The pixel value of Represents the pixel value of the clear image in the neighborhood after compensation, represents the Gaussian kernel function, represents the normalization constant, , represents the radius of the Gaussian kernel, , represents the offset of the filter, Represents double summation, which accumulates all positions of the filter and then divides by a normalization constant to calculate a new pixel value.

8. The depth compensation system for motion blurred images according to claim 7, characterized in that: The formula for compensating the image to enhance contrast is as follows: In the formula, Indicates that the enhanced compensated image is at position The pixel value of represents the cumulative distribution function, which is used to map pixel values ​​to a new contrast range, Indicates the maximum range of pixel values.

9. The depth compensation system for motion blurred images according to claim 8, characterized in that: The formula for compensating image color correction is as follows: In the formula, Indicates the color-corrected image at position The pixel value of Represents the white balance factor, used for image color correction.

10. The depth compensation system for motion blurred images according to claim 9, characterized in that: The calculation formula for the compensation image sharpening process is as follows: In the formula, Indicates that the compensated image after sharpening is at position The pixel value of Indicates the sharpening strength, Represents the Laplacian operator, which is used to highlight the edge of the compensated image.

Images