A method, device, medium and product for removing complex motion blur from an image

Abstract

This invention discloses a method, apparatus, medium, and product for removing complex motion blur from images, relating to the field of image processing. The method includes: acquiring a motion image of a moving object in a target scene; performing optical flow tracing on the motion image using dense optical flow to obtain an optical flow map; classifying the pixels in the optical flow map according to the inter-frame displacement direction to obtain multiple masks of different pixel categories; segmenting the motion image using the multiple masks of different pixel categories to obtain multiple image blocks; calculating the point spread function of each image block using a differential autocorrelation function; restoring and recombining all image blocks according to the point spread function to obtain a reconstructed image of the moving object; the reconstructed image is an image with motion blur removed from the motion image. This invention can accurately and efficiently remove complex motion blur from images.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method, apparatus, medium, and product for removing complex motion blur from images. Background Technology

[0002] Image deblurring technology plays a vital role in industrial manufacturing, medical imaging, and other fields by removing blur artifacts, improving image quality, and extracting valuable information from motion-blurred images. Motion blur is a phenomenon commonly observed when capturing images of fast-moving objects. It occurs when the relative speed between the object and the camera exceeds a certain threshold during the exposure time. This causes the light signal reflected from the object to act on a MOS capacitor, activating another MOS capacitor before reaching its stable value within the integration time. The main challenge in current image motion blur removal tasks lies in the unknown motion of the target, the different motions of different parts of the same object relative to the camera due to non-rigid body changes, and the varying motion errors in the imaging, which existing methods struggle to address.

[0003] In recent years, motion blur removal algorithms for static scenes have achieved considerable success. However, for motion blur removal in dynamic scenes, there is still no universally accepted algorithm. Constructing the point spread function is crucial for image deblurring. Non-rigid motion blur in dynamic scenes is a time- and space-varying process, and its point spread function should also satisfy these time- and space-varying characteristics. Existing learning-based algorithms face difficulties in acquiring motion-blurred image datasets and in training unsupervised learning methods. Traditional motion blur removal methods struggle to simultaneously meet accuracy and speed requirements, and they cannot accurately construct point spread functions for scenarios where different parts of the same object exhibit varying motion characteristics. Therefore, how to accurately and efficiently remove complex motion blur from images has become an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, medium, and product for removing complex motion blur from images, so as to accurately and efficiently remove complex motion blur from images.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for removing complex motion blur from an image, the method comprising:

[0007] Acquire motion images of moving objects in the target scene;

[0008] The motion image is optically traced using the dense optical flow method to obtain an optical flow map.

[0009] The pixels in the optical flow map are classified according to the inter-frame displacement direction to obtain multiple masks with different pixel categories;

[0010] The moving image is segmented using multiple masks of different pixel categories to obtain multiple image blocks;

[0011] The point spread function of each image patch is calculated using the differential autocorrelation function;

[0012] The image blocks are restored and recombined according to the point spread function to obtain the reconstructed image of the moving object; the reconstructed image is an image in which motion blur has been removed from the moving image.

[0013] Optionally, dense optical flow is used to perform optical flow tracing on the moving image to obtain an optical flow map, specifically including:

[0014] The moving objects in the motion image are segmented to obtain a motion segmentation image;

[0015] The dense optical flow method is used to trace the displacement of pixels in the motion segmentation image between different frames, and the inter-frame error of the image is determined based on the displacement.

[0016] The optical flow map is determined based on the inter-frame error of the image.

[0017] Optionally, the pixels in the optical flow map are classified according to the inter-frame displacement direction to obtain multiple masks with different pixel categories, specifically including:

[0018] The pixels in the optical flow diagram with displacement direction angles between 0 and π are divided into multiple intervals on an average basis.

[0019] Binarization and morphological dilation are performed on the pixels in each interval of the optical flow map to obtain multiple masks of different pixel categories; one interval corresponds to one mask of one pixel category.

[0020] Optionally, the point spread function of each image patch is calculated using the differential autocorrelation function, specifically including:

[0021] For any image patch, perform differential autocorrelation operations on the image patch in the x-axis and y-axis directions in an orthogonal coordinate system to obtain the projection width of the motion blur trail of the image patch in the x-axis direction and the projection width in the y-axis direction.

[0022] For any image block, the motion blur direction angle is calculated based on the projection width of the motion blur trail of the image block in the x-axis direction and the projection width in the y-axis direction.

[0023] For any image patch, the image patch is rotated clockwise according to the motion blur direction angle to obtain the rotated image, and the rotated image is subjected to differential autocorrelation operation in the x-axis direction to obtain the motion blur scale;

[0024] For any image patch, a point spread function for the image patch is constructed based on the motion blur direction angle and the motion blur scale.

[0025] Optionally, the reconstructed image of the moving object is obtained by restoring and recombining all image patches according to the point spread function, specifically including:

[0026] Based on the point spread function, Wiener filtering is used to perform deconvolution restoration on each image block to obtain the restored image of each image block;

[0027] All the restored images are fused together to obtain the reconstructed image of the moving object.

[0028] Optionally, the formula for calculating the motion blur direction angle is:

[0029]

[0030] Where angel represents the motion blur direction angle; d X d represents the projection width of the motion blur trail of an image patch along the x-axis. Y This represents the projection width of the motion blur trail of an image patch along the y-axis.

[0031] Optionally, a differential autocorrelation operation is performed on the rotated image along the x-axis to obtain the motion blur scale, specifically including:

[0032] Perform a differential autocorrelation operation on the rotated image along the x-axis to determine the distance between the two negative peaks in the differential autocorrelation function of the rotated image;

[0033] The absolute value of half the distance between the two negative peaks in the differential autocorrelation function of the rotated image is determined as the motion blur scale.

[0034] The present invention also provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method for removing complex motion blur from an image.

[0035] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method for removing complex motion blur from an image.

[0036] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for removing complex motion blur from an image.

[0037] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0038] This invention classifies pixels in the optical flow map according to their inter-frame displacement direction to obtain multiple masks of different pixel categories. These masks are then used to segment the moving image, resulting in multiple image blocks, thus achieving dynamic image segmentation and reducing motion angle errors. The differential autocorrelation function is used to calculate the point spread function of each image block, ensuring the accuracy of the point spread function construction and consequently the accuracy of the reconstructed image. This also guarantees the real-time performance of motion blur removal, meeting the requirements for online motion blur removal. Therefore, this invention can accurately and efficiently remove complex motion blur from images. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic flowchart of the method for removing complex motion blur from images provided in Embodiment 1 of the present invention;

[0041] Figure 2 The point spread function calculation flowchart provided in Embodiment 1 of this invention;

[0042] Figure 3 This is a schematic diagram illustrating the relationship between the projection width and the extreme points of the differential autocorrelation function provided in Embodiment 1 of the present invention;

[0043] Figure 4 This is a schematic diagram illustrating the relationship between the projection width of the two orthogonal directions of motion blur and the motion blur angle provided in Embodiment 1 of the present invention;

[0044] Figure 5 This is a diagram illustrating the implementation process of the method provided in Embodiment 1 of the present invention in a practical application;

[0045] Figure 6 A diagram illustrating the effect of applying the method provided in Example 1 to an aircraft landing video;

[0046] Figure 7 A comparison diagram showing the restoration effect of the method provided in Embodiment 1 of the present invention with that of existing similar methods;

[0047] Figure 8 This is an experimental result diagram of the method provided in Embodiment 1 of the present invention on an aircraft landing video;

[0048] Figure 9 This is a diagram of the internal structure of a computer device. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] The purpose of this invention is to provide a method, apparatus, medium, and product for removing complex motion blur from images, aiming to accurately and efficiently remove complex motion blur from images.

[0051] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Example 1

[0053] like Figure 1 As shown, the method for removing complex motion blur from an image in this embodiment includes:

[0054] Step 101: Acquire motion images of moving objects in the target scene. These motion images contain motion blur and are used as the original images for subsequent processing. The motion image comprises multiple frames representing a single motion process.

[0055] Step 102: Perform optical flow tracing on the motion image using the dense optical flow method to obtain an optical flow map.

[0056] Step 103: Classify the pixels in the optical flow map according to the inter-frame displacement direction to obtain multiple masks of different pixel categories.

[0057] Step 104: The moving image is segmented using multiple masks of different pixel categories to obtain multiple image blocks.

[0058] Step 105: Calculate the point spread function of each image patch using the differential autocorrelation function.

[0059] Step 106: Restore and reassemble all image blocks according to the point spread function to obtain the reconstructed image of the moving object.

[0060] The reconstructed image is an image in which motion blur has been removed from the motion image, i.e., an image of a clear and complete moving object.

[0061] In one example, step 102 specifically includes:

[0062] (1) Segment the moving object (i.e. the target foreground) in the motion image to obtain a motion segmentation image.

[0063] Specifically, a sample set is established for each pixel. The sampled values ​​in the sample set are the pixel values ​​of the past pixels and the pixel values ​​of its neighbors. Then, each new pixel value is compared with the sample set in Euclidean space to determine whether it belongs to the background, thus completing the segmentation of the target foreground. Because the foreground texture is complex and changes rapidly, while the background is flat and changes slowly, if a point is identified as a background point 10 times consecutively, it is considered a "long-term background region," and its sample set sampling rate is reduced to 1 / 10 of that of the foreground points.

[0064] (2) The dense optical flow method is used to trace the displacement of pixels in the motion segmentation image between different frames, and the inter-frame error of the image is determined based on the displacement, and the optical flow map is determined based on the inter-frame error of the image.

[0065] Specifically, the dense optical flow method is used to calculate the pixel displacement between every 5 frames of images, record its direction, determine the inter-frame error of the images, and obtain the optical flow map.

[0066] In one example, step 103 specifically includes:

[0067] (1) In the optical flow diagram, the pixels are classified according to their frame displacement direction. Specifically, the pixels in the optical flow diagram with displacement direction angle θ between 0 and π are divided into multiple intervals. For example, the pixels with displacement direction angle θ between 0 and π are divided into 9 intervals, and the length of each interval is π / 9.

[0068] (2) Binarize and morphologically dilate the pixels in each interval of the optical flow map to obtain multiple masks for different pixel categories; one interval corresponds to one mask for one pixel category. For example, for the optical flow of 9 intervals... Figure 2 Values ​​are then converted, and morphological dilation is performed to obtain masks for different pixel categories (different displacement directions). In subsequent step 104, specifically, the original image (i.e., the moving image) can be segmented using the above 9 masks to obtain 9 image patches with pixels whose inter-frame displacement directions are similar, i.e., 9 image patches.

[0069] In one example, step 105 specifically includes:

[0070] (1) For any image patch, perform differential autocorrelation operations on the image patch in the x-axis and y-axis directions in an orthogonal coordinate system to obtain the projection width d of the motion blur trail of the image patch in the x-axis direction. X and the projection width d in the y-axis direction Y .

[0071] (2) For any image block, calculate the motion blur direction angle based on the projection width of the motion blur trail of the image block in the x-axis direction and the projection width in the y-axis direction.

[0072] (3) For any image block, rotate the image block clockwise according to the motion blur direction angle to obtain the rotated image, and perform differential autocorrelation operation on the rotated image in the x-axis direction to obtain the motion blur scale.

[0073] Specifically, performing a differential autocorrelation operation on the rotated image along the x-axis to obtain the motion blur scale involves: performing a differential autocorrelation operation on the rotated image along the x-axis to determine the distance between two negative peaks in the differential autocorrelation function of the rotated image; and determining the absolute value of half the distance between the two negative peaks in the differential autocorrelation function of the rotated image as the motion blur scale.

[0074] (4) For any image patch, construct the point spread function (PSF) of the image patch based on the motion blur direction angle and the motion blur scale.

[0075] In practical applications, a specific implementation process of the above steps is as follows:

[0076] The process for calculating the spread function of motion blur points in an image is as follows: Figure 2 As shown. Formulas (1) and (2) demonstrate the difference autocorrelation function calculation of the image patch (motion-blurred image) g(x,y) along the x-axis (horizontal axis), the extreme point x = ±Δx of the difference autocorrelation function, and the projection width d of the motion-blurred image patch along that direction. X The relationship, d X =|Δx|, such as Figure 3 As shown in part (a), Figure 3 In part (a), the horizontal axis represents the variable Δx of the differential autocorrelation function in the x-direction of the input image, and the vertical axis represents the value of the differential autocorrelation function. First, calculate the difference autocorrelation function of the input image (image patch) along the two orthogonal coordinate axes to obtain the projection width d of the motion blur trail in these two directions. X and d Y Projection width d Y The calculation principle and projection width d XThe calculation principle is similar and will not be repeated here. Projection width d Y The relationship with the extreme points of the difference autocorrelation function is as follows: Figure 3 As shown in part (b). Figure 3 In part (b), the horizontal axis represents the variable Δy of the differential autocorrelation function in the y-direction of the input image, and the vertical axis represents the value of the differential autocorrelation function.

[0077] g(x,y i ) = h X (x)*f(x, y) i )+n(x,y i (1)

[0078]

[0079] Where g(x,y) i ) represents the i-th row of image patch g(x,y), h X (x) represents the motion fuzziness degradation function in the x-direction. h X The first-order difference function of (x), f(x,y) i ) represents the i-th row of the ideal, clear image f(x,y), and n(x,y) represents the n-th row. i ) represents the additive noise applied to the i-th row of the ideal, clear image f(x,y). Let R represent the difference autocorrelation function of image patch g(x,y). ff (Δx) represents the autocorrelation function of the ideal, sharp image f(x,y). express The value of R at x = 0 ff (Δx-d X f(xd) represents the ideal, sharp image. X The autocorrelation function of y, R ff (Δx+d X f(x+d) represents the ideal, sharp image. X The autocorrelation function of y).

[0080] The g(x,y) obtained from formula (1) above i ) and f(x,y i The relationship between the two determines their difference autocorrelation functions. and R ff The relationship can be determined mathematically. X for Half the distance between the two negative peaks.

[0081] like Figure 4As shown, the motion blur direction angle angel is calculated according to formula (3). Then, after rotating g(x,y) clockwise by angel, a differential autocorrelation calculation is performed on the rotated image in the x-direction, as shown. Figure 3 As shown in part (c). Figure 3 In part (c), the horizontal axis represents the variable Δs of the differential autocorrelation function in the x-direction after the input image is rotated, and the vertical axis represents the value of the differential autocorrelation function. According to d X The motion fuzzy scale is obtained by the relationship =|Δx|, as shown in formula (4).

[0082]

[0083] scale = |d S | (4)

[0084] Where angel represents the motion blur direction angle; d X d represents the projection width of the motion blur trail of an image patch along the x-axis. Y d represents the projection width of the motion blur trail of an image patch along the y-axis, and scale represents the motion blur scale; S This represents half the distance between the two negative peaks in the differential autocorrelation function of the rotated image.

[0085] In one example, step 106 specifically includes:

[0086] Based on the point spread function, Wiener filtering is used to perform deconvolution restoration on each image block, resulting in a restored image for each block. All restored images are then fused to obtain a reconstructed image of the moving object, i.e., a clear and complete image. Specifically: During the fusion process, the brightness of the image blocks is adjusted to maintain consistency with the average grayscale value of the original image blocks. For pixels in the overlapping areas of each image block (caused by the dilation operation during mask creation, aimed at reducing ringing effects), the arithmetic mean of the pixel values ​​at that point (excluding blocks with a pixel value of 0 at that point) is taken as its pixel value.

[0087] In practical applications, the overall process of the above-mentioned method for removing complex motion blur from images is as follows: Figure 5 As shown, specifically:

[0088] Step 1: Acquire the original image and calculate the inter-frame error.

[0089] (1) Overall segmentation of the foreground of the image target.

[0090] (2) Inter-frame error of the tracking image.

[0091] Step 2: Image pixel classification and image block segmentation.

[0092] Step 3: Calculate the point spread function for each image patch. For the multiple image patches obtained in Step 2, use formulas (1)-(4) to obtain their respective point spread functions.

[0093] Step 4: Tile restoration and image reconstruction.

[0094] The effectiveness of the above-mentioned method for removing complex motion blur from images was verified below.

[0095] Existing learning-based algorithms face challenges such as the difficulty in acquiring datasets for motion-blurred images and the difficulty in training unsupervised learning methods. Traditional motion blur removal methods, on the one hand, struggle to simultaneously meet the requirements of accuracy and speed, and on the other hand, they cannot effectively remove motion blur when different parts of the same object exhibit varying degrees of motion.

[0096] This embodiment employs a dynamic segmentation method combined with a differential autocorrelation function. The former segments different motion states of the same object, with the motion angle error of each part not exceeding 20 degrees; the latter, while maintaining a certain level of accuracy, ensures the real-time performance of motion blur removal, meeting the requirements for online removal of complex motion blur.

[0097] The average gradient of an image reflects the rate of pixel change, the root mean square error (RMSE) reflects the degree of gray-level difference between images, and structural similarity (SSIM) comprehensively reflects the similarity between images in terms of brightness, contrast, and structure. Average gradient, RMSE, and SSIM are important metrics for evaluating the performance of image restoration algorithms. For the same scene, the larger the average gradient value, the sharper the image and the higher its visual clarity. For two images, the smaller the RMSE, the closer their gray-level distributions are; the closer the SSIM is to 100%, the more similar they are.

[0098] The method of this embodiment is applied to the task of removing motion blur during the landing of a carrier-based aircraft (the relative speed between the aircraft and the camera is approximately 100 m / s). The effect is shown in the figure below. Figure 6 As shown. A comparison of the method in this embodiment with existing similar methods is as follows. Figure 7 As shown, Figure 7 Part (a) shows the original image. Figure 7 Part (b) shows the effect of the image de-motion blurring method (RT method) based on Radon transform and LoG transform. Figure 7 Section (c) shows the effect diagram of the method of this embodiment. Experimental results evaluating the performance of the image restoration algorithm using average gradient, root mean square error of gray level (RMSE), and structural similarity (SSIM) are as follows. Figure 8As shown, the method in this embodiment significantly improves the average gradient of the reconstructed image (an average increase of 162.4%), significantly reduces the root mean square error (RMSE) (an average reduction of 41.6%), and significantly improves the structural similarity (SSIM) (an average reduction of 61.2% in the maximum distance to "100%)), all of which are superior to existing methods.

[0099] The method in this embodiment dynamically partitions and reassembles an image. The first step is to calculate the inter-frame error. Dense optical flow is used to trace the displacement of image pixels between different frames. The second step is to classify and block the image pixels. A mask is generated to divide the image into several categories according to the inter-frame displacement direction of the pixels, and groups of pixels of the same category are divided into patches. The third step is to calculate the point spread function. The point spread function of each patch is calculated using the method proposed in this patent. The fourth step is image restoration and reconstruction. Wiener filtering and deconvolution are performed on each patch for restoration, and the restored patches are assembled together according to the mask positions, with overlapping parts being smoothly blended. This method achieves dynamic partitioning and restoration of the image.

[0100] This method calculates the point spread function in three steps: First, it calculates the direction of motion blur. Differential autocorrelation is performed on the image along two axes of an orthogonal coordinate system to obtain the width of the pixel's motion blur in these two directions, and the direction of motion blur is calculated accordingly. Second, it rotates the image. Based on the motion blur direction obtained in the first step, the image is rotated by an angle so that the motion blur direction is parallel to the horizontal axis of the image coordinate system. Third, it calculates the scale of motion blur. Differential autocorrelation is performed on the rotated image along the horizontal axis to obtain the scale of the motion blur. In summary, the motion blur direction and scale parameters used to construct the point spread function are obtained. This method calculates the point spread function based on the differential autocorrelation function.

[0101] The method in this embodiment has the following advantages: 1) In the pixel classification and segmentation stage, the dense optical flow method is used to roughly estimate the motion direction of image pixels through inter-frame error, which ensures the speed of image segmentation and constrains the difference in motion direction of pixels within the same block to within 20 degrees; 2) In the point spread function calculation stage, a method based on differential autocorrelation function is proposed, which transforms the direction and scale parameters of the point spread function into the calculation of the minimum point of the cubic differential autocorrelation function. Compared with similar methods, the solution process of two ternary functions is transformed into the solution process of three binary functions, which reduces the amount of computation. At the same time, the differential autocorrelation operation also eliminates the influence of background noise and clutter, which is beneficial for processing low-brightness scenes.

[0102] Example 2

[0103] This embodiment provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps of the method for removing complex motion blur from an image in Embodiment 1.

[0104] Example 3

[0105] This embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method for removing complex motion blur from an image in Embodiment 1.

[0106] Example 4

[0107] This embodiment provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method for removing complex motion blur from an image in Embodiment 1.

[0108] Example 5

[0109] This embodiment provides a computer device, which may be a database, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores pending transactions. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the method for removing complex motion blur from images in Embodiment 1.

[0110] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0111] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0112] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for removing complex motion blur from an image, characterized by, The method includes: Acquire motion images of moving objects in the target scene; The motion image is optically traced using the dense optical flow method to obtain an optical flow map. The pixels in the optical flow map are classified according to the inter-frame displacement direction to obtain multiple masks with different pixel categories; The moving image is segmented using multiple masks of different pixel categories to obtain multiple image blocks; The point spread function of each image patch is calculated using the differential autocorrelation function; The image patches are restored and recombined according to the point spread function to obtain the reconstructed image of the moving object; the reconstructed image is an image in which motion blur has been removed from the motion image. The motion image is subjected to optical flow tracing using the dense optical flow method to obtain an optical flow map, specifically including: The moving objects in the motion image are segmented to obtain a motion segmentation image; The dense optical flow method is used to trace the displacement of pixels in the motion segmentation image between different frames, and the inter-frame error of the image is determined based on the displacement. The optical flow map is determined based on the inter-frame error of the image; The pixels in the optical flow map are classified according to the inter-frame displacement direction to obtain multiple masks with different pixel categories, specifically including: The pixels in the optical flow diagram with displacement direction angles between 0 and π are divided into multiple intervals on an average basis. Binarization and morphological dilation are performed on the pixels in each interval of the optical flow map to obtain multiple masks for different pixel categories; one interval corresponds to one mask for one pixel category. The point spread function of each image patch is calculated using the differential autocorrelation function, specifically including: For any image block, the image block is subjected to a difference auto-correlation operation in the orthogonal coordinate system x direction and y direction, to obtain a projection width of motion blur smear of the image block in the x direction and in the y direction. For any image patch, the motion blur trailing shadow of the image patch is... x Projected width along the axis and in y Calculate the motion blur direction angle based on the projection width along the axis. For any image patch, the image patch is rotated clockwise according to the motion blur direction angle to obtain a rotated image, and the rotated image is then processed... x The motion fuzz scale is obtained by performing differential autocorrelation operation in the axial direction; For any image patch, a point spread function for the image patch is constructed based on the motion blur direction angle and the motion blur scale; Performing the rotation on the image x The axial differential autocorrelation operation yields the motion blur scale, specifically including: Performing the rotation on the image x The axial differential autocorrelation operation determines the distance between two negative peaks in the differential autocorrelation function of the rotated image; The absolute value of half the distance between the two negative peaks in the differential autocorrelation function of the rotated image is determined as the motion blur scale.

2. The method for removing complex motion blur from an image according to claim 1, characterized in that, The reconstructed image of the moving object is obtained by restoring and recombining all image patches according to the point spread function, specifically including: Based on the point spread function, Wiener filtering is used to perform deconvolution restoration on each image block to obtain the restored image of each image block; All the restored images are fused together to obtain the reconstructed image of the moving object.

3. The method for removing complex motion blur from an image according to claim 1, characterized in that, The formula for calculating the motion blur direction angle is: ； in, Indicates the direction angle of motion blur; Indicates motion blur in image blocks x The projected width along the axial direction; Indicates motion blur in image blocks y The projected width along the axial direction.

4. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the method for removing complex motion blur from an image as described in any one of claims 1-3.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method for removing complex motion blur from an image as described in any one of claims 1-3.

6. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method for removing complex motion blur from an image as described in any one of claims 1-3.

Images