An image deblurring processing method and device, electronic equipment and storage medium

Abstract

The application provides an image deblurring processing method and device, electronic equipment and storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring the region mask of each motion target in the blurred image and extracting motion feature information; determining the deformable offset of each motion target based on the motion feature information and combined with edge gradient information; determining the blur kernel corresponding to each motion target according to the deformable offset and obtaining the local restored image of each motion target through restoration processing; extracting the gradient feature of each local restored image and generating a clear image through fusion processing. The method aims to solve the technical problem that the existing technology lacks accurate perception of independent motion targets and heterogeneous blur modeling when processing multi-target motion scenes, resulting in edge distortion and artifacts in image restoration. The method can realize automatic segmentation and differentiated modeling of multi-target motion blur scenes and effectively eliminate non-uniform motion blur in complex backgrounds.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an image deblurring method, apparatus, electronic device, and storage medium. Background Technology

[0002] In digital imaging, when moving objects exist in the scene during camera exposure, the captured image often suffers from motion blur due to object displacement, severely affecting image sharpness and the accuracy of subsequent visual analysis tasks (such as target recognition, tracking, or measurement). Traditional motion-blurred image restoration techniques are mostly based on the assumption of a single global blur kernel, assuming that the entire image is blurred by motion in the same direction and at the same speed. However, in complex real-world scenarios (such as traffic monitoring, sporting events, and drone aerial photography), the image often contains multiple targets moving independently at different speeds and directions, resulting in heterogeneous and inconsistent blur patterns in different areas of the image. In such cases, using a single blur kernel model for restoration not only fails to effectively restore the details of each moving target but also easily introduces artifacts, edge distortion, or structural confusion.

[0003] Existing methods mainly focus on modeling single moving targets or overall blurred scenes. Some studies attempt to improve performance through local blur kernel estimation or spatially varying point spread function (PSF), but lack explicit perception and decoupling mechanisms for multiple moving targets; other solutions rely on special hardware such as event cameras to obtain high temporal resolution information, which is difficult to promote and apply on ordinary RGB imaging devices.

[0004] Therefore, how to automatically identify and separate multiple moving target regions using only a regular single-frame blurred image, and independently model the blur characteristics of each target to achieve accurate restoration, has become a technical bottleneck that urgently needs to be overcome in the current field of image restoration. Summary of the Invention

[0005] The purpose of this invention is to provide an image deblurring processing method, apparatus, electronic device, and storage medium, which aims to solve the technical problem that existing technologies, when processing multi-target motion scenes, suffer from edge distortion and artifacts in image restoration due to the lack of accurate perception of independent moving targets and heterogeneous blur modeling. The invention can achieve automatic segmentation and differentiated modeling of multi-target motion blur scenes, and effectively eliminate non-consistent motion blur in complex backgrounds through the refined application of spatial variation blur kernels.

[0006] In a first aspect, the present invention provides an image deblurring method, comprising the following steps: S1. Acquire a blurred image and extract the region mask of each moving target in the blurred image; S2. Based on the region mask, and combined with a pre-trained multi-target local fuzzy tensor model, extract the motion feature information of the corresponding moving target; S3. Based on the motion feature information and combined with the edge gradient information of the moving target, determine the deformable offset of each moving target in the spatial dimension; S4. Based on the deformable offset, determine the first local spatial variation blur kernel corresponding to each of the moving targets, and use the first local spatial variation blur kernel to perform restoration processing on the image region where each of the moving targets is located, to obtain the local restored image of each of the moving targets; S5. Extract the gradient features of each of the local restored images, and based on the gradient features, perform fusion processing on each of the local restored images to generate a clear image.

[0007] The image deblurring method provided by this invention can effectively avoid the incorrect sharpening of static backgrounds and the mutual interference of blur features between different moving targets when facing complex scenes with multiple moving targets and multiple directions of motion. This significantly improves the accuracy of image restoration and the fidelity of the overall image in scenes with multiple moving targets.

[0008] In a second aspect, the present invention provides an image deblurring processing apparatus, comprising: The acquisition module is used to acquire a blurred image and extract the region mask of each moving target in the blurred image; The extraction module is used to extract motion feature information of the corresponding moving target based on the region mask and in combination with a pre-trained multi-target local fuzzy tensor model; The determination module is used to determine the deformable offset of each of the moving targets in the spatial dimension based on the motion feature information and the edge gradient information of the moving targets; The restoration module is used to determine the first local spatial change blur kernel corresponding to each of the moving targets based on the deformable offset, and to use the first local spatial change blur kernel to perform restoration processing on the image region where each of the moving targets is located, so as to obtain the local restored image of each of the moving targets. The generation module is used to extract the gradient features of each of the local restored images, and based on the gradient features, to generate a clear image by fusing the local restored images.

[0009] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing computer-readable instructions, which, when executed by the processor, perform the steps of the image deblurring processing method provided in the first aspect above.

[0010] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the image deblurring method provided in the first aspect above.

[0011] As can be seen from the above, the image deblurring method provided by this invention, by introducing a multi-target local fuzzy tensor model, achieves independent perception and decoupling of heterogeneous fuzzy characteristics of each moving target region in complex scenes, thus overcoming the limitations of traditional single global fuzzy kernels. Through dynamic feature extraction guided by region masks and deformable offset learning, this application can adaptively restore the local features of different moving targets. While accurately restoring target details, it effectively avoids edge distortion and structural confusion common in traditional methods, significantly improving image clarity and restoration quality in multi-target fuzzy scenes.

[0012] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0013] Figure 1 This is a flowchart of an image deblurring method provided in an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of an image deblurring processing device provided in an embodiment of the present invention.

[0015] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0016] Label Explanation: 100. Acquisition module; 200. Extraction module; 300. Determination module; 400. Restoration module; 500. Generation module; 13. Electronic device; 1301. Processor; 1302. Memory; 1303. Communication bus. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0018] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0019] In digital imaging, when moving objects exist in the scene during the camera's exposure time, the captured image often suffers from motion blur due to the object's displacement, severely affecting image sharpness and the accuracy of subsequent visual analysis tasks. Traditional motion-blurred image restoration techniques are mostly based on the assumption of a single global blur kernel, which assumes that the entire image is blurred by motion in the same direction and at the same speed. However, in complex scenes, multiple moving targets move independently at different speeds and directions, resulting in heterogeneous and inconsistent blur patterns in different areas of the image. Traditional single global blur kernel models cannot effectively restore the details of each moving target and are prone to introducing artifacts.

[0020] For example, in traffic monitoring scenarios, the image contains multiple vehicles and pedestrians traveling at different speeds. Due to the differences in the motion vectors of each target, the degree and direction of blurring in different regions of the image exhibit a spatially non-uniform distribution. In this case, if a global blur kernel is used for processing, the local motion characteristics of each target cannot be decoupled, resulting in structural confusion and artifacts at the target edges in the restored image, which seriously interferes with the execution of subsequent target recognition and trajectory tracking tasks.

[0021] If the above problems are not solved, the image restoration process will be unable to accurately model heterogeneous fuzzy patterns, resulting in defects in detail restoration and edge fidelity of the restored image. This limits the application capability of the visual analysis system in complex dynamic environments, making it difficult for the image processing workflow to meet the needs of high-precision visual tasks.

[0022] For reference, see the appendix. Figure 1 This invention proposes an image deblurring method, comprising the following steps: S1. Acquire the blurred image and extract the region mask of each moving target in the blurred image; S2. Based on the region mask, and combined with a pre-trained multi-target local fuzzy tensor model, extract the motion feature information of the corresponding moving target; S3. Based on motion feature information and combined with the edge gradient information of the moving target, determine the deformable offset of each moving target in the spatial dimension; S4. Based on the deformable offset, determine the first local spatial variation blur kernel corresponding to each moving target, and use the first local spatial variation blur kernel to perform restoration processing on the image region where each moving target is located, so as to obtain the local restored image of each moving target. S5. Extract the gradient features of each local restored image, and based on the gradient features, generate a clear image by fusing the local restored images.

[0023] For ease of understanding, the following explains some key terms in this embodiment: Blurred images: These are images created during digital imaging when moving objects in the scene within the camera's exposure time cause motion blur. This affects image sharpness and blurs details.

[0024] Region mask: A binary image or matrix used to identify specific moving target regions in a blurred image. Each region mask typically corresponds to an individual moving target, and its pixel value indicates whether the pixel belongs to the moving target. For example, a pixel value of 1 indicates that it belongs to the moving target, and a pixel value of 0 indicates that it does not belong to the moving target.

[0025] Multi-objective local blur tensor model: This refers to a pre-trained mathematical model used to describe and analyze the local blurring characteristics caused by multiple moving targets in a blurred image. This model can capture heterogeneous blur patterns generated by different moving targets in different regions of the image, providing a theoretical basis for subsequent deblurring processing.

[0026] Motion feature information: refers to the feature data extracted from a blurred image that is related to the motion state of a moving target. This information can include region masks, feature maps, and motion response features, which are used to characterize the degree of blurring, direction, and speed of the moving target.

[0027] Edge gradient information refers to the rate of change of pixel intensity in an image, typically used to represent image edges and textures. In deblurring, edge gradient information can help determine the boundaries and shape of moving targets, thus allowing for more accurate estimation of their motion trajectories and blur kernels.

[0028] Deformable offsets: These refer to the spatial offsets of pixels relative to their original positions. These offsets are variable, adapting to the non-rigid deformations of the moving target during blurring, thus enabling more accurate modeling of local blur.

[0029] The first local spatial variation blur kernel refers to a blur kernel that varies spatially and is used to describe the blurring effect of a specific moving target in a local area of ​​an image. Unlike the traditional single global blur kernel, this blur kernel can adaptively adjust according to the different positions and motion states of the moving target, achieving refined local deblurring.

[0030] Locally restored images: These are images obtained by deblurring specific moving target regions within a blurred image. Each locally restored image contains the sharpened result of only one moving target, laying the foundation for subsequent image fusion.

[0031] Gradient features refer to the rate of change of pixel intensity in a locally restored image, used to characterize the edges, texture, and details of the image. In image fusion, gradient features serve as an important basis for fusion, ensuring that the fused image maintains consistency and sharpness at edges and details.

[0032] A sharp image refers to an image that has been deblurred, restoring the original scene details and sharpness. This image eliminates the effects of motion blur and has high visual quality and structural fidelity.

[0033] This application proposes an image deblurring method to address the problem that in complex scenes, multiple moving targets move independently at different speeds and directions, resulting in heterogeneous and inconsistent blurring patterns in different regions of the image. Traditional single global blur kernel models cannot effectively restore the details of each moving target and are prone to introducing artifacts.

[0034] This method first acquires a blurred image and extracts the region masks for each moving target within it. In practical applications, acquiring a blurred image can be achieved in various ways, such as directly reading image data from a camera sensor or loading an existing blurred image file from a storage device. The purpose of extracting the region masks is to decouple the complex, globally blurred scene into multiple independent local regions, laying the foundation for subsequent targeted processing. One implementation approach is to use a deep learning-based instance segmentation network, which, after training, can identify different moving targets in the image and generate corresponding binary masks. Another approach is to use traditional image processing techniques, such as combining motion detection algorithms with morphological operations to roughly estimate the regions of moving targets.

[0035] Next, based on the region mask and combined with a pre-trained multi-target local fuzzy tensor model, motion feature information of the corresponding moving targets is extracted. The multi-target local fuzzy tensor model breaks through the traditional single-fuzzy-kernel assumption, explicitly modeling heterogeneous fuzzy patterns caused by different moving targets, making the image degradation process more consistent with the real physical imaging mechanism, and providing theoretical support for high-precision restoration. In practice, a fuzzy invariant feature extractor can be used in conjunction with the multi-target local fuzzy tensor model to extract feature maps from the blurred image. Subsequently, based on the region mask, the motion response features of each moving target are obtained by performing element-wise multiplication on the feature maps. This motion feature information, including the region mask, feature maps, and motion response features, can focus on the fuzzy characteristics of specific moving targets, avoiding interference from background or other target features.

[0036] Subsequently, based on motion feature information and combined with the edge gradient information of the moving targets, the deformable offset of each moving target in the spatial dimension is determined. In specific implementation, a learnable mapping can be used, taking motion feature information and edge gradient information as input and outputting the deformable offset. Edge gradient information can be obtained by calculating the gradient of a region mask. By jointly driving the blur kernel deformation with edge gradients and features, the estimated local point spread function can be stretched along the direction of the real motion trajectory, significantly improving the modeling capability for non-uniform and nonlinear motion blur.

[0037] Then, based on the deformable offset, a first local spatial variation blur kernel is determined for each moving target. Using this kernel, the image region containing each moving target is restored, resulting in a locally restored image of each target. When determining the first local spatial variation blur kernel, another learnable mapping can be used, combining the feature map and the deformable offset, to obtain the blur kernel through a normalized exponential function. This design allows the blur kernel to stretch along the motion direction, better conforming to the physical imaging process. During the restoration process, a regularized iterative reconstruction function can be used, taking the blurred image and the first local spatial variation blur kernel as input and outputting a locally restored image. The spatial variation blur kernel dynamically adapts to the blur level of different regions, achieving refined restoration of local regions of each moving target.

[0038] Finally, gradient features of each local restored image are extracted, and based on these gradient features, a clear image is generated by fusing the local restored images. To avoid artifacts from multi-region stitching, a gradient-guided Poisson fusion strategy can be adopted. This strategy obtains a seamless fusion result by defining a target gradient field and solving the Poisson equation. The fusion weights can be determined based on the gradient features of each local restored image, emphasizing high-gradient regions as the dominant fusion area. Using gradient features as the fusion basis effectively preserves edge details in each local restored image, eliminates boundary discontinuities that may be caused by local processing through fusion, and ultimately synthesizes a high-quality, clear image. This effectively eliminates brightness jumps, color mismatches, and structural breaks at multi-region boundaries, achieving seamless visual stitching, significantly improving the overall naturalness and structural integrity of the output image, and overcoming the "jigsaw puzzle effect" common in existing block-based deblurring methods.

[0039] The following example will provide a more detailed explanation of the above technical solution: Suppose a traffic monitoring scenario where a surveillance camera captures a blurry image. This image contains multiple moving targets, such as a fast-moving car, a jogging pedestrian, and a slowly moving bicycle. Because their speeds and directions of motion differ, the blur patterns in different areas of the image also vary. Traditional deblurring methods typically use a single global blur kernel to process the entire image. This results in poor blurring of the car, potential introduction of artifacts for the pedestrian, and over-sharpening of the bicycle, failing to effectively restore the details of each moving target.

[0040] To address this issue, this application proposes an image deblurring method. First, the system acquires the blurred image. Then, the blurred image is input into a pre-defined motion-aware instance segmentation network. This network integrates semantic features and motion saliency cues, enabling it to accurately identify cars, pedestrians, and bicycles in the image and generate an independent region mask for each moving target. For example, a mask is generated for a car. Generate a mask for pedestrians. Generate a mask for the bicycle. These masks are binary, precisely defining the spatial region of each moving target in the image. In this way, complex globally blurred scenes are effectively decoupled into multiple independent local regions, laying the foundation for subsequent targeted processing.

[0041] Next, based on these region masks and combined with a pre-trained multi-object local fuzzy tensor model, the system extracts motion feature information of the corresponding moving targets. Specifically, it utilizes a pre-defined fuzzy invariant feature extractor in the motion-aware instance segmentation network, combined with the multi-object local fuzzy tensor model, to extract feature maps from the blurred image. Then, by masking each region (e.g.) ) and feature map Element-wise multiplication is performed, followed by processing with a lightweight motion response head and a sigmoid function to obtain the motion response characteristics of the corresponding moving target (e.g., a car). These motion feature information, including region masks, feature maps, and motion response features, can focus on the blurring characteristics of a specific moving target, avoiding interference from background or other target features.

[0042] Subsequently, based on the extracted motion feature information and combined with the edge gradient information of each moving target, the deformable offset of each moving target in the spatial dimension is determined. For example, for a car, the system calculates its region mask. Edge gradient Then, using a first learnable mapping , feature map Local features corresponding to the car region and edge gradient As input, the deformable offset of the car at pixel coordinates (x, y) is calculated. By jointly driving the fuzzy kernel deformation with edge gradients and features, the estimated local point spread function can be stretched along the direction of the real motion trajectory, significantly improving the modeling capability for non-uniform and nonlinear motion fuzz.

[0043] Next, based on these deformable offsets, a first local spatial variation blur kernel is determined for each moving target. These blur kernels are then used to restore the image regions containing each moving target, resulting in locally restored images of each target. For example, for a car, the system utilizes a second learnable mapping. Combined with feature maps Local features corresponding to the car region and the deformable offset of the car The L2 norm is used to calculate the first local spatial variation blur kernel of the car at pixel coordinates (x, y) using the normalized exponential function Softmax. This design stretches the blur kernel along the direction of motion, better conforming to the physical imaging process. Subsequently, a regularized iterative reconstruction function is used. The original blurred image The first local spatial variation fuzzy kernel field corresponding to the automobile As input, the image region containing the car is processed to obtain a locally restored image of the car. Similarly, pedestrians and bicycles were treated in a similar way, and the results were obtained respectively. and By using a spatially varying fuzzy kernel, the system can dynamically adapt to the degree of fuzziness in different regions, enabling refined restoration of local areas of each moving target.

[0044] Finally, the system extracts the gradient features of each locally restored image and, based on these gradient features, fuses the locally restored images to generate the final clear image. Specifically, the system calculates... , and gradient features , and To avoid artifacts from multi-region stitching, the system employs a gradient-guided Poisson fusion strategy. This strategy calculates the fusion weight for each pixel based on the gradient features of each locally restored image. The high-gradient regions are given higher weights to emphasize their dominant role in the fusion process. Finally, by solving the Poisson equation, these locally restored images are seamlessly fused together to generate a single, clear, detailed, and artifact-free image. By utilizing gradient features as the fusion basis, edge details in each local restored image can be effectively preserved. The fusion process eliminates boundary discontinuities that may be caused by local processing, ultimately synthesizing a high-quality, clear image. It effectively eliminates brightness jumps, color mismatches, and structural breaks at the boundaries of multiple regions, achieving seamless visual stitching, significantly improving the overall naturalness and structural integrity of the output image, and overcoming the "jigsaw puzzle effect" common in existing block-based deblurring methods.

[0045] The aforementioned technical solution achieves accurate modeling and restoration of heterogeneous blur patterns by decomposing a globally blurred image into independent processing steps for multiple locally moving targets. Compared to traditional single global blur kernel models, the core contribution of this application lies in achieving explicit decoupling of multi-target blur through region masking and realizing local adaptive modeling of heterogeneous blur by combining spatially variable blur kernels with deformable offsets. This processing method overcomes the limitations of traditional single-kernel models in handling multi-moving target scenes. Through local-to-global restoration and fusion, it significantly improves the restoration quality and structural fidelity of images in complex motion scenes. For example, in the aforementioned traffic monitoring scenario, traditional methods may not be able to simultaneously clarify fast-moving cars and slow-moving bicycles, and may even produce obvious stitching marks between them. However, the method in this application, by assigning an independent deblurring process to each target and employing gradient-guided Poisson fusion, can ensure that cars, pedestrians, and bicycles are restored with high quality and achieve smooth transitions at their boundaries, ultimately generating a visually seamless and detailed clear image. This refined and adaptive deblurring strategy greatly improves the usability and analytical value of images in complex motion scenes.

[0046] In some embodiments, a further step is included between step S1 and step S2: S6. Assign an independent deblurring processing branch to each moving target according to the number of region masks, and execute steps S3 and S4 in parallel for each deblurring processing branch; Step S6 is followed by the following steps: S71. Based on motion saliency cues, obtain motion saliency indicators for each moving target; S72. When the motion saliency index is lower than the preset threshold, the corresponding deblurring branch is cropped, and the original blurring features of the image region where the corresponding moving target is located are preserved or replaced by a lightweight global deblurring algorithm.

[0047] Among them, "independent deblurring branch" refers to each detected moving target (by region mask) (Identifier) ​​Create an independent computational path or module specifically responsible for deblurring the target. This independence ensures that the deblurring processes for different targets do not interfere with each other and can be optimized according to their respective characteristics. For example, a deblurring network or processing pipeline can be instantiated for each moving target, receiving the local image region of the target and the corresponding region mask as input. Alternatively, a shared deblurring model can be used, but through dynamic parameter tuning or attention mechanisms, it can exhibit independent computational behavior when processing different targets.

[0048] "Parallel execution" refers to the simultaneous computation of multiple independent deblurring branches, rather than sequential execution. This can be achieved through multithreading, multiprocessing, GPU parallel computing, or distributed computing. For example, when the system detects N moving targets, N deblurring tasks can be launched simultaneously, with each task processing one target, thus significantly reducing the overall processing time. Alternatively, at the hardware level, multiple processing cores or computing units can be used to allocate different deblurring branches to different cores for synchronous computation.

[0049] "Motion saliency cues" refer to information that reflects the importance of a moving target in an image or its impact on visual perception. These cues can include, but are not limited to, the target's size, speed, direction of motion, distance from the image center, semantic importance of the target in the scene (e.g., faces, vehicles), or degree of saliency. For example, they can be obtained by analyzing the pixel variation in the target region, the size of the bounding box, the target's position in the image, or by using a pre-trained model to classify the target and assign different saliency weights.

[0050] The "motion saliency index" is a quantitative value calculated based on motion saliency cues to measure the importance of each moving target. This index can be a single numerical value or a multi-dimensional vector. For example, it can be a weighted score that considers factors such as target area, degree of motion blur (e.g., blur kernel size or trajectory length), and the target's visual centrality in the image. Alternatively, it can be a target importance score predicted by a machine learning model.

[0051] A "preset threshold" refers to a pre-defined boundary value used to determine whether the motion saliency index is low enough during branch pruning. This threshold can be adjusted based on the needs of the actual application scenario, computational resource limitations, and the requirements for deblurring quality. For example, an empirical value can be determined experimentally, such that targets below this value are considered minor and can be simplified. Alternatively, the threshold can be a dynamically adjusted parameter, updated in real time based on the current system load or user preferences.

[0052] "Pruning" refers to stopping or simplifying the calculation of the corresponding deblurring branch when the motion saliency index falls below a preset threshold. This can include completely terminating the execution of the branch or replacing it with a less computationally intensive processing method. For example, the subsequent complex iterative reconstruction process in that branch can be interrupted, and the original blurred region can be output directly. Alternatively, certain computationally complex feature extraction or kernel estimation steps can be skipped.

[0053] "Preserving original blurred features" means that for the moving target region corresponding to the cropped deblurring branch, no deblurring processing is performed, and its original blurred image data is directly used as part of the output. For example, the pixel values ​​of the target region in the original blurred image can be directly copied into the final clear image.

[0054] "Lightweight global deblurring algorithms" refer to deblurring methods with low computational complexity and high processing speed. They are typically used to handle blurring of the entire image or a large region, but their deblurring effect may not be as good as more sophisticated local deblurring algorithms. For example, deblurring methods based on Fourier transform, simple deconvolution algorithms, or convolutional neural network models with fewer parameters can be used. When processing secondary targets, these algorithms can significantly save computational resources while maintaining a certain level of deblurring effect.

[0055] This application's solution significantly improves the flexibility and computational efficiency of multi-target deblurring by introducing parallel processing and dynamic resource scheduling mechanisms. In the aforementioned image deblurring method, a blurred image is first acquired, and the region masks of each moving target are extracted. Based on these region masks and a pre-trained multi-target local blur tensor model, the motion feature information of the corresponding moving targets is extracted. Furthermore, this application allocates independent deblurring processing branches to each moving target according to the number of region masks. These branches are designed to execute in parallel the subsequent steps of determining deformable offsets and determining the first local spatial change blur kernel and performing restoration processing. This parallel processing mechanism allows multiple moving targets to undergo deblurring calculations simultaneously, greatly shortening the overall processing time and avoiding the high latency problem caused by traditional serial processing.

[0056] To further optimize resource allocation, this application introduces a motion saliency evaluation mechanism after parallel processing. The system obtains motion saliency indices for each moving target based on motion saliency cues. These indices quantify the importance of each target in the image or its impact on visual quality. When the saliency index of a moving target falls below a preset threshold, the system prunes its corresponding deblurring branch. This means that the complex computation process of that branch will be terminated or simplified. For the pruned target region, the system can choose to retain its original blurred features (i.e., not perform deblurring) or use a lightweight global deblurring algorithm with lower computational cost for alternative processing. This strategy effectively concentrates computational resources on key areas with high visual saliency and significant blurring impact, ensuring the restoration quality of core targets while achieving a lightweight and efficient overall processing flow, solving the computational bottleneck in multi-target processing in complex scenes. In this way, the solution in this application can not only decouple the processing of multiple moving targets but also dynamically adjust computational resources according to the actual importance of the targets, thereby significantly improving processing efficiency and real-time performance while ensuring deblurring effects.

[0057] The following is a concrete example to illustrate this. Suppose a blurred image contains three moving targets: a person in the center of the image who is moving relatively fast (target 1), a vehicle at the edge of the image who is moving relatively slowly (target 2), and a small, blurry bird in the background (target 3).

[0058] First, the system acquires the blurred image and extracts the region masks for target 1, target 2, and target 3 using a motion-aware instance segmentation network. , , Next, based on these region masks and pre-trained multi-target local fuzzy tensor models, their respective motion feature information is extracted.

[0059] Subsequently, based on the number of masks for these three regions, the system assigns independent deblurring processing branches to target 1, target 2, and target 3 respectively. , , These three branches are configured to execute in parallel. For example, on a multi-core processor or GPU, , , They can simultaneously start calculating their respective deformable offsets and the first local spatial change blur kernel, and perform restoration processing on their respective image regions.

[0060] During or after parallel processing, the system obtains the motion saliency index for each target based on motion saliency cues. For example, target 1 (human) may obtain a high saliency index (e.g., 0.9) due to its central position, large size, and fast speed; target 2 (vehicle) may obtain a medium saliency index (e.g., 0.6); while target 3 (bird) may obtain a low saliency index (e.g., 0.2) due to its small size, edge position, and inconspicuous blur.

[0061] Assuming the preset threshold is set to 0.3, the system will detect that the motion saliency index (0.2) of target 3 is lower than the preset threshold. Therefore, the system will perform deblurring processing on the branch corresponding to target 3. Perform the trimming. This means... The complex deblurring calculations will be stopped. For the image region containing target 3, the system can choose to preserve its original blurred features, i.e., directly copy the blurred pixels of that region into the final image, or use a computationally insignificant, lightweight global deblurring algorithm (e.g., a simple mean filter or a small deconvolution kernel) to process it quickly. The deblurring branches for targets 1 and 2... and Then it will continue to execute the complete, high-precision deblurring process.

[0062] Through the above technical solution, this application effectively solves the problem of low computational efficiency caused by the traditional method of using a uniform computational resource allocation strategy for all targets when processing complex scenes containing multiple moving targets. By allocating an independent deblurring processing branch for each moving target and executing it in parallel, the processing speed is significantly improved and the overall deblurring time is shortened. At the same time, the introduction of a motion saliency evaluation mechanism can intelligently identify and distinguish moving targets of different importance. For targets with low motion saliency and limited impact on overall visual quality, their processing branches are pruned and replaced with alternative processing methods that preserve the original blurring features or use a lightweight global deblurring algorithm. This effectively avoids unnecessary waste of computational power and concentrates limited computational resources on key targets that have a greater impact on image quality. Thus, while ensuring the restoration quality of core targets, the overall processing flow is made lightweight and efficient, improving the system's real-time performance and resource utilization.

[0063] In some embodiments, step S1, specifically the steps of extracting the region mask of each moving target in the blurred image, include: S1. Input the blurred image into a preset motion-aware instance segmentation network, and use the motion-aware instance segmentation network to generate N region masks (e.g., binary masks, represented as...) by fusing semantic features and motion saliency cues. , ,in, Let i be the region mask for the i-th moving target. This represents the total number of region masks generated. The height of the blurred image, (where the width is the blurred image), and the weighted sum of the masks of each region in the spatial dimension is not greater than 1 (i.e., satisfies...). ).

[0064] Specifically, a blurred image refers to an original input image in which details are blurred due to movement of objects in the scene or camera shake during camera exposure. It is usually represented in the form of a three-dimensional matrix or tensor, for example, This indicates It is a highly ,width A three-dimensional matrix or tensor with three color channels (e.g., RGB). Motion-aware instance segmentation networks are specially designed deep learning models whose core function is to identify independent moving targets in an image and accurately delineate the pixel-level boundaries of each target. This network can not only distinguish different instances in an image but also perceive and utilize the motion information of the targets. As one possible implementation, this network can be extended based on existing instance segmentation architectures (e.g., Mask R-CNN or YOLACT) by introducing additional feature extraction branches or attention mechanisms to capture motion cues. Another implementation approach is to adopt a Transformer-based architecture, such as DETR or Swin Transformer, and combine it with an optical flow estimation module or an inter-frame difference analysis module to enhance its motion perception capabilities.

[0065] Fusion of semantic features and motion saliency cues refers to the process of effectively combining the semantic information and motion saliency information of an image within a motion-aware instance segmentation network. Semantic features provide a high-level understanding of the image content, such as identifying categories like "person" and "vehicle"; while motion saliency cues indicate which regions in the image are moving and the intensity of that movement. One fusion approach is to design the network as a multi-task learning model, simultaneously training it for semantic segmentation and motion saliency prediction tasks, and then concatenating or weighting the features extracted from both at intermediate or output layers. Another approach is to employ a cross-modal attention mechanism, allowing semantic features to dynamically adjust based on motion saliency cues, and vice versa, thereby generating more discriminative fused features. Generating N region masks means that after processing a blurred image, the motion-aware instance segmentation network outputs N independent pixel-level labels representing the spatial regions of N moving targets in the image. These masks provide precise geometric extents for subsequent local deblurring. One generation method involves the network directly outputting N independent binary images, each corresponding to a moving target. A pixel value of 1 indicates that the pixel belongs to the moving target, and a pixel value of 0 indicates that it does not. Another generation method involves the network outputting a multi-channel feature map, where each channel represents the probability distribution of a moving target. These probability maps are then converted into binary masks by setting a threshold. These region masks are, for example, binary masks represented as follows: , ,in, Let i be the region mask for the i-th moving target. This represents the total number of region masks generated. The height of the blurred image, The width of the blurred image. The region mask representing the i-th moving target It is a size of × A binary matrix, where The height of the blurred image, The width of the blurred image. Each element in the matrix. The value can only be 0 or 1, precisely indicating whether the pixel coordinates (x, y) belong to the i-th moving target.

[0066] The weighted sum of the regional masks in the spatial dimension is no greater than 1, i.e., it satisfies This is a key constraint designed to ensure that the region masks of different moving targets in an image are spatially mutually exclusive, meaning that any pixel can belong to at most one moving target. One way to implement this constraint is to include it as part of the loss function during the training of the motion-aware instance segmentation network, for example, by introducing a penalty term to suppress mask overlap. Another approach is to enforce this constraint through post-processing algorithms (such as non-maximum suppression or confidence-based pixel assignment strategies) after the network generates the initial region masks, ensuring that the final output mask set satisfies spatial mutual exclusivity.

[0067] When blurry image When input into a motion-aware instance segmentation network, the image extracts semantic features and motion saliency cues in parallel or sequentially. Semantic features help the network understand high-level content in the image, such as identifying the categories of objects contained within; while motion saliency cues provide low-level or high-level hints about which regions in the image are in motion. By effectively fusing these two different levels of features, the network can accurately distinguish between static backgrounds and dynamic moving targets in an image, even when the image is severely blurred, making traditional features difficult to discern. Subsequently, the network generates N binary masks based on the fused features. Each Each mask represents a precise spatial region in an image representing an independent moving target, with a pixel value of 1 indicating belonging to the target and 0 indicating not belonging. The generation process of these masks is designed to satisfy a key constraint: the weighted sum of the regional masks in the spatial dimension is no greater than 1. This means that any pixel in the image can be assigned to at most one moving target, thus forcibly guaranteeing spatial exclusivity between different moving target regions. This exclusivity avoids potential overlap and interference between different target regions in subsequent processing, providing each moving target with a clear and independent pixel-level effective range. Through this mechanism, the proposed solution overcomes the shortcomings of traditional methods in accurately segmenting moving subjects in fuzzy environments, achieving automated and refined extraction of multiple moving targets. These precise and mutually exclusive regional masks provide a reliable geometric basis and accurate localization information for subsequent steps S2 to S5, which involve independent motion feature extraction, deformable offset determination, generation of the first local spatial change fuzzy kernel, and local restoration processing for each moving target. Especially in the context of multi-target local fuzzy tensor models, precise regional masks are crucial. This allows the model to explicitly model the unique blur characteristics of each target, thereby significantly improving the accuracy and robustness of the overall image deblurring process.

[0068] In some embodiments, the specific steps in step S2 include: S21. Using the preset fuzzy invariant feature extractor in the motion-aware instance segmentation network, combined with the multi-target local fuzzy tensor model, extract the feature map of the fuzzy image; Specifically, feature maps are extracted using the following formula: ; The multi-objective local fuzzy tensor model is represented as: ; in, For feature maps, Represented as the coding backbone, For blurred images ( This indicates It is a highly ,width (and a 3D matrix or tensor with 3 color channels (e.g., RGB)). pixel coordinates in a blurred image The local blurred image at that location, pixel coordinates The x-axis coordinate value at that location. pixel coordinates The y-coordinate value at that location. Represented as pixel coordinates The corresponding location is about and Local convolution (i.e., each pixel has a different kernel). For the i-th moving target in the region The first local spatial variation fuzzy kernel field defined above, Let be the spatial region of the i-th moving target in the blurred image. Represented as The set of all pixels with non-zero values. Let i be the region mask for the i-th moving target. For the original clear image (it should be noted that defining a technical solution usually requires the use of a mathematical model to describe the internal physical relationships of the system. The image appearing here...) This is to rigorously define the spatial variation fuzzy kernel. The mathematical mechanism does not imply that a clear image needs to be obtained in advance when executing the method. For the i-th moving target at pixel coordinates Local region mask at (it should be noted that, That is to say, all pixel positions When combined, they form a complete system. Written as This is to accurately express the weight of each coordinate point in pixel-wise mathematical operations. pixel coordinates in the original sharp image A clear image of the local area. pixel coordinates Additive noise at the location; S22. Based on the region mask, the motion response features of each moving target are obtained by performing element-wise multiplication on the feature map; the motion feature information includes the region mask, the feature map, and the motion response features. Specifically, the motion response characteristics are calculated using the following formula: ; in, Let i be the motion response characteristics of the i-th moving target. For the Sigmoid function, It is a lightweight motion response head. Represented as and Perform element-wise multiplication.

[0069] To enhance the robustness of target recognition under blur, this application introduces a blur-invariant feature extractor. This extractor is a component designed to extract robust features from an image that are minimally affected by motion blur. Its role is to provide a stable representation of image content regardless of the degree of blur, which is crucial for accurately identifying and representing moving targets. The extractor can be a deep learning-based convolutional neural network (CNN) module that learns blur-invariant features during training through data augmentation (e.g., introducing different degrees of blur) or a specific loss function (e.g., perceptual loss); or it can be a hybrid model combining traditional image processing techniques (e.g., wavelet transform, Fourier transform, etc.) with machine learning methods, where traditional methods are used to extract features insensitive to frequency domain changes, and the machine learning part further refines these features. This application also proposes a multi-objective local blur tensor model, a mathematical model used to describe how blurred images are formed when multiple objects move independently in a scene. It explicitly considers the spatial variability of blur by associating a unique local blur kernel with each moving object, thereby more accurately representing complex blurred scenes in the real world. This model can be implemented as a parameterized function, whose parameters (such as the shape, size, and trajectory of the local blur kernel) can be estimated from the blurred image using an optimization algorithm; alternatively, the model can be a differentiable layer in a deep learning network, whose internal parameters are learned and optimized through backpropagation during network training, thereby achieving end-to-end modeling of complex blurring processes. (Code backbone) The encoder is the core part of a neural network, responsible for downsampling the input image and extracting hierarchical features. It typically consists of multiple convolutional layers, pooling layers, and activation functions, aiming to capture low-level visual cues and high-level semantic information. It can use classic deep convolutional network architectures such as ResNet, VGG, and EfficientNet as its encoding backbone; or it can use the encoder part of U-Net or Hourglass networks. Element-wise multiplication is an operation where corresponding elements of two matrices or tensors are multiplied. In this context, it is used to pass through region masks. For feature maps Spatial gating or weighting effectively isolates features belonging to a specific moving target while suppressing features from other regions. In software implementation, element-wise multiplication functions provided by scientific computing libraries such as NumPy, TensorFlow, or PyTorch can be used for direct computation. In hardware implementation, a dedicated multiplier array can be designed to perform parallel multiplication operations on corresponding pixel values ​​of the feature map and mask to improve processing efficiency. Lightweight motion response head. It is a small, efficient neural network module connected to the main feature extractor. Its main function is to process the masked features. This generates concise and discriminative motion feature representations for each individual target. "Lightweight" means it has fewer parameters and computational cost compared to the main backbone. It can consist of one or two convolutional layers (e.g., 1x1 or 3x3 convolutions) followed by an activation function to map features to the motion response space; or it can be a simple network containing global average pooling layers and fully connected layers to aggregate global motion information from local features. The Sigmoid function is used. The sigmoid function is a commonly used non-linear activation function that maps any real-valued input to a value between 0 and 1, making it suitable for generating probabilistic outputs or for gating mechanisms. In this context, it can normalize motion response features, highlighting salient motion patterns. In deep learning frameworks, the built-in sigmoid function can be directly called; in custom implementations, it can be programmed based on its mathematical definition.

[0070] This application's scheme achieves explicit decoupling and feature extraction of multiple moving targets in blurred images by introducing a fuzzy invariant feature extractor and a multi-target local fuzzy tensor model. First, the blurred image is encoded using the fuzzy invariant feature extractor to generate a feature map F containing global semantic information. This process is achieved by encoding the backbone. This ensures the robustness of feature extraction. Subsequently, a multi-objective local fuzzy tensor model is introduced to mathematically model the blurred image. Represented as the local fuzzy kernel of each moving target With clear images The weighted sum of convolutions clarifies the spatial independence of different moving targets, providing theoretical support for subsequent local reconstruction. In the feature extraction stage, region masks are used... With feature map Element-wise multiplication effectively filters out background interference and focuses on local regions of specific moving targets, thereby obtaining targeted motion response characteristics. Utilizing a lightweight motion response head The result of the multiplication is processed and passed through the Sigmoid function. Normalization further enhances the feature representation of moving targets, enabling subsequent steps to perform differentiated deblurring based on these precise motion response features for different moving targets. This precise and decoupled motion feature information (including region masks) Feature map and motion response characteristics This provides high-quality input for subsequently determining the deformable offsets of each moving target in the spatial dimension. Since the feature information has been refined for each target, the motion trajectory and deformation of each target can be estimated more accurately, thus generating a more precise first local spatial variation blur kernel in subsequent steps. Finally, using these highly accurate blur kernels to restore the image regions where each moving target is located yields a more faithful and detailed locally restored image, significantly improving the quality of the final clear image.

[0071] As a specific implementation method, a scenario can be envisioned in which a blurred image... It is a street scene image containing multiple moving objects, such as a car and a pedestrian. First, a motion-aware instance segmentation network (as described in step S1) generates region masks for the car and the pedestrian. and Next, a fuzzy invariant feature extractor, such as a pre-trained ResNet-50 network, encodes the backbone. Blur image Processed into a multi-channel feature map The feature map It incorporates global semantic information of the image and exhibits a certain degree of robustness to blurring. The multi-object local blur tensor model is implicitly learned during network training to understand how different moving objects cause local blurring. Subsequently, in order to extract the motion response features of the car... , feature map Area mask of the car Perform element-wise multiplication. This operation effectively transforms the feature map... Information about the automotive region in the Central African region is set to zero, thus focusing on the features of the vehicle itself. This masked feature map is then input into a lightweight motion response head. In this context, the response header can be a simple network consisting of a 3x3 convolutional layer and a ReLU activation function, used to further refine the motion-related features of the vehicle. Finally, the Sigmoid function is used... right The output is normalized to obtain the final vehicle motion response characteristics. A similar approach is used for pedestrians, by combining the feature map F with the pedestrian region mask. Perform element-wise multiplication and use a lightweight motion response head. and the Sigmoid function Generate pedestrian motion response features These extracted region masks Feature map and motion response characteristics Together, they constitute the motion feature information used for subsequent deblurring processing.

[0072] By employing the aforementioned technical solution, this application effectively addresses the problem that when processing blurred images with overlapping or significantly different motion characteristics of multiple moving targets in complex scenes, relying solely on global feature extraction is insufficient to accurately capture the local motion responses of each target. This leads to incomplete decoupling of features for different moving targets during subsequent restoration, thus affecting the quality of the locally restored image. This application introduces a fuzz-invariant feature extractor and a multi-target local fuzzy tensor model to extract a robust global semantic feature map from the blurred image. Based on this, element-wise multiplication of the feature map using precise region masks, combined with a lightweight motion response head and a sigmoid function, generates highly specific and decoupled motion response features for each moving target. This refined motion feature information provides a more accurate and reliable basis for subsequently determining the deformable offset of each moving target and the first local spatial variation fuzzy kernel, thereby enabling higher fidelity and richer detail restoration when restoring the image regions where each moving target is located. This significantly improves the quality of the final clear image, effectively avoiding restoration artifacts and detail loss caused by feature confusion in multi-target scenes, thereby improving the overall performance of image deblurring.

[0073] In some embodiments, the specific steps in step S3 include: S31. Calculate the deformable offset according to the following formula: ; in, For the i-th moving target at pixel coordinates Deformable offset at that location. For the first learnable mapping, pixel coordinates The corresponding local feature map is obtained from the feature map. For the i-th moving target at pixel coordinates The edge gradient at point i is determined by the pixel coordinates of the i-th moving target. The local area mask is obtained.

[0074] Deformable offset This represents the displacement vector between the actual and observed positions of the i-th moving target at pixel coordinates (x, y) due to motion blur. Its function is to quantify the local deformation of the moving target on the image plane and is a key parameter for constructing the spatial variation blur kernel. This offset can be a two-dimensional vector, with components representing the offset along the x and y axes, or it can be a higher-dimensional representation, such as including offset information in the time dimension, to accommodate more complex motion patterns. First Learnable Mapping A local feature map is a trainable function or network module that maps input feature information (local feature maps and edge gradients) to deformable offsets. It learns complex nonlinear relationships within the data, fusing multi-source information and transforming it into accurate motion displacement estimates. This mapping can be a multilayer perceptron, a convolutional neural network module (e.g., containing multiple convolutional layers, activation functions, and fully connected layers), or an attention-based network structure. This is a feature representation extracted from a blurred image, possessing local semantics and blur information at pixel coordinates (x, y). Its function is to provide deep information about image content, texture, color, and degree of blur, forming the basis for understanding the state of local regions in an image. This feature map can be a local slice at (x, y) of a depth feature map extracted from a blurred image by a pre-defined blur-invariant feature extractor, or it can be a feature representation after dimensionality reduction or upsampling. Edge gradient This represents the local region mask of the i-th moving target at pixel coordinates (x, y). The gradient information reflects the intensity and direction of the moving target's boundary. Its function is to provide contour information and boundary location of the moving target, helping the model accurately identify the actual edges of the moving target, especially when edges become blurred in a blurred image. This edge gradient can be obtained by masking a local region. Edge detection filters such as the Sobel operator, Prewitt operator, or Canny operator can be used to calculate the edge, or it can be extracted directly from the mask through the gradient calculation layer in a deep learning model.

[0075] The operating mechanism of this application is as follows: First, for the input blurred image, a preset motion-aware instance segmentation network is used to generate region masks for each moving target. Subsequently, a fuzzy invariant feature extractor is used to extract feature maps from the blurred image. The feature map It contains deep semantic and ambiguous information about the image. This is used to determine the deformable offset of each moving target. At that time, this scheme no longer relies solely on local feature maps. Instead, it uses local feature maps. Edge gradient of the i-th moving target at pixel coordinates (x, y) The images are then stitched together. This stitching operation combines the semantic content and ambiguity of the images with the precise boundary information of the moving targets. Next, the stitched joint features are input into the first learnable mapping. In the middle. The first learnable mapping. Through its internal nonlinear transformation capabilities, it learns how to infer the precise spatial displacement of a moving target at each pixel from these fused features. Edge gradient As an important motion boundary prior, it can guide the first learnable mapping. More accurately identify the actual contours and deformation directions of moving targets, especially when blurring causes indistinct edges. In this way, the first learnable mapping... It can output more accurate deformable offset. This offset reflects not only the intensity of the motion but also its direction and local deformation. This method, which combines local feature maps and edge gradients, makes the deformable offset... The estimation is more robust and accurate. In the subsequent step S4, these accurate offsets will be directly used to determine the first local spatial variation fuzzy kernel corresponding to each moving target. This allows the blur kernel to more accurately simulate the real motion trajectory and deformation of moving targets, significantly improving the modeling capability for non-uniform and nonlinear motion blur. Finally, by using these precise blur kernels to restore the image regions where each moving target is located, high-quality local restored images can be obtained, laying a solid foundation for generating the final clear image.

[0076] Through the above technical solution, this application, in determining the deformable offset of each moving target in the spatial dimension, no longer relies solely on local feature maps, but creatively introduces the edge gradient information of the moving targets. This approach integrates local feature maps... With edge gradient Perform fusion, and through the first learnable mapping The processing method enables the model to more comprehensively perceive the semantic information, fuzzy characteristics, and precise boundary contours of moving targets. Specifically, edge gradients... As an important prior for motion boundaries, it can effectively compensate for the shortcomings of local feature maps in perceiving the boundaries of moving targets under complex backgrounds and severe blurring conditions. In blurred images, the edges of moving targets often become indistinct, and a single feature map may struggle to accurately distinguish the boundary between the target and the background. By introducing edge gradients, the model can obtain a clear indication of the target contour, thereby improving the computation of deformable offsets. This allows for more accurate capture of the local deformation and displacement of moving targets in space. This refined offset estimation provides a basis for constructing the first local spatial variation fuzzy kernel in subsequent step S4. It provides a more accurate geometric offset reference. The blur kernel can more closely match the true motion trajectory and deformation of the moving target, significantly improving the modeling ability for non-uniform and nonlinear motion blur. Therefore, when restoring the image regions where each moving target is located, it can effectively avoid artifacts, edge distortion, or structural confusion caused by inaccurate offset estimation, ensuring the integrity of the restored image in terms of structure and detail. Especially in the edge regions of moving targets, the restoration effect is significantly improved. This enables the method of this application to achieve higher-precision image deblurring results when dealing with complex scenes containing multiple independently moving targets with different speeds and directions.

[0077] In some embodiments, the specific steps in step S4 include: S41. Determine the first local spatial variation fuzzy kernel according to the following formula: ; in, For the i-th moving target at pixel coordinates The first local spatial variation fuzzy kernel at the location, For normalized exponential functions, For the second learnable mapping, The preset exercise intensity adjustment factor, Indicated as to Find the L2 norm; S42. Obtain the locally restored image according to the following formula: ; ; in, Let be the local restored image of the i-th moving target. For regularized iterative reconstruction functions, Represented as the pixel coordinates of the i-th moving target Located in the region The first local spatial variation fuzzy kernel field defined above.

[0078] First Local Spatial Change Fuzzy Kernel This refers to the kernel function estimated for the i-th moving target in an image at pixel coordinates (x, y), used to describe the blur characteristics of that pixel. This kernel function is spatially variable, meaning that pixels at different locations in the image may have different blur kernels, thus enabling accurate capture of non-uniform motion blur in complex scenes. Its function is to provide an accurate blur model for subsequent image restoration, allowing the restoration process to perform deconvolution or deblurring operations on the local blur characteristics of each pixel. The Softmax normalization exponential function is a commonly used normalization function that transforms a real-number vector into a probability distribution such that each element in the vector is in the interval (0, 1), and the sum of all elements is 1. In this application, the Softmax function is used to normalize the original estimate of the blur kernel, ensuring that the generated first locally spatially variable blur kernel... The sum of its elements is 1, thus satisfying the characteristics of a probability distribution, which is more in line with the physical definition of a fuzzy kernel. The second learnable mapping... It is a parameterized function or network module whose function is to process the input local feature map. Mapping to an intermediate representation, which is then combined with motion intensity information, is used to generate the first local spatial variation fuzzy kernel. This mapping is typically implemented through neural network layers, whose parameters are learned during model training via backpropagation and optimization algorithms. It extracts deep semantic information related to the shape and intensity of the blur kernel from the visual features of an image. (Local feature map) From a blurry image The extracted features are the pixel coordinates (x, y) corresponding to the image. These features are typically extracted by the encoding backbone of a deep neural network and contain the image's texture, edges, color, and potential motion information. As a second learnable mapping The input provides rich contextual information for the generation of the fuzzy kernel. The preset motion intensity adjustment factor... It is a hyperparameter used to balance the second learnable mapping. Output and deformable offset The relative importance of the L2 norm in the fuzzy kernel generation process. By adjusting... The value of controls the degree to which the motion offset affects the shape and intensity of the blur kernel. Find the L2 norm Indicates the calculation of deformable offset. The Euclidean length or modulus. Deformable offset. It is a two-dimensional vector representing the displacement of a pixel (x, y) during the blurring process. Its L2 norm quantifies the intensity or distance of the pixel's motion and is used as a direct indicator of motion intensity in the generation of the blur kernel. Deformable offset This refers to the estimated displacement vector of the i-th moving target at pixel coordinates (x, y), used to describe the pixel's displacement relative to its original position during the blurring process. It provides pixel-level motion trajectory information and is a key input for constructing an accurate spatial variation blur kernel. (Locally restored image) This refers to the sharp image fragment obtained after deblurring the i-th moving target region. This image fragment contains only the restored result of the corresponding moving target region, aiming to restore the details, texture, and edges of the moving target region and eliminate blurring effects. Regularized iterative reconstruction function. It is a tool for extracting from blurred images And the estimated first local spatial variation fuzzy kernel field Restore localized images An algorithm or module. This function is typically based on image deconvolution or optimization theory, achieving image restoration by iteratively solving an inverse problem. The introduction of a regularization term is to constrain the reconstruction process, avoiding noise amplification and artifact generation. Blurred image. This refers to the original input image containing motion blur, which is the starting point for deblurring processing. The first local spatial variation blur kernel field... It refers to the spatial region of the i-th moving target. The first local spatial transformation blur kernel corresponding to all pixels (x, y) defined above. A collection. It is a "field," meaning within a region. Each pixel within the region has its unique blur kernel, fully describing the non-uniform motion blur characteristics within the i-th moving target region. This indicates that the i-th moving target is in the blurred image. The spatial region within. Refers to area mask The set of all pixels with non-zero values ​​in the middle defines the blur kernel field. The scope of the operation is determined to ensure that the deblurring operation is performed only on the target motion area.

[0079] This application's solution achieves refined modeling and local restoration of motion blur by constructing a mapping mechanism from the feature space to the blur kernel space. Specifically, after acquiring the blurred image... And extract the region mask for each moving target. And based on these masks, motion feature information and deformable offsets are extracted. Subsequently, this application further proposes a method for determining the fuzzy kernel of the first local spatial variation. And obtain the locally restored image The specific steps are as follows. First, determine the first local spatial change fuzzy kernel. During the process, the system utilizes the second learnable mapping From blurry image Local feature map extracted Processing is performed to capture the deep semantic and texture information of the image. Simultaneously, deformable offsets are... L2 norm As a direct measure of exercise intensity, and The outputs are combined. This combination is achieved through a preset motion intensity adjustment factor. A weighted balancing process is applied to ensure that the generation of the blur kernel depends not only on the visual features of the image but also fully considers pixel-level motion displacement information. Finally, the combined result is normalized using the Softmax normalization exponential function to generate a pixel-level first local spatial variation blur kernel that satisfies physical constraints. This mechanism ensures that the generated blur kernel accurately reflects the local motion characteristics experienced by each pixel in the image during the blurring process, thus providing a highly accurate blur model for subsequent deblurring operations. Secondly, after obtaining the locally restored image... During the process, the system utilizes a regularized iterative reconstruction function. The original blurred image And for the i-th moving target region The first local spatial variation fuzzy kernel field defined above Combine them. Iterative reconstruction function. It can effectively utilize the fuzzy prior information of local regions and improve the quality of blurred images through an iterative optimization process. Deconvolution or reconstruction processing is performed. The introduction of a regularization term helps suppress noise amplification and artifact generation during the restoration process, ensuring the visual quality of the restored result. In this way, the system can perform differentiated and high-quality restoration for each moving target region, effectively suppressing artifacts caused by blurring while preserving the edge details of the moving target. Overall, the scheme in this application organically fuses feature information extracted from the blurred image with precise pixel-level motion offsets to construct a highly adaptive first local spatial variation blur kernel, and combines it with a regularized iterative reconstruction mechanism to achieve accurate modeling and efficient restoration of non-uniform blur in multi-moving target scenes.

[0080] Through the above technical solution, this application can effectively transform discrete deformable offset information into a precise pixel-level first local spatial variation blur kernel, and use this blur kernel to achieve fine restoration of a specific moving target region. Specifically, through a second learnable mapping... Fusion of local feature maps and deformable offset L2 norm The first local spatial variation fuzzy kernel is generated after normalization by the Softmax function. It can accurately capture the local motion characteristics of each pixel in an image, allowing the shape and intensity of the blur kernel to be stretched along the direction of motion, which is more consistent with the physical imaging process. Based on this, a regularized iterative reconstruction function is used. For blurry images and the first local spatial variation fuzzy core field This processing effectively utilizes prior information about blurred local areas, performing deconvolution or reconstruction on blurred images through an iterative optimization process. This preserves edge details of moving targets while suppressing artifacts caused by blurring. This significantly improves the ability to adaptively and faithfully restore various moving target regions in complex motion scenes, effectively solving the technical problem of traditional methods struggling to achieve accurate restoration when dealing with multiple moving targets.

[0081] In some embodiments, the specific steps in step S5 include: S51. A clear image is obtained by solving the following formula: ; ; ; ; in, for The second-order gradient features, Represented as the Laplace operator, To generate a clear image, Represented as the divergence operator, For the target gradient field, for The first-order gradient features, for The first-order gradient features, Represented as the gradient operator, Represented as pixel coordinates The pixel at that location belongs only to the i-th moving target. , indicating that it is for seeking and The difference set, Indicated as seeking union, Let be the spatial region of the j-th moving target in the blurred image. For the i-th moving target at pixel coordinates The fusion weight at the point, Indicated as to Find the L2 norm. for The first-order gradient features, For the i-th moving target at pixel coordinates Locally restored image at the location, Indicated as to Find the L2 norm. for The first-order gradient features, For the j-th moving target at pixel coordinates Locally restored image at the location, This is a regularization term.

[0082] Specifically, the Laplace operator It is a second-order differential operator, commonly used in image processing for edge detection and image sharpening. Its function is to capture the second-order gradient information of an image, reflecting changes in the image's curvature. (Divergence operator) Used to measure the degree of "divergence" of a vector field at a point, and when applied to a gradient field, it can describe the changes in image gradient. Target gradient field It is a vector field, where the vector at each point represents the gradient direction and magnitude of the image at that point. Constructing a smooth and consistent target gradient field is key to achieving seamless fusion. First-order gradient features and Representing the rate of change of image pixel grayscale values ​​in space, it can effectively capture edge information and texture details of the image and can be calculated by operators such as Sobel, Prewitt, or Roberts. Fusion weights This is a coefficient used to weighted average the gradients of different local reconstructed images in the overlapping region. It is determined based on the L2 norm (i.e., gradient magnitude) of the first-order gradient feature of each local reconstructed image at pixel coordinates (x, y). The larger the gradient magnitude, the more significant the edge information of that region, and the larger its corresponding fusion weight, thus dominating the fusion process. For example, normalization can be used to ensure that the sum of the weights of all local images is 1, or the gradient magnitude can be non-linearly mapped using the Sigmoid function before normalization. Regularization term It is a small positive number used to prevent the denominator from being zero or too small, thereby improving the numerical stability of the calculation. For example, It can be set to a very small constant, such as 1*10. -6 Or 1*10 -8 Alternatively, it can adaptively adjust based on the dynamic range of the image. (Expression) The region in the image that belongs only to the i-th moving target and does not overlap with any other moving target is defined. By taking the difference set, the unique pixel region of each moving target can be accurately identified. Within these unique regions, the gradient of the final clear image is directly taken from the gradient of the corresponding local restored image to preserve its original detail information.

[0083] This application's scheme achieves smooth reconstruction from locally restored images to a globally clear image through a gradient-guided Poisson fusion strategy. The scheme first processes each locally restored image... Gradient calculation is performed to obtain its first-order gradient characteristics. These gradient features can accurately capture the edge and texture information of each moving target region. Then, based on these first-order gradient features, especially their L2 norm (i.e., gradient magnitude), the fusion weight at pixel coordinates (x, y) of each local restored image is calculated. When a pixel belongs only to the i-th moving target (i.e. The region, the first-order gradient features of the final clear image. The gradient of the corresponding locally restored image is used directly. In areas where multiple moving target regions overlap, Then, by applying the gradient of each local restored image We obtain the weighted sum, where the weights are... Adaptive allocation based on gradient saliency ensures that high-gradient regions (i.e., regions rich in edge information) dominate the fusion process. This is achieved by introducing a regularization term. This effectively avoids numerical instability caused by zero or excessively small denominators, thus ensuring the stable and reliable calculation of the fusion weights. The target gradient field v is constructed using the above method, and this target gradient field v maintains smoothness and consistency across the entire image range. Finally, the Poisson equation is solved... That is, by utilizing the mathematical relationship between the divergence operator and the Laplacian operator, the constructed target gradient field is... Converted into a final clear image Second-order gradient features This allows for the reconstruction of a globally clear image. This gradient-guided Poisson fusion strategy, combined with the aforementioned local restoration processing based on region masks, multi-target local blur tensor models, deformable offsets, and a first local spatially varying blur kernel, forms a complete and efficient deblurring process from local to global, from blur kernel estimation to image reconstruction. It fully leverages the advantages of local restoration while overcoming the challenges of local image fusion, enabling the generation of high-quality, clear images even in complex multi-moving target scenes.

[0084] Through the above technical solution, this application can effectively eliminate brightness jumps, color mismatches, and structural breaks at the boundaries of multiple regions, achieving seamless visual stitching and significantly improving the overall naturalness and structural integrity of the output image, overcoming the "jigsaw puzzle effect" common in existing block-based deblurring methods. This gradient-guided Poisson fusion strategy, when using the gradient information of each locally restored image for fusion, can adaptively allocate weights according to the saliency of the gradient, ensuring a smooth transition in overlapping areas and avoiding artifacts and discontinuities common in traditional fusion methods. Simultaneously, by solving the Poisson equation to reconstruct the image from the gradient field, the gradient consistency of the final clear image is guaranteed globally, thereby generating high-quality, high-fidelity clear images in complex multi-moving target scenes.

[0085] In some embodiments, the trained multi-objective local fuzzy tensor model is obtained by training according to the following steps: A1. Obtain a preset standard clear image, and overlay at least two motion trajectories (represented as...) onto the standard clear image. ,in, Let be the trajectory of the i-th moving target at time t. (The total number of discrete sampling steps of the motion trajectory during the camera exposure time) generates the corresponding second local spatial change blur kernel; Specifically, the second local spatial transformation fuzzy kernel is generated according to the following formula: ; in, Let i be the pixel coordinates of the i-th moving target in the standard-resolution image after superimposing the motion trajectory. The second local spatial variation fuzzy kernel at that location, pixel coordinates The x-axis coordinate value at that location. pixel coordinates The y-coordinate value at that location. This represents the total number of discrete sampling steps of the motion trajectory during the camera's exposure time. Represented as a two-dimensional Dirac impulse function, Let x be the x-axis offset corresponding to the trajectory of the i-th moving target at time t. Let y be the y-axis offset corresponding to the trajectory of the i-th moving target at time t; A2. Convolve the second local spatial variation blur kernel with the standard clear image to synthesize training samples (i.e., blurred images). ); A3. Based on the preset total loss function, the untrained multi-objective local fuzzy tensor model is trained end-to-end using training samples to obtain the trained multi-objective local fuzzy tensor model; the total loss function includes an L1 loss term to constrain the overall image reconstruction accuracy (pixel level), a structural similarity loss term to constrain the instance restoration fidelity, and a gradient loss term to constrain edge consistency. Specifically, the total loss function is expressed as follows: ; in, For the total loss function, Indicated as to Find the L1 norm. For standard clear images, The first balance coefficient, Indicated as seeking and Structural similarity index between them Represented as and Perform element-wise multiplication. This is the second balance coefficient. Indicated as to Find the square of the L2 norm. for The first-order gradient feature.

[0086] The acquisition of a pre-defined standard clear image refers to obtaining the original image unaffected by any blur, typically used as the true label for training deep learning models. This can be achieved by capturing images with a professional high-resolution camera in a motion-free, shake-free environment, or by acquiring images from publicly available, rigorously selected and labeled image datasets, such as COCO and ImageNet. Superimposing at least two motion trajectories refers to the displacement path of a moving target on the image plane within the camera's exposure time. Superimposing motion trajectories can simulate the blurring caused by object motion in the real world. For example, simulated trajectories can be generated using mathematical functions (such as linear, curvilinear, or random walk functions), or by recording the motion path of real objects using an optical motion capture system. Generating a corresponding second local spatially varying blur kernel is a mathematical model describing the blur characteristics of local image regions. It reflects the light diffusion pattern caused by the displacement of the moving target at a specific pixel location. This blur kernel is spatially varying, meaning that the blur kernel may differ in different regions of the image, thus enabling accurate simulation of heterogeneous blur in multi-object scenes. The above formula sums the discretely sampled motion trajectories and combines them with a two-dimensional Dirac impulse function. To construct the fuzzy kernel. Among them, the two-dimensional Dirac impulse function... A value of 1 at the trajectory point and 0 at other locations indicates that at each sampling time, the light travels from the original clear position. Pixels diffused into the blurred image By exposure time Averaging the impulse functions of all sampling points yields the average blur effect of the moving target at that pixel location, thus forming the second local spatial variation blur kernel. Convolving this second local spatial variation blur kernel with a standard clear image synthesizes training samples, i.e., the blurred image. This convolution operation is a standard method for simulating blurring processes in image processing. Convolving the generated second local spatial variation blur kernel with a standard clear image can simulate the diffusion of light within a moving target region, thereby generating training samples with realistic blur characteristics. For example, frequency domain convolution can be performed using Fast Fourier Transform (FFT), or a sliding window multiplication-addition operation can be performed directly in the spatial domain. Based on a preset total loss function, the untrained multi-objective local blur tensor model is trained end-to-end using training samples to obtain a fully trained multi-objective local blur tensor model. The total loss function is an indicator that measures the difference between the model's prediction results and the true labels, guiding the optimization direction of the model parameters. End-to-end training refers to training the entire model as a whole, directly outputting a clear image from the input blurred image without intermediate manual intervention. The untrained multi-objective local blur tensor model is usually a deep neural network, whose parameters are iteratively updated during training through backpropagation and optimization algorithms (such as stochastic gradient descent, Adam, etc.) until the loss function converges, thus obtaining a fully trained model. The overall loss function includes an L1 loss term to constrain the overall image reconstruction accuracy (pixel-level), a structural similarity loss term to constrain the instance restoration fidelity, and a gradient loss term to constrain edge consistency. The L1 loss term (Mean Absolute Error, MAE) measures the sharpness of the image output by the model. With true standard clear image The absolute difference in pixel values ​​between two images. It operates directly at the pixel level, ensuring accurate reconstruction of basic visual attributes such as brightness and color of the overall image. The Structural Similarity Index Measure (SSIM) is a perceptual loss that measures the similarity between two images in terms of brightness, contrast, and structure. This is achieved by locally reconstructing the image for each moving target. With real and clear images Mask in the corresponding area Constraining the internal structural similarity ensures that when the model recovers a moving target, not only are pixel values ​​similar, but its internal structure and texture are also restored with high fidelity. The gradient loss term measures the difference between the gradient features of the model's output image and the gradient features of a truly sharp image. The image gradient reflects its edge and texture information. By constraining gradient consistency, the model can generate images with sharp edges and clear details, effectively avoiding common edge blurring or artifacts during deblurring. The above formula combines these three losses to form a multi-task joint optimization objective. Among them, It is the L1 loss term, which directly measures pixel-level differences; This is the structural similarity loss term, calculated by subtracting the SSIM value from 1. This ensures that the loss is small when the SSIM value is high and large when the SSIM value is low. Ensure that structural similarity is calculated only within the region of the moving target; It is a gradient loss term in L2 norm squared form, used to constrain edge consistency. and It is a balance coefficient used to adjust the weight of different loss terms in the total loss in order to achieve the best balance of the model across different performance metrics.

[0087] This training method provides a high-quality training foundation for multi-objective local blur tensor models by constructing an end-to-end supervised learning framework. Its core lies in accurately simulating the complex blurring process caused by multiple moving targets in the real world and designing a multi-dimensional loss function to comprehensively optimize model performance. Specifically, firstly, a pre-defined standard clear image is acquired and at least two motion trajectories are superimposed on it. A corresponding second local spatial variation blur kernel is generated using a two-dimensional Dirac impulse function. This method can accurately simulate, mathematically, the heterogeneous blur trajectories generated by different moving targets in a real scene within the camera exposure time, thus solving the problem of the lack of real physical correspondence in model training data. Subsequently, the generated second local spatial variation blur kernel is convolved with the standard clear image to synthesize training samples with multi-object blur characteristics, i.e., blurred images. This step enables mathematical modeling of complex blurring processes, providing reliable input-output pairs for model learning and inverse reconstruction. Based on this, this application uses a pre-defined total loss function and synthesized training samples to perform end-to-end training on an untrained multi-target local blurring tensor model. This total loss function is one of the key innovations of this scheme, cleverly combining L1 loss, structural similarity loss, and gradient loss. The L1 loss term constrains the reconstruction accuracy of the overall image at the pixel level, ensuring that the clear image output by the model is highly consistent with the real clear image in basic visual attributes such as brightness and color. The structural similarity loss term constrains the structural similarity between the locally restored image of each moving target and the real clear image within the corresponding region mask, enhancing the fidelity of instance restoration and enabling the model to better restore the internal structure and texture details of moving targets. Furthermore, the gradient loss term effectively improves the edge sharpness and structural integrity of the restored image by constraining the consistency of the first-order gradient features between the output image and the real clear image, avoiding edge blurring or artifacts common in deblurring processes. This multi-dimensional loss constraint mechanism enables the model to be comprehensively optimized from global to local, and from pixel to structure, thus significantly improving its image deblurring capabilities in complex multi-object scenes. The multi-object local blur tensor model trained using the above method can more accurately understand and reverse complex blur patterns caused by multiple independent moving targets. This is closely integrated with the steps in the aforementioned image deblurring method, which involve using this model to extract motion feature information, determine deformable offsets, generate the first local spatial change blur kernel, and perform restoration processing. This allows the entire deblurring process to perform refined processing for each moving target, ultimately generating high-quality, clear images through a gradient-guided Poisson fusion strategy, effectively overcoming the limitations of traditional methods in multi-object heterogeneous blur scenes.

[0088] As a specific implementation method, the trained multi-objective local blur tensor model can be trained in the following way. First, when acquiring a preset standard sharp image, a large number of high-resolution unblurred images can be selected from publicly available image datasets, such as ImageNet or COCO. These images encompass a rich variety of scenes and objects, which helps improve the model's generalization ability. Secondly, when overlaying at least two motion trajectories onto a standard, clear image, various strategies can be employed. For example, linear motion trajectories can be simulated, where the moving target moves at a constant speed and direction within the exposure time; or curved motion trajectories can be simulated, for example, by generating more complex non-linear motion paths using Bézier curves or random walk algorithms. Suppose we simulate two moving targets, the first target moving horizontally at a constant speed on the left side of the image, its trajectory... It can be represented as ,in The velocity is horizontal. The second target moves in a circle on the right side of the image; its trajectory is... It can be represented as ,in With the center of the circle, For radius, Angular velocity. Camera exposure time. The time interval can be set to 100 milliseconds, during which the motion trajectory is discretely sampled, for example, once every 1 millisecond, resulting in 100 sampling points. Based on these trajectories, a corresponding second local spatial variation blur kernel can be generated for each moving target using the above formula. For example, for linear motion, the blur kernel will appear as a straight line; for circular motion, the blur kernel will appear as an arc. Next, the generated second local spatial variation blur kernel is convolved with a selected standard sharp image to synthesize training samples. For example, for a standard sharp image containing cars and pedestrians, we can convolve the car region with a simulated linear motion blur kernel and the pedestrian region with a simulated curvilinear motion blur kernel, thus obtaining a blurred image containing two different motion blur modes. Finally, based on the preset total loss function, the untrained multi-objective local fuzzy tensor model is trained end-to-end using synthetic training samples. This model can be a deep convolutional neural network based on an encoder-decoder architecture, such as a variant of U-Net or ResNet. During training, the Adam optimizer can be used, with a learning rate set to 0.0001 and a batch size of 16. The balance coefficient in the total loss function... and Adjustments can be made based on experience or through hyperparameter search, for example... It can be set to 0.8. It can be set to 0.01. The training process will continue for multiple epochs until the model's performance on the validation set no longer shows significant improvement. In each training iteration, the model receives a blurred image. As input, the output is a predicted sharp image. Local restored images of each moving target The loss function calculates these outputs against the real, sharp image. The differences between them are analyzed, and the model's weights are updated through backpropagation, allowing the model to gradually learn how to accurately deblur.

[0089] By employing the aforementioned training method, this application effectively addresses the challenges of constructing training samples that accurately simulate the complex blurring characteristics of multiple moving targets during the training phase of a multi-target local blur tensor model, and of ensuring the model's comprehensive performance in pixel-level reconstruction, instance fidelity, and edge consistency through scientific loss function constraints. Specifically, by acquiring a pre-defined standard clear image and superimposing at least two motion trajectories to generate a second local spatial variation blur kernel, and further synthesizing training samples, this application can generate a large amount of training data with realistic physical correspondences and complex multi-target heterogeneous blur patterns. This significantly improves the model's generalization ability and adaptability to complex blurry scenes in the real world, overcoming the performance bottleneck caused by insufficient or unrealistic training data in traditional methods. Furthermore, the total loss function designed in this application, through the organic combination of L1 loss term, structural similarity loss term, and gradient loss term, achieves multi-dimensional and refined constraints on the model training process. The L1 loss term ensures the overall image reconstruction accuracy at the pixel level, making the deblurred image highly consistent with the real clear image in terms of basic visual attributes such as brightness and color. The structural similarity loss term imposes fidelity constraints on the local restored images of each moving target, effectively avoiding detail loss or structural distortion caused by mutual influence between targets during multi-target deblurring, ensuring high-quality restoration of each moving target. The gradient loss term enhances edge consistency, making the edges of the restored image sharper and clearer, effectively suppressing common edge blurring or artifacts during deblurring, thereby improving the visual quality of the image and the accuracy of subsequent analysis tasks. This comprehensive training strategy enables the trained multi-target local blur tensor model to more accurately extract motion feature information, determine deformable offsets, and generate a precise first local spatial change blur kernel, allowing for refined deblurring of each moving target in subsequent restoration processing. Finally, combined with a gradient-guided Poisson fusion strategy, seamless, high-quality clear images are generated, significantly improving image deblurring performance in complex multi-target scenes and avoiding artifacts and distortion problems caused by traditional methods in multi-target heterogeneous blur scenes.

[0090] Please refer to Figure 2 , Figure 2 This is an image deblurring processing device according to some embodiments of the present invention (the image deblurring processing system adopts the image deblurring processing method of the above embodiments, and the specific process is referred to the corresponding steps above). The image deblurring processing device is integrated into the back-end control device in the form of a computer program, including: The acquisition module 100 is used to acquire a blurred image and extract the region mask of each moving target in the blurred image; The extraction module 200 is used to extract motion feature information of the corresponding moving target based on the region mask and combined with the pre-trained multi-target local fuzzy tensor model; The determination module 300 is used to determine the deformable offset of each moving target in the spatial dimension based on motion feature information and the edge gradient information of the moving target. The restoration module 400 is used to determine the first local spatial change blur kernel corresponding to each moving target based on the deformable offset, and to use the first local spatial change blur kernel to perform restoration processing on the image region where each moving target is located, so as to obtain the local restored image of each moving target. The generation module 500 is used to extract the gradient features of each local restored image, and based on the gradient features, to generate a clear image by fusing the local restored images.

[0091] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The present invention provides an electronic device 13, including: a processor 1301 and a memory 1302. The processor 1301 and the memory 1302 are interconnected and communicate with each other through a communication bus 1303 and / or other forms of connection mechanism (not shown). The memory 1302 stores computer-readable instructions executable by the processor 1301. When the electronic device is running, the processor 1301 executes the computer-readable instructions to execute the image deblurring processing method in any optional implementation of the above embodiments, so as to achieve the following functions: acquiring a blurred image and extracting each moving object in the blurred image. The system first identifies the target region mask; then, based on the region mask and a pre-trained multi-target local fuzzy tensor model, it extracts the motion feature information of the corresponding moving targets; based on the motion feature information and the edge gradient information of the moving targets, it determines the deformable offset of each moving target in the spatial dimension; according to the deformable offset, it determines the first local spatial change fuzzy kernel corresponding to each moving target, and uses the first local spatial change fuzzy kernel to perform restoration processing on the image region where each moving target is located, obtaining the local restored image of each moving target; finally, it extracts the gradient features of each local restored image, and based on the gradient features, it performs fusion processing on each local restored image to generate a clear image.

[0092] This invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs an image deblurring method in any optional implementation of the above embodiments to achieve the following functions: acquiring a blurred image and extracting the region mask of each moving target in the blurred image; based on the region mask and combined with a pre-trained multi-target local blur tensor model, extracting motion feature information of the corresponding moving targets; based on the motion feature information and combined with the edge gradient information of the moving targets, determining the deformable offset of each moving target in the spatial dimension; determining the first local spatial change blur kernel corresponding to each moving target according to the deformable offset, and using the first local spatial change blur kernel to perform restoration processing on the image region where each moving target is located to obtain a local restored image of each moving target; extracting the gradient features of each local restored image, and generating a clear image by fusing the local restored images based on the gradient features.

[0093] The computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0094] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0095] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] Furthermore, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0097] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0098] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image deblurring method, characterized in that, Includes the following steps: S1. Acquire a blurred image and extract the region mask of each moving target in the blurred image; S2. Based on the region mask, and combined with a pre-trained multi-target local fuzzy tensor model, extract the motion feature information of the corresponding moving target; S3. Based on the motion feature information and combined with the edge gradient information of the moving target, determine the deformable offset of each moving target in the spatial dimension; S4. Based on the deformable offset, determine the first local spatial variation blur kernel corresponding to each of the moving targets, and use the first local spatial variation blur kernel to perform restoration processing on the image region where each of the moving targets is located, to obtain the local restored image of each of the moving targets; S5. Extract the gradient features of each of the local restored images, and based on the gradient features, perform fusion processing on each of the local restored images to generate a clear image.

2. The image deblurring method according to claim 1, characterized in that, The steps between step S1 and step S2 include: S6. Assign an independent deblurring processing branch to each of the moving targets according to the number of region masks, and each of the deblurring processing branches executes steps S3 and S4 in parallel; Step S6 is followed by the following steps: S71. Obtain the motion saliency index of each of the aforementioned moving targets; S72. When the motion saliency index is lower than a preset threshold, the corresponding deblurring processing branch is cropped, and the original blurring features of the corresponding image region where the moving target is located are preserved or replaced by a lightweight global deblurring algorithm.

3. The image deblurring method according to claim 1, characterized in that, The specific steps in step S2 include: S21. Using the preset fuzzy invariant feature extractor in the motion-aware instance segmentation network, combined with the multi-target local fuzzy tensor model, extract the feature map of the fuzzy image; S22. Based on the region mask, the motion response features of each moving target are obtained by performing element-wise multiplication on the feature map; the motion feature information includes the region mask, the feature map, and the motion response features.

4. The image deblurring method according to claim 3, characterized in that, The specific steps in step S3 include: S31. Calculate the deformable offset according to the following formula: ； in, For the i-th moving target at pixel coordinates Deformable offset at that location. For the first learnable mapping, pixel coordinates The corresponding local feature map, which is obtained based on the feature map. For the i-th moving target at pixel coordinates The edge gradient at the location, the edge gradient being based on the i-th moving target at pixel coordinates The local area mask is obtained.

5. The image deblurring method according to claim 4, characterized in that, The specific steps in step S4 include: S41. Determine the first local spatial variation fuzzy kernel according to the following formula: ； in, For the i-th moving target at pixel coordinates The first local spatial variation fuzzy kernel at the location, For normalized exponential functions, For the second learnable mapping, The preset exercise intensity adjustment factor, Indicated as to Find the L2 norm; S42. The locally restored image is obtained according to the following formula: ； ； in, This is a local reconstruction image of the i-th moving target. For regularized iterative reconstruction functions, Represented as the pixel coordinates of the i-th moving target Located in the region The first local spatial variation fuzzy kernel field defined above.

6. The image deblurring method according to claim 5, characterized in that, The specific steps in step S5 include: S51. The clear image is obtained by solving the following formula: ； ； ； ； in, for The second-order gradient features, Represented as the Laplace operator, For the generated clear image, Represented as the divergence operator, For the target gradient field, for The first-order gradient features, for The first-order gradient features, Represented as the gradient operator, Represented as pixel coordinates The pixel at that location belongs only to the i-th moving target. , indicating that it is for seeking and The difference set, Indicated as seeking The union of, The spatial region of the j-th moving target in the blurred image. For the i-th moving target at pixel coordinates The fusion weight at the point, Indicated as to Find the L2 norm. for The first-order gradient features, For the i-th moving target at pixel coordinates Locally restored image at the location, Indicated as to Find the L2 norm. for The first-order gradient features, For the j-th moving target at pixel coordinates Locally restored image at the location, This is a regularization term.

7. The image deblurring method according to claim 6, characterized in that, The trained multi-objective local fuzzy tensor model is obtained by the following steps: A1. Obtain a preset standard clear image, and superimpose at least two motion trajectories onto the standard clear image to generate a corresponding second local spatial change blur kernel; A2. Convolve the second local spatial variation blur kernel with the standard clear image to synthesize training samples; A3. Based on the preset total loss function, the training samples are used to perform end-to-end training on the untrained multi-objective local fuzzy tensor model to obtain the trained multi-objective local fuzzy tensor model.

8. An image deblurring processing apparatus, characterized in that, include: The acquisition module is used to acquire a blurred image and extract the region mask of each moving target in the blurred image; The extraction module is used to extract motion feature information of the corresponding moving target based on the region mask and in combination with a pre-trained multi-target local fuzzy tensor model; The determination module is used to determine the deformable offset of each of the moving targets in the spatial dimension based on the motion feature information and the edge gradient information of the moving targets; The restoration module is used to determine the first local spatial change blur kernel corresponding to each of the moving targets based on the deformable offset, and to use the first local spatial change blur kernel to perform restoration processing on the image region where each of the moving targets is located, so as to obtain the local restored image of each of the moving targets. The generation module is used to extract the gradient features of each of the local restored images, and based on the gradient features, to generate a clear image by fusing the local restored images.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions, which, when executed by the processor, perform the steps of the image deblurring method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the steps in the image deblurring method as described in any one of claims 1-7.

Images