Method for deblurring images taken by a buoy in abnormal weather conditions

Abstract

The present application relates to a kind of weather abnormal situation buoy shooting image deblurring method, the method comprises the following steps: S1, image acquisition and pretreatment: the original fuzzy ship image that is shot by the camera carried by buoy under weather abnormal condition is acquired;S2, model construction and training: the deep learning model for the deblurring of buoy ship image is constructed and trained, and is recorded as MCP-DeblurGANv2 model;S3, image deblurring processing: the fuzzy ship image obtained in step S1 is input to the MCP-DeblurGANv2 model trained in step S2, and the clear ship image after deblurring is output.The present application has the advantages that: not only solve the core bottleneck of prior art in bad sea conditions, such as poor image quality, structure distortion, dependence on paired data, but also provide key technical support for building high robustness, high automation of marine intelligent vision system.

Description

Technical Field

[0001] This invention relates to a method for deblurring images captured by buoys under abnormal weather conditions, belonging to the field of image processing technology. Background Technology

[0002] In fields such as marine environmental monitoring, maritime target identification, and intelligent shipping, optical imaging systems mounted on buoys are crucial for acquiring dynamic information about the sea surface. However, under abnormal weather conditions such as typhoons, heavy rain, dense fog, and strong winds, the buoys experience severe motion blur, out-of-focus blur, or compound blur in their images due to violent shaking and atmospheric disturbances, significantly impacting the accuracy of subsequent image analysis and target identification. Therefore, effectively restoring clear details from blurred images has become a key technical challenge for improving the visual perception capabilities of buoys.

[0003] In recent years, image deblurring technology has made significant progress, mainly divided into two categories: traditional methods and deep learning-based methods. Traditional methods, such as Wiener filtering and blind deconvolution, rely on accurate modeling of the point spread function (PSF). However, in real-world complex marine scenes, the blurring process is highly nonlinear and spatially variable, making it difficult to accurately estimate the blur kernel, resulting in limited restoration effects.

[0004] End-to-end deblurring methods based on convolutional neural networks (CNNs) and generative adversarial networks (GANs) are gradually becoming mainstream. "DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks" (CVPR 2018) was the first to introduce conditional GANs into image deblurring tasks, improving the visual quality of restored images through adversarial training. Its improved version, DeblurGAN-v2, further introduced a feature pyramid structure and an Inception-ResNet backbone network, achieving superior deblurring performance on multiple public datasets. However, these general deblurring models are typically trained on synthetic data, indoor data, and urban street view data, without considering the unique dynamic blurring characteristics of marine buoy platforms (such as long-trajectory blurring caused by low-frequency large-amplitude swaying, and non-uniform blurring caused by water vapor scattering). When directly applied to buoy images, they often exhibit artifacts, structural distortion, or loss of detail.

[0005] Recent works have attempted to jointly optimize fuzzy kernel estimation and image restoration. For example, SRN-DeblurNet implicitly learns fuzzy trajectories through a multi-scale recursive network. However, its computational cost is high, making it difficult to deploy on resource-constrained edge devices such as buoys. Lightweight designs often sacrifice restoration quality, failing to meet the demands of high-precision ocean observation. Furthermore, existing generator architectures typically optimize only a single task (such as sharpness), neglecting the multi-dimensional quality requirements of the restored image, such as geometric consistency and edge continuity. This leads to deformation, breakage, or proportional distortion of restored targets like ships, affecting subsequent downstream tasks such as target detection and size measurement.

[0006] In summary, current buoy image deblurring technology still faces three major bottlenecks: (1) a lack of explicit motion prior modeling for the blurring characteristics under abnormal marine weather conditions; (2) insufficient mechanisms to ensure physical rationality in the context of scarce monitoring signals; and (3) difficulty in balancing visual quality and geometric structure in the restoration target. Therefore, a novel deblurring method is urgently needed that is lightweight, physically consistent, and multi-task optimization-capable, for buoy application scenarios, in order to achieve high-quality and high-reliability restoration of blurred ship images. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention provides a method for deblurring images captured by buoys under abnormal weather conditions. The technical solution of this invention is as follows: A method for deblurring images captured by a buoy under abnormal weather conditions, the method comprising the following steps: S1. Image Acquisition and Preprocessing: Acquire raw, blurry images of the ship taken by the camera mounted on the buoy under abnormal weather conditions; S2. Model Building and Training: Build and train a deep learning model for deblurring images of buoy vessels, denoted as MCP-DeblurGANv2 model; S3. Image deblurring: Input the blurred ship image obtained in step S1 into the MCP-DeblurGANv2 model trained in step S2, and output the clear ship image after deblurring.

[0008] Step S2 includes the following sub-steps: S2.1 Constructing a training dataset: Based on clear ship images collected in real sea scene, simulate the movement pattern of buoys under abnormal weather conditions, perform motion convolution and Gaussian blur processing on the clear images to generate corresponding synthetic blurred images, and construct a pair of clear-blurred ship image datasets. S2.2 Training the deblurring model: Using the image dataset constructed in step S2.1 as input, train the MCP-DeblurGANv2 model end-to-end.

[0009] Step S2.1 specifically includes: For the acquired clear ship images, the blur angle and blur degree parameters are randomly set, and motion convolution and Gaussian blur are applied to simulate the image blur effect caused by sea wave swaying and atmospheric disturbance. The generated sharp-blurred image pairs are divided into training, validation, and test sets according to a ratio of 80%, 10%, and 10%, respectively.

[0010] The MCP-DeblurGANv2 model trained in step S2.2 has a generator that includes the following modules connected in sequence: A shared feature extraction backbone network module is used to extract multi-scale features from the input blurred image; A lightweight fuzzy trajectory estimation subnetwork module, connected to the backbone network, is used to predict spatially changing motion trajectory maps from the input image; The feature fusion and guidance module is used to fuse the motion trajectory map with the multi-scale features extracted by the backbone network module through an attention injection mechanism; The multi-task output head consists of a main branch for generating a clear image and an auxiliary branch for outputting geometric correction information.

[0011] The lightweight fuzzy trajectory estimation subnetwork is implemented based on the Transformer architecture, and its execution process includes: The input feature map is divided into multiple image blocks and embedded and encoded to convert it into sequence data. A multi-head self-attention mechanism is used to globally model the sequence and capture the dynamic dependencies between image patches; The modeled sequence data is deconvolved to restore the motion trajectory map in the spatial domain. The motion trajectory map is injected into the features of each layer of the feature pyramid network through element-wise multiplication attention to guide the deblurring process.

[0012] The specific execution flow of the lightweight fuzzy trajectory estimation subnetwork is as follows: (1) Flatten the features of each layer and concatenate them into a sequence for input into the Transformer: Where N is the total number of patches and d is the embedding dimension; (2) Introduce a lightweight Transformer module for global modeling: ; Includes multi-head self-attention network (MHSA) and feedforward network (FFN); It is the number of channels for motion priors; (3) Motion trajectory graph generation: Restore the Transformer output to a motion map in the spatial domain: ; (4) Incorporate prior knowledge of motion As a conditional signal, the attention mechanism guides the features of each layer of FPN: constructing attention weights for the FPN's... Layer features Calculate its cross-attention with the motion prior: ; ; in: A l For cross-attention, features from the l-th layer of FPN are used. With prior information about motion The similarity between them is used to measure their relevance. Scaling factor For activation functions; For motion trajectory diagram Height and width; Features are derived from prior motion information. Updated, α is the injection intensity coefficient. The attention injection function is defined here, and Conv is the convolution operation. (5) Finally, the generator receives the enhanced features. Perform sampling and reconstruction: .

[0013] In step S2.2, the model is trained using a composite loss function that includes a physical consistency cycle constraint, wherein the cycle consistency loss... Defined as: ; It is a cycle consistency constraint. It is the original blurred image that was input. It is a generator. It is a motion trajectory estimation subnetwork. It's a heavy blur operation, simulating the real imaging process; the blur is caused by camera shake due to abnormal weather. Modeling is done as a spatial variation convolution along the estimated trajectory: ; For blurring, it represents the blurring operator for a sharp image. Applying fuzzy kernel The resulting blurred image; Indicates the location The weight at the l-th displacement direction reflects spatial variability; This indicates that a clear image will be displayed in... The image is sampled after being offset along the direction of motion to simulate pixel trailing caused by camera shake. Applying an inverse problem regularization to the forward imaging model forces the deblurring mapping and physical blurring process learned by G. These are approximately inverse operations of each other, improving the physical plausibility of the solution: ; Represents a true, clear image. The true blurry trajectory, It is a fuzzy trajectory estimated by the network; The clear image output by the generator, after being reblurred again through the estimated motion trajectory map, should approximate the original blurred input, thus forming a self-supervised physical closed-loop constraint: this constraint encourages the network to learn a deblurring mapping relationship consistent with the real wind and wave motion, preventing the generation of pseudo-clear or non-physical image results; Through a cyclic consistency training mechanism based on fuzzy trajectories, the model can maintain consistency between motion and imaging processes even in the absence of clear labels, effectively improving its generalization ability and stability for complex motion fuzziness at sea. Complete loss function: ; It is pixel-level L1 loss. It is perceived loss. This is a hyperparameter.

[0014] The main branch generates a clear image, while the auxiliary branch predicts the horizon mask and camera pose angle. Multi-task collaborative optimization is achieved by introducing a joint loss function. For the deblurring task, MSE+Perceptual+Adversarial loss is used. ; in: ; ; ; Loss to the main task For pixel-level reconstruction loss, In order to perceive loss, To combat the losses, , These are the weighting coefficients for each sub-loss; The estimated sharp image is output by the generator's main branch. For true and clear images, For pre-training, For the first Weights of layer features; It is an expectation; The auxiliary task uses horizon segmentation or camera pose angle, with a total loss of: ; in: ; ; = It is the horizon mask for auxiliary branch prediction. It is the true horizon mask. It is a small constant, so the method is to divide by zero; It is the predicted camera attitude angle. It is a true deflection angle.

[0015] The total loss function Ltotal for model training is a weighted sum of multiple losses, expressed as: ; L aux For auxiliary branch loss; L consistency The loss is due to consistency constraints between tasks. , A reverse rotation transformation will produce a sharper image. Rotate in the opposite direction to the predicted deflection angle θ; λ aux , λ cons , λ cycle These are preset weighting coefficients used to balance the contributions of the auxiliary branch loss, consistency constraint loss, and cyclic consistency loss to the total loss.

[0016] Step S3 specifically includes: The preprocessed blurred image is input into the MCP-DeblurGANv2 model; The model generates motion trajectory maps through the lightweight fuzzy trajectory estimation subnetwork and guides the main branch to generate a preliminary clear image, while simultaneously predicting the horizon mask and camera pose angle through the auxiliary branch. Based on the output information of the auxiliary branch, the clear image generated by the main branch is geometrically corrected: if the output is a horizon mask, the horizon is fitted and the tilt angle is calculated for reverse rotation; if the output is an attitude deviation angle, the reverse rotation is performed directly according to that angle, and finally a geometrically consistent high-quality clear image is obtained.

[0017] The method is deployed on the edge computing device of the buoy platform, with a single frame image processing time of less than 100 milliseconds. It is suitable for online deblurring of real-time captured ship images to support subsequent target recognition, size measurement or behavior analysis tasks.

[0018] The advantages of this invention are: (1) First, this method can effectively improve the clarity and usability of blurred ship images captured by buoys under severe weather conditions such as strong winds, heavy rain, and dense fog. Traditional buoy imaging systems often result in severely blurred images due to violent shaking and atmospheric disturbances in abnormal weather, making them difficult to use for subsequent analysis; however, this invention introduces lightweight motion prior modeling, physical consistency self-supervised constraints, and a multi-task joint optimization mechanism to automatically recover structurally complete and detailed ship images, greatly improving the all-weather working capability of the maritime visual monitoring system.

[0019] (2) Secondly, clear and reliable ship images provide a high-quality data foundation for downstream tasks such as maritime target identification, ship type classification, size estimation, and behavior analysis. Relevant maritime supervision, fisheries law enforcement, or intelligent shipping systems can use this data to achieve more accurate target tracking and situational awareness, improve operational efficiency and decision-making accuracy, and reduce manual review costs and the risk of misjudgment.

[0020] Furthermore, this method can be deployed on a microserver at the buoy end, exhibiting excellent lightweight design and practicality, making it suitable for large-scale ocean observation networks. Its ability to model real-world physical fuzzy processes also enhances the model's generalization performance under different sea areas and platform conditions, avoiding the performance drop issues caused by scene migration in traditional defuzzification methods.

[0021] In summary, the buoy-based image deblurring method for ships under abnormal weather conditions, as presented in this invention, not only solves the core bottlenecks of existing technologies such as poor image quality, structural distortion, and reliance on paired data under adverse sea conditions, but also provides key technical support for building a highly robust and automated marine intelligent vision system. The widespread application of this method will strongly promote the development of marine monitoring, maritime security, and intelligent shipping towards greater reliability, efficiency, and intelligence, providing a solid technical guarantee for the national marine strategy and the development of the blue economy. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the present invention.

[0023] Figure 2 This is the architecture diagram of the MCP-DeblurGANv2 model of the present invention.

[0024] Figure 3 This is a structural block diagram of the fuzzy trajectory estimation subnetwork module of the present invention.

[0025] Figure 4 This is a comparison chart of the processing effects of the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to specific embodiments, and the advantages and features of the present invention will become clearer as a result. However, these embodiments are merely exemplary and do not constitute any limitation on the scope of the present invention. Those skilled in the art should understand that modifications or substitutions can be made to the details and form of the technical solutions of the present invention without departing from the spirit and scope of the present invention, but all such modifications and substitutions fall within the protection scope of the present invention.

[0027] See Figures 1 to 4 This invention relates to a method for deblurring images captured by a buoy under abnormal weather conditions. From images captured by a camera mounted on the buoy, clear and blurred images are separated from the image on a microserver through preprocessing. The selected clear image is used as source data, and based on the typical movement pattern of the buoy under abnormal weather conditions, it undergoes motion convolution and Gaussian blurring to construct a synthesized clear-blurred image pair. This image pair serves as input for model training. First, it is fed into a lightweight blur trajectory estimation subnetwork to achieve explicit prior modeling of the motion blur process. Subsequently, features with prior information are fed into an improved generative adversarial network, whose loss function incorporates a cyclic consistency constraint based on physical imaging mechanisms. That is, through closed-loop verification of "clear → blurred → restored," the restoration process is ensured to conform to the laws of real optical blur. Finally, the image output by the network is jointly optimized by a multi-task generator, improving visual clarity while effectively correcting geometric distortions such as horizontal line curvature caused by platform shaking, making the restored result closer to the real scene in terms of detail fidelity and structural rationality.

[0028] (1) The construction method of image deblurring model includes: preprocessing based on the data set of ship images taken in real sea scene.

[0029] 1) Methods for constructing a dataset of sharp-blurred ship images include: Images of ships on the sea surface were captured, including images of ships in motion, amidst waves, in low light, under strong exposure, and in sharp conditions. The sharp ship images in the sample images were then subjected to simulated blurring. The processed sharp-blurred image dataset was divided into a training set and a validation set according to a preset ratio, with the original blurred images used as the test set, resulting in the deblurred ship image dataset.

[0030] 2) The training method for the ship image deblurring model includes: preprocessing the ship image dataset to obtain preprocessed images; designing a deep learning image deblurring algorithm model, selecting the DeblurGANv2 deblurring algorithm, and embedding a lightweight fuzzy trajectory estimation sub-network in the generator for preprocessing the blurring. A cycle consistency constraint based on a fuzzy physical model is introduced into the loss function, forming a self-supervised physical closed-loop constraint.

[0031] The preprocessed image is input into the improved deblurring network for training. Error is backpropagated to update each weight and adjust the model's hyperparameters. The above steps are repeated, and the validation value and error value are compared to select the optimal model, thus obtaining the ship image deblurring model.

[0032] (2) Auxiliary branch predicts horizon mask branch to generate clear image The dual-branch method ensures that the horizon of the generated sharp image remains horizontal. After the sharp image is rotated in the opposite direction using the predicted attitude angle, it should be consistent with the original viewpoint, effectively avoiding the result of "sharp but distorted".

[0033] Specifically, the method includes the following steps: S1. Image Acquisition and Preprocessing: Acquiring raw, blurred images of the vessel taken by a camera mounted on the buoy under abnormal weather conditions. The raw images are captured by the camera installed on the ocean buoy and include the sea background, waves, the vessel, and blurring caused by camera shake due to wind and waves. Images are stored in JPEG or PNG format on a microserver mounted on the buoy, with a resolution of 1920×1080 and a frame rate of 5–10 fps. Acquisition periods cover various weather conditions, including sunny, cloudy, light rain, and heavy rain, to ensure data diversity.

[0034] S2. Model Construction and Training: A deep learning model for deblurring buoy vessel images, denoted as MCP-DeblurGANv2, is constructed and trained. The parameters of the adversarial network are continuously optimized through joint training of the generator and discriminator, ultimately obtaining a deblurring network capable of effectively processing blurred vessel images. The construction method of the vessel image deblurring model includes: constructing a dataset of "clear-blurred" image pairs; and training the image deblurring model based on the image pair dataset.

[0035] S3. Image Deblurring: The blurred ship image obtained in step S1 is input into the MCP-DeblurGANv2 model trained in step S2, and the output is a clear, deblurred ship image. Blurred ship images newly captured by buoys under abnormal weather conditions are used as input, uniformly preprocessed to 640×640 resolution RGB format, and pixel values ​​are normalized (scaled to the [0,1] interval). The preprocessed blurred image is input into the trained deblurring model. The model first analyzes the local motion pattern of the image through a built-in lightweight blurred trajectory estimation subnetwork, generating a corresponding spatial variation motion trajectory map; this trajectory map serves as a conditional prior, injected into the multi-scale features of the backbone network via an attention mechanism, guiding the deblurring process.

[0036] The backbone network extracts and fuses multi-level semantic features, which are then fed into two output branches: the main branch generates a preliminary deblurred and clear ship image, while the auxiliary branch simultaneously predicts the horizon mask and camera attitude angle.

[0037] After the model outputs, the results of the auxiliary branch are used to perform geometric correction on the main branch image: if the output is a horizon mask, the horizon line is fitted by Hough transform, the angle θ between it and the horizontal direction is calculated, and the clear image is rotated in reverse (using OpenCV's cv2.getRotationMatrix2D and cv2.warpAffine functions) to restore the horizon to horizontal; if the output is an attitude angle, the inverse rotation correction is performed directly according to the predicted angle.

[0038] The geometrically corrected image is the high-quality, clear ship image obtained by this invention. It preserves hull details while ensuring the rationality of the overall geometric shape, and can be directly used for subsequent tasks such as target detection, size measurement, or behavior analysis. The entire inference process can be completed in real time on a microserver, with a single frame processing time of less than 100 milliseconds.

[0039] Step S2 includes the following sub-steps: S2.1 Constructing a training dataset: Based on clear ship images collected in real sea scene, simulate the movement pattern of buoys under abnormal weather conditions, perform motion convolution and Gaussian blur processing on the clear images to generate corresponding synthetic blurred images, and construct a pair of clear-blurred ship image datasets. S2.2 Training the deblurring model: Using the image dataset constructed in step S2.1 as input, train the MCP-DeblurGANv2 model end-to-end.

[0040] Step S2.1 specifically includes: For the collected clear ship images, the blur angle and blur degree parameters are randomly set, and motion convolution and Gaussian blur are applied to simulate the image blur effect caused by sea wave swaying and atmospheric disturbance. The generated clear-blurred image pairs are divided into training set, validation set and test set according to the ratio of 80%, 10% and 10% respectively.

[0041] The MCP-DeblurGANv2 model trained in step S2.2 has a generator that includes the following modules connected in sequence: A shared feature extraction backbone network module is used to extract multi-scale features from the input blurred image; A lightweight fuzzy trajectory estimation subnetwork module, connected to the backbone network, is used to predict spatially changing motion trajectory maps from the input image; The feature fusion and guidance module is used to fuse the motion trajectory map with the multi-scale features extracted by the backbone network module through an attention injection mechanism; The multi-task output head consists of a main branch for generating a clear image and an auxiliary branch for outputting geometric correction information.

[0042] The lightweight fuzzy trajectory estimation subnetwork is implemented based on the Transformer architecture, and its execution process includes: The input feature map is divided into multiple image blocks and embedded and encoded to convert it into sequence data. A multi-head self-attention mechanism is used to globally model the sequence and capture the dynamic dependencies between image patches; The modeled sequence data is deconvolved to restore the motion trajectory map in the spatial domain. The motion trajectory map is injected into the features of each layer of the feature pyramid network through element-wise multiplication attention to guide the deblurring process.

[0043] To address the complex causes and dramatic spatial variations of blurring in maritime scenes, this paper embeds a lightweight Motion Trajectory Estimation Subnetwork (FPN) into the generator. This subnetwork explicitly predicts the direction and intensity distribution of spatially varying motion from the input blurred ship image. The subnetwork employs a Transformer-based structure to capture the dynamic dependencies between local blocks, thereby generating a motion map reflecting the swaying patterns of wind and waves. Structurally, this motion map serves as a conditional prior, embedded into the features of the FPN at various scales through an attention mechanism. This guides the generator to adaptively recover sharp details in different spatial regions. This approach achieves a fusion of data-driven learning and physical prior guidance, enhancing the generation process's physical interpretability and regional adaptability. The structure diagram of the Motion Trajectory Estimation Subnetwork is shown below. Figure 3 As shown.

[0044] The specific execution flow of the lightweight fuzzy trajectory estimation subnetwork is as follows: (1) Flatten the features of each layer and concatenate them into a sequence for input into the Transformer: Where N is the total number of patches and d is the embedding dimension; (2) Introduce a lightweight Transformer module for global modeling: ; Includes multi-head self-attention network (MHSA) and feedforward network (FFN); It is the number of channels for motion priors; (3) Motion trajectory graph generation: Restore the Transformer output to a motion map in the spatial domain: ; (4) Incorporate prior knowledge of motion As a conditional signal, the attention mechanism guides the features of each layer of FPN: constructing attention weights for the FPN's... Layer features Calculate its cross-attention with the motion prior: ; ; in: ; A l For cross-attention, features from the l-th layer of FPN are used. With prior information about motion The similarity between them is used to measure their relevance. Scaling factor For activation functions; For motion trajectory diagram Height and width; Features are derived from prior motion information. Updated, α is the injection intensity coefficient. The attention injection function is defined here, and Conv is the convolution operation. (5) Finally, the generator receives the enhanced features. Perform sampling and reconstruction: .

[0045] In step S2.2, the model is trained using a composite loss function that includes a physical consistency cycle constraint, wherein the cycle consistency loss... Defined as: ; It is a cycle consistency constraint. It is the original blurred image that was input. It is a generator. It is a motion trajectory estimation subnetwork. It's a heavy blur operation, simulating the real imaging process; the blur is caused by camera shake due to abnormal weather. ; For blurring, it represents the blurring operator for a sharp image. Applying fuzzy kernel The resulting blurred image; Indicates the location The weight at the l-th displacement direction reflects spatial variability; This indicates that a clear image will be displayed in... The image is sampled after being offset along the direction of motion to simulate pixel trailing caused by camera shake. Applying an inverse problem regularization to the forward imaging model forces the deblurring mapping and physical blurring process learned by G. ; Represents a true, clear image. The true blurry trajectory, It is a fuzzy trajectory estimated by the network; The clear image output by the generator, after being reblurred again through the estimated motion trajectory map, should approximate the original blurred input, thus forming a self-supervised physical closed-loop constraint: this constraint encourages the network to learn a deblurring mapping relationship consistent with the real wind and wave motion, preventing the generation of pseudo-clear or non-physical image results; Through a cyclic consistency training mechanism based on fuzzy trajectories, the model can maintain consistency between motion and imaging processes even in the absence of clear labels, effectively improving its generalization ability and stability for complex motion fuzziness at sea. Complete loss function: ; It is pixel-level L1 loss. It is perceived loss. For hyperparameters; Considering that blurred images of ships at sea often contain both visual degradation and geometric perturbations (such as horizon tilt and camera pose changes), the generator is further extended to a multi-task structure. Based on the main branch, two output branches are designed. The main branch generates the sharp image, while the auxiliary branches predict the horizon mask and camera pose angle. Multi-task collaborative optimization is achieved by introducing a joint loss function. For the deblurring task, MSE+Perceptual+Adversarial loss is used. ; in: ; ; ; Loss to the main task For pixel-level reconstruction loss, In order to perceive loss, To combat the losses, , These are the weighting coefficients for each sub-loss; The estimated sharp image is output by the generator's main branch. For true and clear images, For pre-training, For the first Weights of layer features; It is an expectation; The auxiliary task uses horizon segmentation or camera pose angle, with a total loss of: ; in: ; ; = It is the horizon mask for auxiliary branch prediction. It is the true horizon mask. It is a small constant, so the method is to divide by zero; It is the predicted camera attitude angle. It is a true deflection angle.

[0046] This invention also proposes inter-task consistency constraints to ensure that the horizon of the generated sharp image remains horizontal. This is achieved by rotating the sharp image in reverse using the predicted attitude angle, ensuring it matches the original viewpoint. This design enables the model to possess both visual sharpness and geometric consistency, effectively avoiding "sharp but distorted" results, and is particularly suitable for real-world image restoration in swaying marine environments. To ensure that the generated sharp image is also geometrically consistent, two inter-task consistency constraints are proposed. consistency .

[0047] The total loss function Ltotal for model training is a weighted sum of multiple losses, expressed as: ; L aux For auxiliary branch loss; L consistency The loss is due to consistency constraints between tasks. , A reverse rotation transformation will produce a sharper image. Rotate in the opposite direction to the predicted deflection angle θ; λ aux , λ cons , λ cycle These are preset weighting coefficients used to balance the contributions of the auxiliary branch loss, consistency constraint loss, and cyclic consistency loss to the total loss.

[0048] The MCP-DeblurGANv2 model consists of a shared feature extraction backbone network module, a lightweight fuzzy trajectory estimation sub-network module, a feature fusion and guidance module, and a multi-task output head. The process involves generating "sharp-blurred" image pairs, normalizing them, and then feeding them into the MobileNet-DSC backbone to extract multi-scale features. A feature pyramid is constructed using an FPN structure. An embedded lightweight motion trajectory estimation subnetwork predicts the spatially varying motion direction field from the local gradient field, generating a 2-channel motion map. This motion map, activated by a sigmoid function, serves as attention weights and is injected into the features of each FPN layer through element-wise multiplication, providing motion prior guidance. The backbone features are fused by a Concentration Layer and an Addition Layer, resulting in a main branch and an auxiliary branch: the main branch outputs a sharp RGB image, while the auxiliary branch outputs a horizon mask or camera pose angle. The loss function includes MSE, perceptual loss, adversarial loss, cycle consistency loss, and Dice auxiliary loss. Cycle consistency is achieved by re-blurring the generated sharp image according to the predicted trajectory and comparing it with the original blurred image. Global and local discriminators jointly optimize adversarial training, ultimately achieving joint optimization of visual sharpness and geometric consistency, effectively improving the image restoration quality in complex maritime environments. The entire process is achieved through forward propagation; during training, the network parameters are optimized in conjunction with the backpropagation algorithm and the aforementioned composite loss function, enabling the model to accurately recover ship images with reasonable structure, clear details, and consistent geometry.

[0049] The training process of the ship image deblurring model is as follows: First, a training dataset consisting of clear-blurred ship image pairs is prepared, and real-shot blurred ship images are used as independent test sets. Then, the network parameters are initialized according to the MCP-DeblurGANv2 network structure. Next, the network is trained end-to-end using the backpropagation algorithm and a composite loss function: the blurred image is input into the generator, and the clear image and horizon mask are output. By calculating the MSE loss, perceptual loss, adversarial loss, cycle consistency loss, and Dice loss for the auxiliary task, the difference between the restored result and the real clear image and geometric prior is comprehensively evaluated, and all learnable parameters of the generator and discriminator are updated accordingly. During the training process, a total of 300 epochs of iterative training are performed, that is, the entire training set is traversed 300 times, with a learning rate decay strategy (initial learning rate 2×10). -4 The model's performance is reduced to 0.5 times its original value every 50 epochs to ensure optimal convergence. Finally, the validation set is used to monitor metrics such as PSNR, SSIM, and horizon angle error, and the model's generalization ability and restoration quality for real-world blurred sea images are evaluated on an independent test set. The ship image deblurring model training is now complete.

[0050] To verify the good effect of the MCP-DeblurGANv2 algorithm proposed in this invention on image deblurring, an ablation experiment was conducted.

[0051] Step S3 specifically includes: The preprocessed blurred image is input into the MCP-DeblurGANv2 model; The model generates motion trajectory maps through the lightweight fuzzy trajectory estimation subnetwork and guides the main branch to generate a preliminary clear image, while simultaneously predicting the horizon mask and camera pose angle through the auxiliary branch. Based on the output information of the auxiliary branch, the clear image generated by the main branch is geometrically corrected: if the output is a horizon mask, the horizon is fitted and the tilt angle is calculated for reverse rotation; if the output is an attitude deviation angle, the reverse rotation is performed directly according to that angle, and finally a geometrically consistent high-quality clear image is obtained.

[0052] The method is deployed on the edge computing device of the buoy platform, with a single frame image processing time of less than 100 milliseconds. It is suitable for online deblurring of real-time captured ship images to support subsequent target recognition, size measurement or behavior analysis tasks.

[0053] In practical application, step S3 involves taking a newly captured blurred image of a ship under abnormal weather conditions as input, preprocessing it to a uniform 640×640 resolution RGB format, and normalizing the pixel values ​​(scaling them to the [0,1] range). The preprocessed blurred image is then input into a pre-trained deblurring model. The model first analyzes the local motion pattern of the image through a built-in lightweight blurred trajectory estimation sub-network, generating a corresponding spatial variation motion trajectory map. This trajectory map serves as a conditional prior, injected into the multi-scale features of the backbone network via an attention mechanism, guiding the deblurring process.

[0054] The backbone network extracts and fuses multi-level semantic features, which are then fed into two output branches: the main branch generates a preliminary deblurred and clear ship image, while the auxiliary branch simultaneously predicts the horizon mask and camera attitude angle.

[0055] After the model outputs, the results of the auxiliary branch are used to perform geometric correction on the main branch image: if the output is a horizon mask, the horizon line is fitted by Hough transform, the angle θ between it and the horizontal direction is calculated, and the clear image is rotated in reverse (using OpenCV's cv2.getRotationMatrix2D and cv2.warpAffine functions) to restore the horizon to horizontal; if the output is an attitude angle, the inverse rotation correction is performed directly according to the predicted angle.

[0056] The geometrically corrected image is the high-quality, clear ship image obtained by this invention. It preserves hull details while ensuring the rationality of the overall geometric shape, and can be directly used for subsequent tasks such as target detection, size measurement, or behavior analysis. The entire inference process can be completed in real time on a microserver, with a single frame processing time of less than 100 milliseconds.

[0057] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for deblurring images captured by a buoy under abnormal weather conditions, characterized in that, The method includes the following steps: S1. Image Acquisition and Preprocessing: Acquire raw, blurry images of the ship taken by a camera mounted on the buoy under abnormal weather conditions; S2. Model Building and Training: Build and train a deep learning model for deblurring images of buoys and ships, denoted as MCP-DeblurGANv2 model; S3. Image deblurring: Input the blurred ship image obtained in step S1 into the MCP-DeblurGANv2 model trained in step S2, and output the clear ship image after deblurring.

2. The method for deblurring images captured by a buoy under abnormal weather conditions according to claim 1, characterized in that, Step S2 includes the following sub-steps: S2.1 Constructing a training dataset: Based on clear ship images collected in real sea scene, simulate the movement pattern of buoys under abnormal weather conditions, perform motion convolution and Gaussian blur processing on the clear images to generate corresponding synthetic blurred images, and construct a pair of clear-blurred ship image datasets. S2.2 Training the deblurring model: Using the image dataset constructed in step S2.1 as input, train the MCP-DeblurGANv2 model end-to-end.

3. The method for deblurring images captured by a buoy under abnormal weather conditions according to claim 2, characterized in that, Step S2.1 specifically includes: For the acquired clear ship images, the blur angle and blur degree parameters are randomly set, and motion convolution and Gaussian blur are applied to simulate the image blur effect caused by sea wave swaying and atmospheric disturbance. The generated sharp-blurred image pairs are divided into training, validation, and test sets according to a ratio of 80%, 10%, and 10%, respectively.

4. The method for deblurring images captured by a buoy under abnormal weather conditions according to claim 2, characterized in that, The MCP-DeblurGANv2 model trained in step S2.2 has a generator that includes the following modules connected in sequence: A shared feature extraction backbone network module is used to extract multi-scale features from the input blurred image; A lightweight fuzzy trajectory estimation subnetwork module, connected to the backbone network, is used to predict spatially changing motion trajectory maps from the input image; The feature fusion and guidance module is used to fuse the motion trajectory map with the multi-scale features extracted by the backbone network module through an attention injection mechanism; The multi-task output head consists of a main branch for generating a clear image and an auxiliary branch for outputting geometric correction information.

5. The method for deblurring images captured by a buoy under abnormal weather conditions according to claim 4, characterized in that, The lightweight fuzzy trajectory estimation subnetwork is implemented based on the Transformer architecture, and its execution process includes: The input feature map is divided into multiple image blocks and embedded and encoded to convert it into sequence data. A multi-head self-attention mechanism is used to globally model the sequence and capture the dynamic dependencies between image patches; The modeled sequence data is deconvolved to restore the motion trajectory map in the spatial domain. The motion trajectory map is injected into the features of each layer of the feature pyramid network through element-wise multiplication attention to guide the deblurring process.

6. The method for deblurring images captured by a buoy under abnormal weather conditions according to claim 5, characterized in that, The specific execution flow of the lightweight fuzzy trajectory estimation subnetwork is as follows: (1) Flatten the features of each layer and concatenate them into a sequence for input into the Transformer: ;in, It is the total number of patches. It is the embedded dimension; (2) Introduce a lightweight Transformer module for global modeling: ; Includes multi-head self-attention network (MHSA) and feedforward network (FFN); It is the number of channels for motion priors; (3) Motion trajectory graph generation: Restore the Transformer output to a motion map in the spatial domain: ； (4) Incorporate prior knowledge of motion As a conditional signal, the attention mechanism guides the features of each layer of FPN: constructing attention weights for the FPN's... Layer features Calculate its cross-attention with the motion prior: ； ； in: ; For cross-attention, through the FPN... Layer features With prior information about motion The similarity between them is used to measure their relevance. Scaling factor For activation functions; For motion trajectory diagram Height and width; Features are derived from prior motion information. renew, The injection strength coefficient, The attention injection function is defined here, and Conv is the convolution operation. (5) Finally, the generator receives the enhanced features. Perform sampling and reconstruction: 。 7. The method for deblurring images captured by a buoy under abnormal weather conditions according to claim 6, characterized in that, In step S2.2, the model is trained using a composite loss function that includes a physical consistency cycle constraint, wherein the cycle consistency loss... Defined as: ; It is a cycle consistency constraint. It is the original blurred image input. It is a generator. It is a motion trajectory estimation subnetwork. It is a heavy blur operation that simulates the real imaging process; The blurriness is caused by camera shake due to abnormal weather conditions. Modeling is done as a spatial variation convolution along the estimated trajectory: ; For blurring, it represents the blurring operator for a sharp image. Applying fuzzy kernel The resulting blurred image; Indicates the location The weight at the l-th displacement direction reflects spatial variability; This indicates that a clear image will be displayed in... The image is sampled after being offset along the direction of motion to simulate pixel trailing caused by camera shake. Applying an inverse problem regularization to the forward imaging model forces the deblurring mapping and physical blurring process learned by G. These are approximately inverse operations of each other, improving the physical plausibility of the solution: ； Represents a true, clear image. The true blurry trajectory, It is a fuzzy trajectory estimated by the network; The clear image output by the generator, after being reblurred again through the estimated motion trajectory map, should approximate the original blurred input, thus forming a self-supervised physical closed-loop constraint: this constraint encourages the network to learn a deblurring mapping relationship consistent with the real wind and wave motion, preventing the generation of pseudo-clear or non-physical image results; Through a cyclic consistency training mechanism based on fuzzy trajectories, the model can maintain consistency between motion and imaging processes even in the absence of clear labels, effectively improving its generalization ability and stability for complex motion fuzziness at sea. Complete loss function: ； It is pixel-level L1 loss. It is perceived loss. This is a hyperparameter.

8. The method for deblurring images captured by a buoy under abnormal weather conditions according to claim 4, characterized in that, The main branch generates a clear image, while the auxiliary branch predicts the horizon mask and camera pose angle. Multi-task collaborative optimization is achieved by introducing a joint loss function. For the deblurring task, MSE+Perceptual+Adversarial loss is used. ； in: ; ； ； Loss to the main task For pixel-level reconstruction loss, In order to perceive loss, To combat the losses, , These are the weighting coefficients for each sub-loss; The estimated sharp image is output by the generator's main branch. For true and clear images, For pre-training, For the first Weights of layer features; It is an expectation; The auxiliary task uses horizon segmentation or camera pose angle, with a total loss of: ； in: ； ； = It is the horizon mask for auxiliary branch prediction. It is the true horizon mask. It is a small constant, so the method is to divide by zero; It is the predicted camera attitude angle. It is a true deflection angle.

9. The method for deblurring images captured by a buoy under abnormal weather conditions according to claim 8, characterized in that, The total loss function Ltotal for model training is a weighted sum of multiple losses, expressed as: ; L aux For auxiliary branch loss; L consistency The loss is due to consistency constraints between tasks. , A reverse rotation transformation will produce a sharper image. Rotate in the opposite direction to the predicted deflection angle θ; λ aux , λ cons , λ cycle These are preset weighting coefficients used to balance the contributions of the auxiliary branch loss, consistency constraint loss, and cyclic consistency loss to the total loss.

10. The method for deblurring images captured by a buoy under abnormal weather conditions according to claim 1 or 4, characterized in that, Step S3 specifically includes: The preprocessed blurred image is input into the MCP-DeblurGANv2 model; The model generates motion trajectory maps through the lightweight fuzzy trajectory estimation subnetwork and guides the main branch to generate a preliminary clear image, while simultaneously predicting the horizon mask and camera pose angle through the auxiliary branch. Based on the output information of the auxiliary branch, the clear image generated by the main branch is geometrically corrected: if the output is a horizon mask, the horizon is fitted and the tilt angle is calculated for reverse rotation; if the output is an attitude deviation angle, the reverse rotation is performed directly according to that angle, and finally a geometrically consistent quality clear image is obtained.

Images