Underwater image sharpening imaging method and system based on deep unfolding neural network

Abstract

This invention discloses an underwater image sharpening method and system based on a deep unfolded neural network: A deep feature map is obtained by feature encoding of a blurred underwater image; multiple physical variables in the deep feature map are initialized to obtain initial physical variable values; based on the initial physical variable values ​​and the deep feature map, the solution of a preset augmented Lagrangian function is iteratively optimized using a target optimization algorithm through a deep unfolded neural network to obtain optimized reflectivity features; the spatial resolution of the optimized reflectivity features is restored to the same size as the blurred underwater image, and the number of channels is adjusted to the number of channels corresponding to the blurred underwater image, resulting in an optimized and reconstructed reflectivity image. In this invention, each layer of the deep unfolded neural network has a clearly defined physical variable, making the image restoration process transparent. This fundamentally corrects color distortion, achieving interpretable, learnable, highly generalizable, and efficient underwater image sharpening.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to an underwater image sharpening imaging method and system based on a deep unfolded neural network. Background Technology

[0002] Underwater image sharpening aims to reverse the degradation process in underwater imaging, restoring the true colors and clear details of a scene. It has wide applications in underwater robot visual navigation, marine resource exploration, underwater archaeology, and ecological monitoring. The core challenge of this technology lies in simultaneously overcoming imaging degradation problems caused by water absorption (leading to color distortion) and particle scattering (leading to hazy blurring)—such as severe color distortion, decreased contrast, and blurred details. Therefore, effectively improving underwater image quality and achieving underwater image sharpening is a crucial technology that must be overcome.

[0003] In recent years, deep learning methods have developed rapidly in the field of image processing. Among related technologies, end-to-end training based on deep learning methods is commonly used. These methods train CNN (such as U-Net), GAN, or Transformer architectures by inputting blurred-sharp image pairs, such as Water-Net and Ucolor networks. Although end-to-end deep network models can directly learn the mapping of sharp images from blurred images, their internal working mechanism is similar to a "black box." This leads to a sharp drop in performance and insufficient generalization ability when facing new scenes with large differences from the training data distribution (such as different water quality and lighting conditions). At the same time, these methods usually have difficulty in explicitly separating and compensating for scattering and absorption effects, and cannot fundamentally correct color distortion. Summary of the Invention

[0004] To address the aforementioned problems in the existing technology, this invention provides an underwater image sharpening method based on a deep unfolded neural network. According to a first aspect of the present invention, an underwater image sharpening method based on a deep unfolded neural network is provided, the method comprising: The input underwater blurred image is feature encoded to obtain a deep feature map, and several specific physical variables in the deep feature map are initialized to obtain initial physical variable values; Based on the initial physical variable values ​​and the deep feature map, a specific depth-unfolded neural network is used to iteratively optimize the solution of the preset augmented Lagrangian function using a target optimization algorithm to obtain the optimized reflectivity features; the preset augmented Lagrangian function is a function obtained after mathematically modeling and transforming the underwater blurred image; the deep unfolded neural network contains a specified number of layers, and each layer corresponds to one iterative optimization, the specified number being equal to the number of iterations; The spatial resolution of the optimized reflectance features is restored to the same size as the underwater blurred image, and the number of channels is adjusted to the number of channels corresponding to the underwater blurred image to obtain the optimized and reconstructed reflectance image.

[0005] In one embodiment of the present invention, the physical variables include reflectance characteristics, Lagrange multipliers, auxiliary variables, illumination components, and transmittance maps. The step of mathematically modeling and transforming the underwater blurred image includes: Based on the Jaffe-McGlamery model, scene irradiance is decomposed into illumination components according to Retinex theory. and reflectivity characteristics Establish a joint degradation model ;in, For underwater blurred images, This is a transmittance diagram. This is the global background light map for the blurred underwater image. Through formula Based on the aforementioned joint degradation model, an optimization problem is constructed that includes a data fidelity term and a regularization term; wherein, The characteristic transformation matrix, Yes The applied non-smooth regularization term, For balance parameters; The optimization problem is transformed into an augmented Lagrangian function:

[0006] in, This represents the augmented Lagrange function. As an auxiliary variable, For Lagrange multipliers, This is a penalty factor.

[0007] In one embodiment of the present invention, the step of unfolding a neural network at a specific depth based on the initial physical variable values ​​and the deep feature map, and using a target optimization algorithm to iteratively optimize the solution of a preset augmented Lagrangian function to obtain optimized reflectivity features includes: By unfolding the first layer of the deep neural network, based on the initial illumination component, initial transmittance map, initial Lagrange multipliers, and the deep feature map, a variable optimization algorithm is used to perform the first iteration optimization on the auxiliary variables and reflectance features to obtain the auxiliary variables. and reflectivity characteristics ; Based on the initial reflectivity features, initial Lagrange multipliers, initial transmittance map, and the deep feature map, an illumination optimization algorithm is used to perform the first iteration optimization of the illumination components to obtain the illumination components. ; Based on the initial reflectivity features, initial illumination components, and the deep feature map, a transmittance optimization algorithm is used to perform the first iteration of transmittance optimization to obtain the transmittance map. ; Based on the initial reflectivity characteristics and initial auxiliary variables, the Lagrange multipliers are optimized in the first iteration using a coefficient optimization algorithm to obtain the Lagrange multipliers. ; The second layer of the deep neural network is used to expand the reflectivity feature. Lagrange multipliers Auxiliary variables Light component Transmittance diagram And the deep feature map, and perform a second iteration of optimization using the target optimization algorithm; The first through deep unfolded neural network m Layer, based on the first m The physical variables optimized in the -1st iteration and the deep feature map are then used to execute the target optimization algorithm in the -1st iteration. m The next iteration optimizes the reflectivity characteristics. m ≥3.

[0008] In one embodiment of the present invention, based on the initial illumination component, initial transmittance map, initial Lagrange multiplier, and the deep feature map, a variable optimization algorithm is used to perform a first iterative optimization of auxiliary variables and reflectance features to obtain auxiliary variables. and reflectivity characteristics ,include: Through formula Taking the partial derivative of the preset augmented Lagrange function with respect to the auxiliary variable yields the optimality condition for updating the auxiliary variable; where α is the balance parameter. As a penalty factor, As an auxiliary variable, The characteristic transformation matrix, The initial reflectivity characteristics, These are the initial Lagrange multipliers; definition for In parameters The proximal operator is obtained. The explicit expression for the update is: ; And through formula Based on the preset augmented Lagrange function, a target function for the reflectivity feature is constructed; wherein, Reflectivity characteristics This is the initial transmittance diagram. This is the initial illumination component. For the initial Lagrange multipliers, This refers to the deep feature map. This is the global background light map for the blurred underwater image. The characteristic transformation matrix, As initial auxiliary variables, As a penalty factor; Obtain the gradient of the objective function of the reflectivity feature, and define this gradient as a function. : And set it to zero; The function Equivalent transformation Through formula right Perform gradient calculation to obtain Explicit expression of updates: ,in, It is a proximal operator The generalized differential; Based on the above The update explicit expression is used to obtain the update direction of the reflectance feature, and the update expression of the reflectance feature is obtained: ,in Step size, k This represents the number of iterations.

[0009] In one embodiment of the present invention, based on the initial reflectivity features, the initial Lagrange multiplier, the initial transmittance map, and the deep feature map, an illumination optimization algorithm is used to perform a first iterative optimization of the illumination components to obtain the illumination components. ,include: Through formula Based on the preset augmented Lagrange function, an objective function for the illumination component is constructed; wherein, For light component, The initial reflectivity characteristics, This is the initial transmittance diagram. This refers to the deep feature map. is the global background light map of the underwater blurred image, and β is the balance parameter; Obtain the gradient of the objective function of the illumination component to obtain the optimality condition for updating the illumination component: ; The optimality condition of the illumination component is abstracted into a regularization term. The relevant network modules obtain the update expressions for the illumination components: , k This represents the number of iterations.

[0010] In one embodiment of the present invention, based on the initial reflectivity features, the initial illumination component, and the deep feature map, a transmittance optimization algorithm is used to perform a first iteration of optimization on the transmittance map to obtain the transmittance map. ,include: Through formula Based on the preset augmented Lagrangian function, the objective function of the transmittance map is constructed; wherein, This is a transmittance diagram. This is the initial illumination component. The initial reflectivity characteristics, This refers to the deep feature map. This is the global background light map of the underwater blurred image, where γ is the balance parameter; Obtain the gradient of the objective function of the transmittance map to obtain the optimality condition for updating the transmittance map: ; The optimality condition of the transmittance map is abstracted into a regularization term. The relevant network modules obtain the update expression for the transmittance map: , k This represents the number of iterations.

[0011] In one embodiment of the present invention, the Lagrange multipliers are optimized in the first iteration using a coefficient optimization algorithm based on initial reflectivity characteristics and initial auxiliary variables to obtain the Lagrange multipliers. ,include: Using formula Update the Lagrange multipliers; among them, For Lagrange multipliers, As a penalty factor, As initial auxiliary variables, The characteristic transformation matrix, The initial reflectivity characteristics, m This represents the number of iterations.

[0012] In one embodiment of the present invention, the step of unfolding a neural network at a specific depth based on the initial physical variable values ​​and the deep feature map, and using a target optimization algorithm to iteratively optimize the solution of a preset augmented Lagrangian function to obtain optimized reflectivity features includes: By unfolding a neural network at a specific depth and employing the alternating direction multiplier method framework, the solution of a predefined augmented Lagrange function is iteratively optimized using the following formula to obtain the optimized reflectivity characteristics: ; in, m For the number of iterations, Reflectivity characteristics This is a transmittance diagram. For light component, For Lagrange multipliers, As a penalty factor, The characteristic transformation matrix, These are initial auxiliary variables.

[0013] According to a second aspect of the present invention, an underwater image sharpening imaging system based on a deep unfolded neural network is provided, the system comprising: The feature encoding and variable initialization module is used to encode the input underwater blurred image to obtain a deep feature map, and to initialize multiple specific physical variables in the deep feature map to obtain initial physical variable values. A deep unfolded neural network is used to perform iterative optimization of the solution of a preset augmented Lagrangian function based on the initial physical variable values ​​and the deep feature map, through a specific deep unfolded neural network and a target optimization algorithm, to obtain the optimized reflectivity features; the preset augmented Lagrangian function is a function obtained by mathematically modeling and transforming the underwater blurred image; the deep unfolded neural network contains a specified number of layers, each layer corresponding to one iterative optimization, and the specified number is equal to the number of iterations; The feature decoding and output module is used to restore the spatial resolution of the optimized reflectance features to the same size as the underwater blurred image, and adjust the number of channels to the number of channels corresponding to the underwater blurred image, so as to obtain the optimized and reconstructed reflectance image.

[0014] In one embodiment of the present invention, the deep unfolded neural network includes a specified number of layers, and each layer includes an image update module, a degradation parameter update module, and a Lagrange multiplier update module; The image update module is used to update based on the first... mThe illumination components, transmittance map, Lagrange multipliers, and deep feature map after the -1st iteration optimization are then used to optimize the auxiliary variables and reflectance features using a variable optimization algorithm. m The next iteration optimizes and obtains auxiliary variables. and reflectivity characteristics ; The degradation parameter update module is used to update the parameters based on the first... m The reflectivity features, Lagrange multipliers, transmittance map, and deep feature map after the -1st iteration optimization are used to optimize the illumination components using an illumination optimization algorithm. m The next iteration of optimization yields the illumination components. ; and used based on the first m The reflectivity features after the -1st iteration optimization, the illumination component, and the deep feature map are used to perform the 1st iteration optimization on the transmittance map. m Subsequent iterations of optimization yield the transmittance map. ; The Lagrange multiplier update module is used to update based on the first... m The reflectivity characteristics and auxiliary variables after the -1st iteration optimization are used to optimize the Lagrange multipliers using a coefficient optimization algorithm. m The next iteration of optimization yields the Lagrange multipliers. ; in, m This represents the number of iterations.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The underwater image sharpening method and system based on a deep unfolded neural network provided in this invention first encodes the input blurred underwater image to obtain a deep feature map, and initializes several specific physical variables in the deep feature map to obtain initial physical variable values. Then, based on these initial physical variable values ​​and the aforementioned deep feature map, a specific deep unfolded neural network is used to iteratively optimize the solution of a preset augmented Lagrangian function using a target optimization algorithm to obtain optimized reflectivity features (wherein the preset augmented Lagrangian function is a function obtained after mathematical modeling and transformation of the blurred underwater image, the aforementioned deep unfolded neural network contains a specified number of layers, each layer corresponds to one iterative optimization, and the aforementioned specified number is equal to the number of iterations). Finally, the spatial resolution of the optimized reflectivity features is restored to the same size as the blurred underwater image, and the number of channels is adjusted to the number of channels corresponding to the blurred underwater image to obtain the optimized and reconstructed reflectivity image. This invention designs a novel network architecture based on an underwater imaging physical model. Unlike the "black box" end-to-end networks in related technologies, the deep unfolded neural network of this invention contains a specified number of layers, and each layer corresponds to an iterative optimization. That is, each network operation has a clear physical variable, making the image restoration process transparent. It can fundamentally correct color distortion and effectively avoid phenomena such as color oversaturation and artifacts. It achieves interpretable, learnable, highly generalizable, and efficient underwater image sharpening.

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] Figure 1 A flowchart illustrating the steps of an underwater image sharpening method based on a deep unfolded neural network, as provided in this embodiment of the invention. Figure 2 A schematic diagram of an underwater image sharpening imaging system based on a deep unfolded neural network provided in an embodiment of the present invention; Figure 3 A schematic diagram of a network architecture provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the network architecture of the image update module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the network design for the degradation parameter update module provided in an embodiment of the present invention; Figure 6 A schematic diagram of a basic module design provided for an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1 The core technical solution of this invention is an underwater image sharpening network based on a semi-smooth Newton's method depth expansion. This method systematically expands the physical model optimization solution process for underwater image restoration into an end-to-end trainable deep learning architecture. The following sections will elaborate on the physical modeling, optimization algorithm design, and network expansion mapping.

[0020] This invention requires the pre-construction of an expandable and efficient physical optimization solution process: an optimization algorithm based on the semi-smooth Newton's method is designed to solve the non-smooth, nonlinear energy minimization problem derived jointly from the underwater imaging physical model and Retinex theory. This solution process has clear and fixed iterative steps, thus providing a stable and structured template for subsequent depth algorithm development.

[0021] Physics-Vision Joint Modeling and Problem Formulation: First, a mathematical model of the underwater degradation process is constructed—based on the Jaffe-McGlamery model, the scene irradiance is decomposed into illumination components according to Retinex theory. and reflectivity characteristics Establish a joint degradation model ;in, For underwater blurred images, This is a transmittance diagram. This is the global background light map for the underwater blurred image.

[0022] In order to Recovering reflectivity characteristics Light component and transmittance diagram We can construct an optimization problem that includes data fidelity terms and regularization terms, namely: Through formula Based on the aforementioned joint degradation model, an optimization problem is constructed that includes a data fidelity term and a regularization term; whereby, The characteristic transformation matrix, Yes The applied non-smooth regularization term is used to introduce prior knowledge. For balancing parameters.

[0023] Constructing an optimization solution framework based on the semi-smooth Newton's method: The optimization problem described above is non-convex and non-smooth. Traditional gradient descent methods converge slowly and are prone to getting trapped in local optima. To solve this problem efficiently, this invention introduces an auxiliary variable. The above optimization problem is transformed into an optimization problem with equality constraints, and an augmented Lagrangian function is constructed: (1) in, This represents the augmented Lagrange function. As an auxiliary variable, For Lagrange multipliers, This is a penalty factor.

[0024] The above process of optimizing the augmented Lagrangian function can then be systematically expanded into an end-to-end trainable deep learning architecture.

[0025] Reference Figure 1 This document illustrates a flowchart of a method for enhancing underwater images based on a deep unfolded neural network, according to Embodiment 1 of the present invention. The underwater image enhancement method based on a deep unfolded neural network in this embodiment includes the following steps: Step 101: Perform feature encoding on the input underwater blurred image to obtain a deep feature map, and initialize several specific physical variables in the deep feature map to obtain initial physical variable values.

[0026] For example, the input underwater blurred image can be... By downsampling, the spatial resolution is reduced by a factor of 16, and the number of channels is increased by a factor of 16. Then, a convolutional layer is used for feature fusion and channel adjustment to output a deep feature map. The purpose of this step is to optimize in a higher-dimensional, lower-resolution feature space to reduce computational costs and capture richer semantic information. Afterwards, several specific physical variables in the deep feature map are initialized to obtain multiple initial physical variable values, such as: initial illumination components, initial transmittance map, initial Lagrange multipliers, initial reflectance features, and initial auxiliary variables. It is worth noting that in this embodiment, due to network performance limitations, directly inputting I into the network does not meet the computational requirements, so I is downsampled. However, downsampling does not result in information loss; instead, the spatial resolution changes from 256×256 to 64×64, and the number of channels increases from 3 to 48. Therefore, I can be considered to be related to... Id They are equivalent.

[0027] Step 102: Based on the initial physical variable values ​​and deep feature maps, expand the neural network to a specific depth, and use the target optimization algorithm to iteratively optimize the solution of the preset augmented Lagrangian function to obtain the optimized reflectivity features.

[0028] Among them, the preset augmented Lagrangian function is the function obtained after mathematical modeling and transformation of the underwater blurred image, i.e., the aforementioned formula (1); the deep unfolded neural network contains a specified number of layers, and each layer corresponds to one iteration optimization, the specified number being equal to the number of iterations.

[0029] Specifically, a neural network can be expanded to a specific depth, and using the alternating direction multiplier method framework, the solution of a predefined augmented Lagrangian function can be iteratively optimized using the following formula to obtain the optimized reflectivity features: ; in, m For the number of iterations, Reflectivity characteristics This is a transmittance diagram. For light component, For Lagrange multipliers, As a penalty factor, The characteristic transformation matrix, These are the initial auxiliary variables. It's clear that the core idea is to use the first... m In this iteration, first fix the other variables, then update the remaining variables in sequence. For example: fix ,conduct and Updates; Fixes Perform illumination component Updates; Fixes Perform transmittance mapping Update.

[0030] It should be noted that the embodiments of the present invention are based on the augmented Lagrange and alternating direction multiplier (ADMM) framework. Similarly, the semi-quadratic splitting method (HQS) or the proximal gradient descent method (PGD) can be used to solve formula (1), and the corresponding iterative steps are expanded into networks with different connection structures and module update orders. The networks expanded by different solvers are all extensions of the ideas protected by this patent.

[0031] In this embodiment, during the first optimization iteration, the first layer of the deep neural network can be unfolded. Based on the initial illumination component, initial transmittance map, initial Lagrange multipliers, and deep feature map, a variable optimization algorithm is used to perform the first iteration optimization on the auxiliary variables and reflectance features to obtain the auxiliary variables. and reflectivity characteristics Based on initial reflectivity features, initial Lagrange multipliers, initial transmittance maps, and deep feature maps, an illumination optimization algorithm is used to perform the first iteration of illumination component optimization to obtain the illumination components. Based on the initial reflectivity features, initial illumination components, and deep feature maps, a transmittance optimization algorithm is used to perform the first iteration of transmittance map optimization to obtain the transmittance map. Based on the initial reflectivity characteristics and initial auxiliary variables, the Lagrange multipliers are optimized in the first iteration using a coefficient optimization algorithm to obtain the Lagrange multipliers. .

[0032] Specifically, based on the initial illumination component, initial transmittance map, initial Lagrange multiplier, and the deep feature map, a variable optimization algorithm is used to perform the first iteration optimization on the auxiliary variables and reflectance features to obtain the auxiliary variables. and reflectivity characteristics At that time, it can be done through the formula Taking the partial derivative of the pre-defined augmented Lagrangian function with respect to the auxiliary variable yields the optimality condition for updating the auxiliary variable, which is equivalent to solving a proximal operator; where α is the equilibrium parameter. As a penalty factor, As an auxiliary variable, The characteristic transformation matrix, The initial reflectivity characteristics, Let the initial Lagrange multipliers be defined; then define... for In parameters The proximal operator is obtained. The explicit expression for the update is: .

[0033] Through formula Based on a pre-defined augmented Lagrange function, an objective function for reflectivity characteristics is constructed; whereby, Reflectivity characteristics This is the initial transmittance diagram. This is the initial illumination component. For the initial Lagrange multipliers, For deep feature maps, This is the global background light map for the blurred underwater image. The characteristic transformation matrix, As initial auxiliary variables, This is the penalty factor. Next, the gradient of the objective function for reflectivity features is obtained, and this gradient is defined as the function... : (Will Substituting this gradient), since It is a strictly convex quadratic function, and its optimality condition is that its gradient is 0. Therefore, for Solving this equation is equivalent to solving the nonlinear equation: Directly solving this expression requires calculating and inverting it. The generalized Jacobian matrix is ​​computationally intensive. This invention employs an equivalent transformation strategy: noting that... It is a certain convex function The gradient of . Therefore, let After the function is zero Equivalently transformed into an optimization problem: Then through the formula right Perform gradient descent calculations to obtain Explicit expression of updates: ,in, It is a proximal operator The generalized differential (for soft threshold, (1 at non-zero locations, 0 at zero locations), thus obtaining the search direction and reflectivity characteristics. The update proceeds along this direction of the search. Finally, based on... The update is explicitly expressed to obtain the update direction of the reflectance feature, and the update expression of the reflectance feature is obtained: ,in Step size, k This represents the number of iterations.

[0034] Based on the initial reflectivity features, initial Lagrange multipliers, initial transmittance map, and the deep feature map, the illumination components are iteratively optimized using an illumination optimization algorithm to obtain the illumination components. At that time, it can be done through the formula The objective function for the illumination component is constructed based on the pre-defined augmented Lagrange function; among which, For light component, The initial reflectivity characteristics, This is the initial transmittance diagram. This refers to the deep feature map. The global background illumination map is given by β, where β is the balance parameter. Next, the gradient of the objective function for the illumination component is obtained, leading to the optimal conditions for updating the illumination component: Then, the optimality condition of the illumination component is abstracted into the regularization term. The relevant network modules obtain the update expressions for the illumination components: , k This represents the number of iterations.

[0035] Based on the initial reflectivity characteristics, initial illumination components, and the deep feature map, a transmittance optimization algorithm is used to perform the first iteration of transmittance map optimization to obtain the transmittance map. At that time, it can be done through the formula The objective function for constructing the transmittance map is based on a pre-defined augmented Lagrange function; where, This is a transmittance diagram. This is the initial illumination component. The initial reflectivity characteristics, This refers to the deep feature map. This is the global background light map of the underwater blurred image, where γ is the balance parameter.

[0036] Next, the gradient of the objective function of the transmittance map is obtained, and the optimality condition for updating the transmittance map is derived: Then, the optimality condition of the transmittance map is abstracted into a regularization term. The relevant network modules obtain the update expression for the transmittance map: , k This represents the number of iterations.

[0037] Based on the initial reflectivity characteristics and initial auxiliary variables, the Lagrange multipliers are optimized in the first iteration using a coefficient optimization algorithm to obtain the Lagrange multipliers. At that time, formulas can be used Update the Lagrange multipliers; among them, For Lagrange multipliers, As a penalty factor, As initial auxiliary variables, The characteristic transformation matrix, The initial reflectivity characteristics, m This represents the number of iterations.

[0038] Thus, a complete iterative optimization has been completed in the first layer of the deep unfolded neural network. Then, in the second, third, and subsequent layers of the deep unfolded neural network... m Each layer undergoes one iteration of optimization: During the second optimization iteration, the second layer of the neural network can be deeply expanded, based on the reflectivity features obtained from the first iteration. Lagrange multipliers Auxiliary variables Light component Transmittance diagram In addition to the aforementioned deep feature map, a second iteration of optimization is performed using the target optimization algorithm.

[0039] During the first m During the optimization iteration, the th iteration of the deep unfolded neural network is performed. mLayer, based on the first m The physical variables optimized in step -1 and the aforementioned deep feature map are then used to execute the objective optimization algorithm in the... m The next iteration optimizes the reflectivity features, where... m ≥3.

[0040] Through multiple iterative optimizations across multiple layers in a deep neural network, physical variables... Under certain conditions, it will converge to a stable point (usually a local minimum) of the original problem. The number of iterations mentioned above can be set based on experience.

[0041] Step 103: Restore the spatial resolution of the optimized reflectance features to the same size as the underwater blurred image, and adjust the number of channels to the number of channels corresponding to the underwater blurred image to obtain the optimized and reconstructed reflectance image.

[0042] go through m After iterative optimization at each stage, the network obtains the final optimized reflectivity features. This feature can then be processed by a feature decoding module (up_scale), which will... The spatial resolution is restored to the size of the original input image (underwater blurred image), and the number of channels is adjusted to the original number of channels to obtain the reconstructed reflectance image. .

[0043] The underwater image sharpening method based on a deep unfolded neural network provided in this invention first encodes the input blurred underwater image to obtain a deep feature map, and initializes several specific physical variables in the deep feature map to obtain initial physical variable values. Then, based on these initial physical variable values ​​and the aforementioned deep feature map, a specific deep unfolded neural network is used to iteratively optimize the solution of a preset augmented Lagrangian function using a target optimization algorithm to obtain optimized reflectivity features (where the preset augmented Lagrangian function is a function obtained after mathematical modeling and transformation of the blurred underwater image, the aforementioned deep unfolded neural network contains a specified number of layers, each layer corresponds to one iterative optimization, and the aforementioned specified number is equal to the number of iterations). Finally, the spatial resolution of the optimized reflectivity features is restored to the same size as the blurred underwater image, and the number of channels is adjusted to the number of channels corresponding to the blurred underwater image to obtain the optimized and reconstructed reflectivity image. This invention designs a novel network architecture based on an underwater imaging physical model, which has the following beneficial effects: (1) Strong interpretability and physical consistency: Unlike the "black box" end-to-end network in related technologies, the deep unfolded neural network of this invention contains a specified number of layers, each layer corresponds to one iteration optimization, and each network operation has clear physical variables (reflectivity features, illumination components, transmittance maps, etc.), making the image restoration process transparent and fundamentally correcting color distortion, effectively avoiding color oversaturation, artifacts, and other phenomena. (2) Excellent generalization and robustness: Although the network parameters are learned from the data, the framework of the physical model provides strong constraints and inductive biases, so that the network learns the essential degradation reversal mechanism, rather than the statistical patterns on the surface of the training data; therefore, when facing water quality, illumination, or scenes not covered by the training set, the model performs more stably and reliably. (3) High-efficiency restoration speed: The traditional optimization process that requires tens or hundreds of iterations and is time-consuming is compressed into a computation that only requires a few forward propagations (corresponding to the network depth), achieving near real-time high-efficiency processing.

[0044] Example 2 The underwater image sharpening imaging system based on a deep unfolded neural network, as described below, will be explained in detail below. Figure 2 The diagram shown is a schematic representation of an underwater image sharpening system based on a deep unfolded neural network provided in an embodiment of this application. The system includes: The feature encoding and variable initialization module 20 is used to encode the input underwater blurred image to obtain a deep feature map, and initialize several specific physical variables in the deep feature map to obtain initial physical variable values. The deep unfolded neural network 21 is used to perform iterative optimization of the solution of a preset augmented Lagrangian function using a target optimization algorithm based on the initial physical variable values ​​and the deep feature map, thereby obtaining optimized reflectivity features. The preset augmented Lagrangian function is a function obtained after mathematical modeling and transformation of the underwater blurred image. The deep unfolded neural network contains a specified number of layers, each corresponding to one iterative optimization, and the specified number equals the number of iterations. The feature decoding and output module 22 is used to restore the spatial resolution of the optimized reflectivity features to a size consistent with the underwater blurred image, and adjust the number of channels to the number of channels corresponding to the underwater blurred image, thereby obtaining the optimized and reconstructed reflectivity image.

[0045] Specifically, in combination Figure 3 , Figure 3 This is a schematic diagram of the network architecture designed according to an embodiment of the present invention. (Refer to...) Figure 3 A deep unfolded neural network contains a specified number of layers ( Figure 3 The green area represents each level, which includes an image update module (IU module), a degradation parameter update module (DU module), and a Lagrange multiplier update module (MU module). The image update module is used to update parameters based on the first... m After the -1 iteration optimization, the illumination components, transmittance map, Lagrange multipliers, and deep feature map are used to optimize the auxiliary variables and reflectance features using a variable optimization algorithm. m The next iteration optimizes and obtains auxiliary variables. and reflectivity characteristics The degradation parameter update module is used to update parameters based on the first... m The reflectivity features, Lagrange multipliers, transmittance map, and deep feature map after the -1st iteration optimization are used to optimize the illumination components using an illumination optimization algorithm. m The next iteration of optimization yields the illumination components. Also used based on the first m The reflectivity features, illumination components, and deep feature maps after the -1st iteration optimization are used to perform the 1st iteration optimization on the transmittance map. m Subsequent iterations of optimization yield the transmittance map. The Lagrange multiplier update module is used to update the Lagrange multipliers based on the first... m The reflectivity characteristics and auxiliary variables after the -1st iteration optimization are used to optimize the Lagrange multipliers using a coefficient optimization algorithm. m The next iteration of optimization yields the Lagrange multipliers. ;in, m This represents the number of iterations.

[0046] refer to Figure 3 In this embodiment, the input underwater blurred image First, through a downsampling module ( Figure 3 (Lower section) This module can reduce the image spatial resolution by 16 times and increase the number of channels by 16 times. After passing through a convolutional layer for feature fusion and channel adjustment, it outputs a deep feature map. Then, in the PI module, the required physical variables are... Perform initialization to obtain the initial physical variable values.

[0047] The initial physical variable values ​​and deep feature maps are input into the core optimization module, which consists of K identical stages cascaded sequentially. The parameter K corresponds to the preset total number of iterations of the optimization algorithm and is a hyperparameter.

[0048] Each stage (level) is the output state of the previous stage. And fixed and As input, then output the updated state. Each stage strictly follows the ADMM (Alternating Direction Multiplier Method) update sequence, sequentially calling specially designed learnable sub-network modules (IU module, DU module, MU module) to update reflectivity (and auxiliary variables), illumination, transmittance, and Lagrange multipliers, respectively. This sequential, non-parallel data flow design ensures that the network behavior is completely consistent with the mathematical logic of the underlying optimization algorithm.

[0049] go through m After iterative optimization at each stage, the network obtains the final optimized reflectivity features. This feature is then processed by a feature decoding module ( Figure 3 The upper part will be processed by this module. The spatial resolution is restored to the size of the original input image, and the number of channels is adjusted to the original number of channels to obtain the reconstructed reflectance image. .

[0050] Clearly, each stage is the core unit for implementing the algorithm of the network designed in this invention. It consists of three learnable functional modules with clear physical and mathematical correspondences, which together simulate a complete ADMM iteration. The following describes each learnable functional module.

[0051] (1) Image update module (IU module) This module is as follows Figure 4 As shown, Figure 4 This is a schematic diagram of the network architecture of the image update module. (a) shows the overall architecture, and (b) and (c) show the learnable components, respectively. and Network design; corresponding to the update of reflectivity feature R and auxiliary variable A. It is designed as a nested learnable optimizer, the core task of which is to learn and generate efficient semi-smooth Newton search directions. The IU module contains two key learnable components. and Networks, respectively corresponding to the aforementioned and The mathematical expression for .

[0052] refer to Figure 4 Here is a brief description of its implementation process: First, Learnable components with input parameters In the middle, via Output The result and input In the middle, at the same time enter In this process, the module outputs the optimal search direction. Thus, through this search direction according to the formula mentioned above Updated results for reflectivity components were obtained. .

[0053] For learnable components and Its internal workflow comparison and The computation is replaced by the MRB module in network design. Reflectivity characteristics Transform to feature domain The nonlinear operator defined in Implementation Example 1 is replaced by the CB module. .

[0054] In learnable components The network design process is as follows: Step 1: Reflectance characteristics Inputting into the MRB module yields the auxiliary variables after feature domain transformation. Then the auxiliary variable is multiplied by the Lagrange multiplier. Add; Step 2: Input the variables obtained in Step 1 into the CB module to obtain the nonlinear operator. ; Step 3: Subtract the variable obtained in Step 1 from the nonlinear operator obtained in Step 2, and input the result into the MRB module to obtain... The latter half ; Step 4: Perform arithmetic operations on the matrix pixels using the input reflectance features and some related variables to obtain... The first half ; Step 5: Add the results from steps 3 and 4 together to obtain the intermediate variable. .

[0055] In learnable components The network design process is as follows: Step 1: Input reflectivity components Inputting into the MRB module yields the auxiliary variables after feature domain transformation. Then add Lagrange multipliers The input is then fed into the CB module to obtain the nonlinear operator. ; Step 2: Transfer intermediate variables Inputting the data into the MRB module yields the variables after feature transformation. Then, with the identity matrix and nonlinear operators The difference is multiplied by matrix elements, and the result is input into the MRB module to transform it from features back to the spatial domain; Step 3: Transfer intermediate variables Perform matrix dot product with relevant variables, and then perform matrix addition with the spatial domain result obtained in the second step to obtain the search direction. .

[0056] (2) Degradation Parameter Update Module (DU Module) This module is as follows Figure 5 As shown, Figure 5 The diagram shows the network design for the degradation parameter update module. (a) is the overall flowchart, and (b) and (c) are the update modules for L and t, respectively, corresponding to the illumination components. and transmittance diagram The updates. They use the same network module design, by... The relevant variables are concatenated along the channel dimension to form a joint feature. This joint feature is then fed into a multi-domain residual block. The multi-domain residual block learns from the training data how to optimally estimate the updated illumination components that conform to physical priors such as smoothness, based on the current scene information. and transmittance diagram This completely replaces the need to manually design regularization terms and their proximal operators.

[0057] refer to Figure 5 Its workflow is as follows: First, input... Entering the light component The update module outputs the updated results of the illumination components. Then update the results with the output illumination components. Other relevant parameters are input into the transmittance map update module, and the output is the transmittance map update result. .

[0058] In the illumination component update module, the downsampled image is first input. And related parameters about light components Then, the two are concatenated along the channel dimension, and then the output is decomposed along the channel dimension by the MRB module. Two components, then... Perform matrix transpose and merge with Perform matrix multiplication and finally output the updated illumination components. .

[0059] In the transmittance map parameter update module, the downsampled image is first input. And related parameters about the transmittance map The subsequent processing is similar to that of the illumination component update module. The components are concatenated along the channel dimension and input into the MRB module, where the output is decomposed along the channel dimension. Two components, then transpose of components Matrix multiplication of the components yields the updated transmittance map. .

[0060] (3) Lagrange multiplier update module (MU module) This module corresponds to the multiplier update step, learning the multiplier update amount through a lightweight multi-residual block. The network uses the current multiplier features... and the latest reflectivity characteristics As input, it outputs an incremental feature, which is eventually updated to a new Lagrange multiplier. This allows the constraint strengthening process to adapt to the data distribution, increasing the overall flexibility of the algorithm.

[0061] The basic module design used in this invention is a multi-domain residual block (MRB), which has the ability to simultaneously capture local information and global context. Its main framework is as follows: Figure 6 As shown, Figure 6 The diagrams show the basic module design: (a) is the channel attention module, and (b) is the multi-domain residual module design. (1) Dual-path information fusion architecture The MRB (Multi-scale Residual Block) employs a dual-path parallel processing design: Path 1 is a spatial channel-aware path, focusing on local feature interactions and channel relationship modeling. Input features first pass through a channel attention module, generating channel weights through global average pooling and a bottleneck fully connected layer to recalibrate the features. Subsequently, a nonlinear transformation is performed through two 3x3 convolutional layers. Path 2 is a frequency-domain global modulation path, designed to efficiently capture long-range dependencies and global structure in the image. Input features are transformed to the frequency domain via Fast Fourier Transform (FFT), multiplied with a learnable complex weight matrix (achieving global linear modulation), then transformed back to the spatial domain via inverse FFT, and finally numerically constrained using the Tanh function.

[0062] (2) Adaptive feature aggregation and residual linking The outputs of the two pathways are fused using a data-dependent gating mechanism. Specifically, the maximum value of each channel in the feature map of pathway one is calculated and used as a dynamic scaling factor. The output of pathway two, after Tanh activation, interacts with this scaling factor to achieve adaptive weighting. Ultimately, the fusion features are: .

[0063] This mechanism allows the network to dynamically adjust the contribution strength of global information based on the saliency of local features. Finally, the fused features are added to the original input of the module through an identity residual connection to obtain the final output. This ensures gradient flow and stabilizes the training of deep networks.

[0064] In summary, the core idea of ​​this invention lies in systematically expanding and mapping the complete iterative process of the aforementioned optimization framework based on the semi-smooth Newton's method and the alternating direction multiplier method (ADMM) into a feedforward neural network with a clearly defined hierarchical structure. Each layer in the network is called a stage, strictly corresponding to one complete iteration cycle of the optimization algorithm. The network designed in this invention adopts the classic encoding-optimization-decoding architecture, whose forward propagation process clearly corresponds to the physical solution process. Furthermore, it replaces all the complex computational components in traditional optimization algorithms that require manual design and fixed parameters—including proximal operators, gradient calculations, and the solution of the semi-smooth Newton search direction—with learnable functional modules composed of parameterized neural networks. This design allows the entire recovery process to strictly adhere to the physical model and mathematical optimization logic of underwater imaging, while also adaptively learning the optimal recovery strategy through a data-driven approach. This results in excellent generalization ability and recovery performance while maintaining strong interpretability.

[0065] The underwater image sharpening method and system based on a deep unfolded neural network provided in this invention address the shortcomings of existing technologies by proposing an underwater image sharpening method based on a semi-smooth Newton's method. This method aims to systematically unfold the optimization process, which is based on an underwater physical model and solved using a semi-smooth Newton's method, into an end-to-end deep learnable network. This achieves efficient, high-precision, and highly generalizable underwater image restoration while maintaining strong physical interpretability. Specific beneficial effects include: An expandable and efficient physical optimization solution process was constructed: an optimization algorithm based on the semi-smooth Newton's method was designed to solve the non-smooth, nonlinear energy minimization problem derived jointly from the underwater imaging physical model and Retinex theory. This solution process has clear and fixed iterative steps, thus providing a stable and structured template for subsequent depth algorithm development.

[0066] A deep unfolded network architecture, strictly corresponding to the physical optimization process, was designed: each iterative calculation of the semi-smooth Newton method is precisely mapped and unfolded into a specific layer or module in deep learning, constructing an end-to-end model called the semi-smooth Newton unfolded network. In this architecture, components that require manual design in traditional optimization algorithms (such as gradient operators, proximal regularization operators, and step sizes) are replaced by learnable neural network modules, enabling the network to automatically learn the optimal "solution strategy" from the data.

[0067] A declarative rendering effect combining physical interpretability, high recovery quality, and strong generalization capability has been achieved: through the deep integration of the aforementioned "expandable optimization process" and "physicalization of network design," a declarative rendering system is ultimately obtained that can both follow the physical mechanisms of underwater degradation and adapt to data distribution. This system aims to output high-quality results with natural colors, clear details, and visual realism at near real-time speeds, and significantly improves generalization robustness to unknown underwater scenarios (different water qualities and lighting conditions), addressing the key pain point of weak generalization capability in existing end-to-end black-box models.

[0068] Example 3 This invention also provides an electronic device, such as... Figure 7 As shown, it includes a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304. Memory 303 is used to store computer programs; When processor 301 executes program 305 stored in memory 303, it performs the following steps: The input underwater blurred image is feature encoded to obtain a deep feature map, and several specific physical variables in the deep feature map are initialized to obtain initial physical variable values; Based on the initial physical variable values ​​and the deep feature map, a specific depth-unfolded neural network is used to iteratively optimize the solution of the preset augmented Lagrangian function using a target optimization algorithm to obtain the optimized reflectivity features; the preset augmented Lagrangian function is a function obtained after mathematically modeling and transforming the underwater blurred image; the deep unfolded neural network contains a specified number of layers, and each layer corresponds to one iterative optimization, the specified number being equal to the number of iterations; The spatial resolution of the optimized reflectance features is restored to the same size as the underwater blurred image, and the number of channels is adjusted to the number of channels corresponding to the underwater blurred image to obtain the optimized and reconstructed reflectance image.

[0069] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0070] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0071] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0072] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0073] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0074] For the electronic device / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant details can be found in the description of the method embodiments.

[0075] It should be noted that the electronic device and storage medium in the embodiments of the present invention are respectively electronic devices and storage media that apply the above-mentioned underwater image sharpening imaging method based on deep unfolded neural network. Therefore, all embodiments of the above-mentioned underwater image sharpening imaging method based on deep unfolded neural network are applicable to the electronic device and storage medium, and can achieve the same or similar beneficial effects.

[0076] The terminal device provided by the embodiments of the present invention can display proper nouns and / or fixed phrases for users to select, thereby reducing user input time and improving user experience.

[0077] This terminal device exists in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and are primarily designed to provide voice and data communication. These terminals include smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones.

[0078] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0079] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players (such as iPods), handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0080] (4) Other electronic devices with data interaction functions.

[0081] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0082] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0083] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0084] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects, all of which are collectively referred to herein as "modules" or "systems." Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The computer program may be stored / distributed in a suitable medium, provided with or as part of other hardware, or may take other distribution forms, such as via the Internet or other wired or wireless telecommunications systems.

[0085] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0088] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for enhancing underwater image clarity based on a deep unfolded neural network, characterized in that, The method includes: The input underwater blurred image is feature encoded to obtain a deep feature map, and several specific physical variables in the deep feature map are initialized to obtain initial physical variable values; Based on the initial physical variable values ​​and the deep feature map, a specific depth-unfolded neural network is used to iteratively optimize the solution of the preset augmented Lagrangian function using a target optimization algorithm to obtain the optimized reflectivity features; the preset augmented Lagrangian function is a function obtained after mathematically modeling and transforming the underwater blurred image; the deep unfolded neural network contains a specified number of layers, and each layer corresponds to one iterative optimization, the specified number being equal to the number of iterations; The spatial resolution of the optimized reflectance features is restored to the same size as the underwater blurred image, and the number of channels is adjusted to the number of channels corresponding to the underwater blurred image to obtain the optimized and reconstructed reflectance image.

2. The method according to claim 1, characterized in that, The physical variables include reflectance characteristics, Lagrange multipliers, auxiliary variables, illumination components, and transmittance maps. The mathematical modeling and transformation of the underwater blurred image includes: Based on the Jaffe-McGlamery model, scene irradiance is decomposed into illumination components according to Retinex theory. and reflectivity characteristics Establish a joint degradation model ;in, For underwater blurred images, This is a transmittance diagram. This is the global background light map for the blurred underwater image; Through formula Based on the aforementioned joint degradation model, an optimization problem is constructed that includes a data fidelity term and a regularization term; wherein, The characteristic transformation matrix, Yes The applied non-smooth regularization term, For balance parameters; The optimization problem is transformed into an augmented Lagrangian function: in, This represents the augmented Lagrange function. As an auxiliary variable, For Lagrange multipliers, This is a penalty factor.

3. The method according to claim 2, characterized in that, Based on the initial physical variable values ​​and the deep feature map, a neural network is unfolded at a specific depth, and an optimization algorithm is used to iteratively optimize the solution of the preset augmented Lagrangian function to obtain the optimized reflectivity features, including: By unfolding the first layer of the deep neural network, based on the initial illumination component, initial transmittance map, initial Lagrange multipliers, and the deep feature map, a variable optimization algorithm is used to perform the first iteration optimization on the auxiliary variables and reflectance features to obtain the auxiliary variables. and reflectivity characteristics ; Based on the initial reflectivity features, initial Lagrange multipliers, initial transmittance map, and the deep feature map, an illumination optimization algorithm is used to perform the first iteration optimization of the illumination components to obtain the illumination components. ; Based on the initial reflectivity features, initial illumination components, and the deep feature map, a transmittance optimization algorithm is used to perform the first iteration of transmittance optimization to obtain the transmittance map. ; Based on the initial reflectivity characteristics and initial auxiliary variables, the Lagrange multipliers are optimized in the first iteration using a coefficient optimization algorithm to obtain the Lagrange multipliers. ; The second layer of the deep neural network is used to expand the reflectivity feature. Lagrange multipliers Auxiliary variables Light Components Transmittance diagram And the deep feature map, and perform a second iteration of optimization using the target optimization algorithm; The first through deep unfolded neural network m Layer, based on the first m The physical variables optimized in the -1st iteration and the deep feature map are then used to execute the target optimization algorithm in the -1st iteration. m The next iteration optimizes the reflectivity characteristics. m ≥3.

4. The method according to claim 3, characterized in that, Based on the initial illumination component, initial transmittance map, initial Lagrange multiplier, and the deep feature map, a variable optimization algorithm is used to perform the first iteration optimization on the auxiliary variables and reflectance features to obtain the auxiliary variables. and reflectivity characteristics ,include: Through formula Taking the partial derivative of the preset augmented Lagrange function with respect to the auxiliary variable yields the optimality condition for updating the auxiliary variable; where α is the balance parameter. As a penalty factor, As an auxiliary variable, The characteristic transformation matrix, The initial reflectivity characteristics, These are the initial Lagrange multipliers; definition for In parameters The proximal operator below, obtain The update is explicitly expressed as: ; And through formula Based on the preset augmented Lagrange function, a target function for the reflectivity feature is constructed; wherein, Reflectivity characteristics This is the initial transmittance diagram. This is the initial illumination component. For the initial Lagrange multipliers, This is the deep feature map. This is the global background light map for the blurred underwater image. The characteristic transformation matrix, As initial auxiliary variables, As a penalty factor; Obtain the gradient of the objective function of the reflectivity feature, and define this gradient as a function. : And set it to zero; The function Equivalent transformation Through formula right Perform gradient calculation to obtain Explicit expression of updates: ,in, It is a proximal operator The generalized differential; Based on the above The update explicit expression is used to obtain the update direction of the reflectance feature, and the update expression of the reflectance feature is obtained: ,in Step size, k This represents the number of iterations.

5. The method according to claim 3, characterized in that, Based on the initial reflectivity features, initial Lagrange multipliers, initial transmittance map, and the deep feature map, the illumination components are optimized in the first iteration using an illumination optimization algorithm to obtain the illumination components. ,include: Through formula Based on the preset augmented Lagrange function, an objective function for the illumination component is constructed; wherein, For light component, The initial reflectivity characteristics, This is the initial transmittance diagram. This is the deep feature map. is the global background light map of the underwater blurred image, and β is the balance parameter; Obtain the gradient of the objective function of the illumination component to obtain the optimality condition for updating the illumination component: ; The optimality condition of the illumination component is abstracted into a regularization term. The relevant network modules obtain the update expressions for the illumination components: , k This represents the number of iterations.

6. The method according to claim 3, characterized in that, Based on the initial reflectivity features, initial illumination components, and the deep feature map, a transmittance optimization algorithm is used to perform the first iteration of optimization on the transmittance map to obtain the transmittance map. ,include: Through formula Based on the preset augmented Lagrangian function, the objective function of the transmittance map is constructed; wherein, This is a transmittance diagram. This is the initial illumination component. The initial reflectivity characteristics, This is the deep feature map. This is the global background light map of the underwater blurred image, where γ is the balance parameter; Obtain the gradient of the objective function of the transmittance map to obtain the optimality condition for updating the transmittance map: ; The optimality condition of the transmittance map is abstracted into a regularization term. The relevant network modules obtain the update expression for the transmittance map: , k This represents the number of iterations.

7. The method according to claim 3, characterized in that, Based on the initial reflectivity characteristics and initial auxiliary variables, the Lagrange multipliers are optimized in the first iteration using a coefficient optimization algorithm to obtain the Lagrange multipliers. ,include: Using formula Update the Lagrange multipliers; among them, For Lagrange multipliers, As a penalty factor, As initial auxiliary variables, The characteristic transformation matrix, The initial reflectivity characteristics, m This represents the number of iterations.

8. The method according to claim 1, characterized in that, Based on the initial physical variable values ​​and the deep feature map, a neural network is unfolded at a specific depth, and an optimization algorithm is used to iteratively optimize the solution of the preset augmented Lagrangian function to obtain the optimized reflectivity features, including: By unfolding a neural network at a specific depth and employing the alternating direction multiplier method framework, the solution of a predefined augmented Lagrange function is iteratively optimized using the following formula to obtain the optimized reflectivity characteristics: ； in, m For the number of iterations, Reflectivity characteristics This is a transmittance diagram. For light component, For Lagrange multipliers, As a penalty factor, The characteristic transformation matrix, These are initial auxiliary variables.

9. An underwater image sharpening imaging system based on a deep unfolded neural network, characterized in that, The system includes: The feature encoding and variable initialization module is used to encode the input underwater blurred image to obtain a deep feature map, and to initialize multiple specific physical variables in the deep feature map to obtain initial physical variable values. A deep unfolded neural network is used to perform iterative optimization of the solution of a preset augmented Lagrangian function based on the initial physical variable values ​​and the deep feature map, through a specific deep unfolded neural network and a target optimization algorithm, to obtain the optimized reflectivity features; the preset augmented Lagrangian function is a function obtained by mathematically modeling and transforming the underwater blurred image; the deep unfolded neural network contains a specified number of layers, each layer corresponding to one iterative optimization, and the specified number is equal to the number of iterations; The feature decoding and output module is used to restore the spatial resolution of the optimized reflectance features to the same size as the underwater blurred image, and adjust the number of channels to the number of channels corresponding to the underwater blurred image, so as to obtain the optimized and reconstructed reflectance image.

10. The system according to claim 9, characterized in that, The deep unfolded neural network contains a specified number of layers, and each layer includes an image update module, a degradation parameter update module, and a Lagrange multiplier update module; The image update module is used to update based on the first... m The illumination components, transmittance map, Lagrange multipliers, and deep feature map after the -1st iteration optimization are then used to optimize the auxiliary variables and reflectance features using a variable optimization algorithm. m The next iteration of optimization yields auxiliary variables. and reflectivity characteristics ; The degradation parameter update module is used to update the parameters based on the first parameter. m The reflectivity features, Lagrange multipliers, transmittance map, and deep feature map after the -1st iteration optimization are used to optimize the illumination components using an illumination optimization algorithm. m The next iteration of optimization yields the illumination components. ; and used based on the first m The reflectivity features and illumination components after the -1st iteration optimization, along with the deep feature map, are used to perform the 1st iteration optimization on the transmittance map. m Subsequent iterations of optimization yielded the transmittance map. ; The Lagrange multiplier update module is used to update based on the first... m The reflectivity characteristics and auxiliary variables after the -1st iteration optimization are used to optimize the Lagrange multipliers using a coefficient optimization algorithm. m The next iteration of optimization yields the Lagrange multipliers. ; in, m This represents the number of iterations.

Images