In-situ high-definition imaging method in turbid water environment based on hybrid feature fusion

Abstract

The present application relates to underwater observation and visual image processing technology, aiming to provide a kind of in-situ high-definition imaging method under turbid water environment based on hybrid feature fusion. Including: the construction CycleGAN network architecture's bidirectional cyclic antagonism system model, the model includes double generator, double discriminator and composite loss function module;By introducing physical turbidity distribution into spatial attention calculation, the hybrid fusion of water turbidity information and depth semantic features is realized;The model is trained, and the network parameter weight is updated by back propagation;The underwater turbid image to be processed is input into the trained model, and the high-definition clear underwater imaging result is output.The present application breaks through the technical bottleneck of blurred imaging, large noise and serious detail loss in low visibility water environment, and can simultaneously solve the problems of color distortion, uneven brightness and detail blur, improve detection accuracy, practical efficiency and reliability, effectively improve the operation perception ability of underwater robot.

Description

Technical Field

[0001] This invention belongs to the field of underwater observation and visual image processing technology, specifically relating to an in-situ high-definition imaging method for turbid water environments that integrates optical physical models and deep learning algorithms, and can be used to improve image clarity and color reproduction. Background Technology

[0002] The underwater environment is a crucial area for human exploration of nature and resource development. With the advancement of marine engineering, ecological monitoring, underwater exploration, intelligent equipment, and unmanned systems, underwater visualization technology is playing an increasingly prominent role in marine resource detection, environmental assessment, operational assistance, and disaster monitoring. However, in estuaries, coastal waters, lakes, and other turbid water bodies, the presence of suspended particles, sediment, and biological debris significantly enhances light scattering and absorption, resulting in extremely low visibility, severe color distortion, and loss of detail, posing a significant challenge to high-definition imaging and information analysis.

[0003] Traditional underwater imaging techniques typically rely on artificial light sources and physical optical model compensation to improve image quality. However, in turbid water environments, the nonlinear attenuation and complex scattering paths of light make image enhancement methods based on simple models unable to accurately recreate the real scene. While commonly used methods such as histogram equalization, Retinex enhancement, and physical dehazing can improve imaging results in some clear waters, their adaptability and imaging performance in turbid waters are poor. In recent years, with the rapid development of artificial intelligence, machine learning, and underwater imaging technologies, deep learning-based high-resolution imaging methods for turbid waters have gradually become a research hotspot in underwater exploration, marine science, and environmental monitoring.

[0004] With the continuous development of underwater imaging theory and artificial intelligence image processing technology, underwater image enhancement and restoration methods have been widely studied and applied in various fields such as marine exploration, environmental monitoring, biometrics, and underwater operations. Currently, there are relatively few high-definition imaging methods for turbid water environments, and these methods suffer from problems such as limited algorithmic diversity, poor adaptability, blurred imaging effects, and poor real-time performance. Efficient high-definition imaging methods for turbid water are still in the research and exploration stage, and mature technical implementation schemes for network structure design, training constraints, and performance optimization systems have not yet been formed. Meanwhile, existing underwater image enhancement technologies, such as those based on generative adversarial networks like CycleGAN, generally suffer from problems such as lack of physical constraints, poor adaptability to changes in turbidity and scattering intensity, susceptibility to color distortion and over-smoothing of details, and unstable generalization across different scenes. Physical model-based methods rely on simplified assumptions and parameter estimation, are easily affected by background light and transmittance estimation errors, and are prone to overcompensation, artifacts, and noise amplification under conditions such as non-uniform illumination and multiple suspended particles, making adaptive migration between different water body types difficult.

[0005] Deep learning technology, through models such as convolutional neural networks and generative adversarial networks, has achieved deep restoration of image details and colors, demonstrating powerful feature extraction and adaptive learning capabilities. In high-definition imaging, deep learning algorithms can integrate modules such as multi-scale feature extraction, attention mechanisms, and physical optics models, thereby effectively improving image clarity and information reconstruction quality. However, existing network structures suffer from high computational cost and poor real-time performance. Furthermore, most current models rely on large amounts of high-quality training data, and due to the scarcity of turbid water datasets, the generalization and reliability of these algorithms are insufficient, making it difficult to stably output high-definition imaging results in complex and dynamic environments. Currently, the application of deep learning in turbid water is still in its early stages, and its application in turbid water and marine environments has not yet been realized. Therefore, there is an urgent need for a systematic technical method for high-definition imaging of the optical characteristics of turbid water. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an in-situ high-definition imaging method for turbid water environments based on the fusion of hybrid features.

[0007] To solve the technical problem, the solution of the present invention is:

[0008] A high-resolution in-situ imaging method based on hybrid feature fusion in turbid water environments is provided, comprising the following steps:

[0009] (1) Collect water environment image samples and preprocess them to construct a dataset containing turbidity domain images and clear domain images;

[0010] (2) Construct a bidirectional recurrent adversarial system model based on the CycleGAN network architecture, including dual generators, dual discriminators, and a composite loss function module; wherein: the dual generators include generator A and generator B with U-Net network structures, both of which have symmetrically arranged multi-level encoder-decoder, and an RFB module is embedded in the bottleneck layer; at the U-Net skip connection of generator A, an improved CBAM attention module incorporating the UDCP physical model is embedded; at the U-Net skip connection of generator B, a standard CBAM attention module is embedded; by introducing the physical turbidity distribution map into the calculation of spatial attention, the hybrid fusion of water turbidity information and depth semantic features is achieved;

[0011] (3) The adversarial system model is trained using the training set and the validation set, and the network parameter weights are updated through backpropagation of the composite loss function to obtain the trained model.

[0012] (4) Input the underwater turbidity image to be processed into the trained model, and output a high-definition and clear underwater imaging result after processing by generator A.

[0013] The present invention further provides a computing device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions that are executed by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the aforementioned in-situ high-definition imaging method based on hybrid feature fusion in turbid water environments.

[0014] The present invention also provides a computer-readable storage medium storing computer instructions for causing the computer to execute the aforementioned in-situ high-definition imaging method based on hybrid feature fusion in turbid water environments.

[0015] Description of the invention principle:

[0016] Current underwater imaging technologies suffer from the following drawbacks: First, the light propagation is affected by suspended particles, silt, and impurities, resulting in generally low clarity and high noise levels, making it difficult to meet the requirements for accurate identification and positioning. Second, existing algorithms are mostly based on a single model, lacking the ability to adapt to changes in lighting, depth, and turbidity in complex environments, and are prone to color distortion and loss of detail when enhancing contrast. Furthermore, most deep learning-based imaging methods are computationally intensive and lack real-time performance, failing to meet the rapid response requirements of dynamic operational scenarios. In addition, the scarcity of high-quality training datasets related to turbid water bodies leads to insufficient generalization ability and inadequate performance validation of the algorithms.

[0017] To address the above technical problems, this invention provides an in-situ high-definition imaging method for turbid water environments based on adaptive fusion of hybrid features.

[0018] 1. Improvement and innovation of existing architecture

[0019] To address the issues of underwater high-definition imaging being susceptible to turbidity, lack of detail, and poor physical realism, a bidirectional recurrent adversarial system model was constructed, which includes a hybrid feature fusion mechanism, generator structure enhancement, discriminator multi-scale optimization, and composite loss function.

[0020] This invention is based on the existing CycleGAN dual generator and dual discriminator architecture, and selects the well-known U-Net as the dual generator backbone network. Combining the unique degradation mechanism of underwater turbidity imaging, the network structure is specifically improved to overcome the shortcomings of traditional general image conversion networks in underwater scenes with poor adaptability.

[0021] 2. Generator Differentiation Structure Design

[0022] Although both generators adopt the U-Net structure with symmetrical multilayer encoder-decoder, they employ differentiated attention configurations: Generator A, which is responsible for restoring murky images, has an improved CBAM attention module set at the U-Net skip connection; Generator B, which is responsible for reverse mapping, is configured with a standard CBAM module to achieve adaptive adjustment of features for different domain transformation tasks, which is different from the traditional dual generator design with completely identical structures.

[0023] By introducing the UDCP physical model, spatial attention can be made to explicitly perceive changes in turbidity distribution; combined with channel attention, cross-scale information enhancement is achieved, which effectively improves the physical consistency and specificity of feature extraction, making it more adaptable to imaging and capable of restoration in turbid water environments.

[0024] 3. Attention Innovation Combining Physics Priors and Deep Learning

[0025] In the spatial attention calculation of the improved CBAM, the existing UDCP underwater dark channel prior physical model is introduced. Based on the physical formula, a turbidity distribution map that can characterize the light attenuation intensity and suspended impurity concentration of the water body is calculated. The max pooling feature and average pooling feature of the intermediate features are concatenated with the turbidity distribution map and convolved together to generate spatial weights. This abandons the traditional spatial attention calculation method that only relies on deep semantic features. The underwater physical degradation law guides the feature weighting, which enhances the ability to preserve details and restore contrast in high turbidity areas.

[0026] 4. Enhanced design for multi-scale perception capabilities

[0027] By embedding an RFB multi-scale receptive field module in the bottleneck layer between the U-Net encoder and decoder, the feature receptive field is effectively expanded, fully capturing texture details and suspended particle interference features at different scales in underwater images, thereby improving the feature extraction and expression capabilities in complex and turbid scenes.

[0028] 5. Collaborative Optimization of Multi-Constraint Composite Loss Functions

[0029] A composite loss function is constructed, which includes adversarial loss, cyclic consistency loss, and physical guidance loss. Bidirectional adversarial constraints are achieved by relying on dual discriminators, and bidirectional cyclic mapping is used to construct a closed loop and calculate reconstruction error to ensure content consistency. Physical guidance loss is constructed by combining UDCP turbidity information. The network parameter update is constrained by multiple dimensions, which is adapted to the training scenario of unpaired underwater image data and realizes accurate cross-domain mapping between turbid and clear domains.

[0030] 6. Innovation in application scenarios and embedding methods

[0031] By embedding the improved attention module into the key positions of the U-Net skip connections, physical prior information is introduced during the fusion stage of shallow detail features and deep semantic features. This accurately solves the problems of reduced contrast and loss of details in underwater images caused by scattering of suspended particles and water attenuation, enabling in-situ high-definition imaging of turbid underwater water bodies with strong scene targeting.

[0032] In summary, this invention proposes a hybrid feature fusion mechanism based on UDCP physical constraints, combined with multi-scale feature extraction and discriminator adversarial optimization, forming a high-definition imaging method that combines physical interpretability and deep learning expressive capabilities. It not only theoretically achieves an effective combination of physical models and deep learning mechanisms, but also allows the model to converge stably in different water environments without paired training samples. Experiments demonstrate high feasibility, high accuracy, and superior imaging quality, and it possesses real-time inference capabilities, making it applicable to scenarios such as underwater robot vision enhancement and marine exploration imaging.

[0033] Compared with the prior art, the technical advantages of the present invention are:

[0034] 1. This invention proposes a high-definition in-situ imaging method based on the "self-adaptive fusion of hybrid features" mechanism for extreme environments of turbid water bodies. It breaks through the technical bottlenecks of blurred imaging, high noise and severe loss of details in low visibility water environments. It can also solve the problems of color distortion, uneven brightness and blurred details at the same time, improve detection accuracy, practical efficiency and reliability, and effectively enhance the operational perception capability of underwater robots.

[0035] 2. The turbidity applicable level of the method of the present invention is NTU 0~200, and the application medium can cover multiple water types such as seawater and freshwater. The maximum application water depth can reach 11,000 meters. It has wide applicability, the system model structure is small and lightweight, the overall imaging efficiency is improved by 65%, and the operation and maintenance cost is reduced by more than 50%.

[0036] 3. Compared with traditional physical model methods, the method of this invention can improve the underwater image quality by more than 30%, the peak signal-to-noise ratio (PSNR) by about 10%, and the structural similarity index (SSIM) by about 10%; at the same time, the edge details and texture contrast are significantly enhanced.

[0037] 4. This invention provides a solid foundation for exploring multidisciplinary applications, not only enhancing the operational perception capabilities of underwater robots but also broadening the application scope of machine vision underwater, and providing a reference for interdisciplinary collaborative operations. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the in-situ high-definition imaging method described in this invention.

[0039] Figure 2This is a schematic diagram of the overall network structure for in-situ high-definition imaging as described in this invention.

[0040] Figure 3 This is a schematic diagram of the network structure of generator A in a bidirectional cyclic adversarial system model.

[0041] Figure 4 This is a schematic diagram of the network structure of generator B in a bidirectional cyclic adversarial system model.

[0042] Figure 5 This is a schematic diagram of the encoding and decoding units in generator A. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments.

[0044] Part 1 Overview of the Implementation Scheme of this Invention

[0045] like Figure 1 and Figure 2 As shown, the in-situ high-definition imaging method for turbid water environments provided by this invention adopts a bidirectional cyclic adversarial system model. Based on a hybrid feature adaptive fusion mechanism, it constructs a multi-system fusion architecture that includes generator structure enhancement, discriminator multi-scale optimization, and composite loss function, enabling it to have stronger imaging adaptability and restoration capability in turbid water environments.

[0046] The bidirectional cyclic adversarial system model adopts an improved CycleGAN architecture, which includes two generators (A / B), two discriminators (D), and a composite loss function module, forming a bidirectional cyclic mapping structure to realize the mutual conversion between turbid and clear images.

[0047] (1) Generator

[0048] like Figure 3 and Figure 4As shown, the generator part includes Generator A and Generator B, both with a U-Net network structure, each featuring a symmetrically arranged multi-level encoder-decoder architecture and an embedded RFB module in the bottleneck layer. An improved CBAM attention module incorporating the UDCP physical model is embedded at the U-Net skip connections of Generator A; a standard CBAM attention module is embedded at the U-Net skip connections of Generator B. By incorporating the physical turbidity distribution map into the spatial attention calculation, a hybrid fusion of water turbidity information and depth semantic features is achieved. The two generators, employing a symmetrical structure, jointly constitute a bidirectional recurrent adversarial system framework, introducing a physical model-based hybrid feature fusion attention mechanism to achieve collaborative driving of physical constraints and depth features. Generator A is responsible for mapping from the turbidity domain to the clear domain, while Generator B is responsible for the inverse mapping from the clear domain to the turbidity domain to maintain recurrent consistency.

[0049] The dual generators construct a cyclically consistent mapping relationship during processing, forming a complete bidirectional mapping closed loop between generator A and generator B. This closed loop is achieved through two symmetrical cyclic paths: first, the turbid domain image is converted into a clear domain image by generator A, and then converted back into a reconstructed turbid domain image by generator B; second, the clear domain image is converted into a turbid domain image by generator B, and then converted back into a reconstructed clear domain image by generator A. By calculating the error between the original input image and the final reconstructed image in each cyclic path, the mapping accuracy of the dual generators is constrained, thereby achieving cross-domain image restoration in scenarios without paired data, adapting to the restoration requirements of underwater turbid images.

[0050] To fully integrate turbidity physical features with deep semantic features, this invention embeds an improved CBAM attention module at some U-Net skip connections in generator A, and introduces the UDCP (Underwater DarkChannel Prior) physical model into the spatial attention branch. By concatenating and fusing the Attention Map generated by standard CBAM with the turbidity distribution map D calculated by UDCP, a hybrid attention weight with both data semantics and physical constraints is generated, achieving a focused response to optical attenuation regions and significantly improving image detail restoration and noise suppression. Furthermore, by adding an RFB (Receptive Field Block) multi-scale feature extraction module to the bottleneck layer of the U-Net, multi-branch convolutions are combined with receptive fields of different dilation rates to achieve multi-scale context aggregation while preserving resolution, enhancing the generator's global structural understanding and local texture representation capabilities.

[0051] like Figure 5As shown, the encoder contains 5 levels of cascaded coding units, each of which consists of sub-units repeated twice; the sub-units include a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function layer connected in sequence; the decoder contains 5 levels of decoding units corresponding to the encoder, each of which includes an upconvolutional upsampling layer, a feature concatenation layer, a 3×3 convolutional layer, and a ReLU activation function layer; an improved CBAM attention module is embedded at the U-Net skip connections of levels 2 to 5 in generator A; a standard CBAM attention module is embedded at the U-Net skip connections of levels 2 to 5 in generator B.

[0052] (1.1) Hybrid Feature Fusion Attention Mechanism Based on Physical Model

[0053] To alleviate the problem of reduced contrast and loss of detail in underwater images caused by turbidity and scattering from suspended particles, this invention innovatively proposes to embed an improved CBAM (Convolutional Block Attention Module) attention module at the skip connections of U-Net. Simultaneously, the calculation results of the physical model UDCP (Underwater Dark Channel Prior) are introduced into the spatial attention calculation process. The turbidity distribution map D, which represents the turbidity of the water body and the intensity of light attenuation, is concatenated with the max-pooling and average-pooling features of the input intermediate features, and then convolved to generate spatial attention weights, thus achieving the fusion of physical prior information (water turbidity information) and deep learning features.

[0054] In the process of calculating channel attention, the input features Global average pooling and max pooling are performed separately, followed by a 1×1 convolution and a sigmoid function to generate channel attention weights. :

[0055]

[0056] in, It is the Sigmoid activation function. This is a 1×1 convolution operation. Indicates global average pooling. This indicates global max pooling.

[0057] For input intermediate features The system first performs global average pooling to obtain statistics reflecting the overall response of the channels, and then performs global max pooling to obtain statistics reflecting the local salient response. Together, these two measures constitute the channel-level semantic description. Next, a learnable linear mapping is performed using 1×1 convolution to model the correlation between channels and output the weight parameters of each channel. Finally, the weights are compressed to the 0 to 1 range using the Sigmoid function to form channel attention weights, which are used to weight the original features channel by channel.

[0058] Turbidity distribution map calculated using the UDCP physical model :

[0059]

[0060] in, For neighborhood windows Any pixel position within; In pixels A neighborhood window centered on the center; For the input image at pixels Color channel grayscale value; This represents the grayscale values ​​of the RGB three channels.

[0061] The UDCP physical model is an existing mature a priori physical model for underwater dark channels. It is a classic existing technique used in the fields of turbidity images and underwater images to characterize the degree of light attenuation and the distribution of impurity concentration in a region. To characterize the light attenuation intensity and impurity concentration in different regions, a pixel neighborhood minima constraint is used for solution, with a neighborhood window defined centered on pixel x. Iterate through each pixel and the grayscale values ​​of the RGB three channels within the window, and obtain the turbidity value at the corresponding position through double minimum value operation.

[0062] In the process of computing spatial attention, the physical turbidity distribution map D is compared with the intermediate features of the input spatial attention module. After concatenating the channels, the concatenated features are then convolved and mapped using a convolutional layer. Finally, an adaptive spatial attention weight is generated by applying a sigmoid activation function. This enables the effective fusion of physical prior information and deep learning features, enhancing the adaptive modulation capability of turbid region features.

[0063] Spatial attention weights The calculation formula is as follows:

[0064]

[0065] in, For intermediate features of the input; max( ) and avg( ) represent the maximum pooling feature and the average pooling feature of the intermediate feature, respectively; [ ] represents the channel splicing operation; D is the turbidity distribution map calculated by the UDCP physical model; This is a 7×7 convolution operation; σ is the Sigmoid activation function.

[0066] Existing CBAM spatial attention mechanisms have demonstrated that multiple two-dimensional feature maps with spatial representation capabilities can be concatenated and then used to generate spatial attention weights. In underwater turbidity imaging scenarios, the UDCP underwater dark channel prior model can estimate transmittance or degradation degree based on blue-green channel features, and the results can reflect the turbidity degree, light attenuation intensity, and scattering distribution at different spatial locations. Therefore, this invention proposes to use the turbidity distribution map calculated by UDCP... As a spatial prior map with physical meaning, and the max-pooling feature of intermediate features. and average pooling characteristics Spatial attention weights are generated by concatenating and mapping these weights together. This ensures that attention not only relies on the semantic responses learned by the network but is also guided by underwater physical degradation laws, thereby more effectively enhancing the feature representation of highly turbid, strongly scattering, and detail-vulnerable regions. Therefore, this formula is supported by the CBAM attention mechanism, the UDCP physical model, and underwater imaging degradation theory.

[0067] Finally, joint attention is applied, combining channel attention weights and spatial attention weights to the original intermediate features to achieve hybrid feature fusion.

[0068]

[0069] in, It is a hybrid feature that has been fused; These are intermediate features of the input; Channel attention weights; Spatial attention weights; This indicates an element-wise multiplication operation.

[0070] This mechanism enables the network to take into account both the physical laws of water optical degradation and the distribution of deep learning features when selecting features, thereby effectively suppressing noise and highlighting key structural regions.

[0071] (1.2) Generator A Structure and Calculation

[0072] like Figure 3 As shown, generator A is based on the U-Net network and consists of a multi-level encoder on the left and a multi-level decoder on the right. It is used to extract features and reconstruct high-definition images from the input murky image.

[0073] The encoder comprises five cascaded encoding blocks for progressive downsampling and feature extraction of the input underwater image. Each encoding block consists of two repeated sub-units with the following structure: a 3×3 convolutional layer → a batch normalization layer (BatchNorm) → a ReLU activation function layer, i.e., (Conv3×3→BatchNorm→ReLU)×2. This structure allows the network to progressively extract and enhance the semantic features of the image while maintaining feature stability.

[0074] The decoder block contains 5 decoding units corresponding to the encoder, used for feature upsampling and image reconstruction. The structure of each decoder block is as follows: Upsampling stage: Upconv2×2 (2×2 upconvolution) → BatchNorm → ReLU, realizing the spatial size restoration of low-resolution features; Feature fusion stage: Upsampled features and features of the corresponding layer of the encoder are fused through Concat (channel concatenation) operation, and then sequentially passed through Conv3×3 convolutional layer and ReLU activation function layer to complete feature fusion and nonlinear transformation.

[0075] A bottleneck layer is set between the encoder and the decoder, and an RFB (Receptive Field Block) module is embedded in this layer to enhance the multi-scale feature extraction capability, expand the receptive field, capture texture and detail information at different scales in underwater images, and provide richer contextual features for the decoder.

[0076] Multiple skip connections are set between the encoder and decoder of the U-Net network to achieve the fusion of shallow detail features and deep semantic features. Specifically, an improved CBAM attention module is embedded at the skip connections of levels 2-5. The features output by the encoder are processed by the improved CBAM module before being fed into the decoder for fusion. By introducing a turbidity distribution map calculated by the UDCP physical model, the focus on features in turbid areas is enhanced, alleviating the problems of decreased contrast and loss of detail in underwater images.

[0077] The specific implementation process of encoding and decoding is explained below:

[0078] An example of the original input image size is 256×256×3 (RGB), which is first normalized:

[0079]

[0080] in, These represent the mean and standard deviation of the input image, respectively. This represents the pixel intensity value of the original input image at spatial coordinates (x, y); This represents the pixel value after normalization.

[0081] The output feature map of the i-th layer of the encoder is denoted as The calculation is as follows:

[0082]

[0083] in, ReLU is a non-linear activation function. This represents the convolution operation; The first The convolution kernel weight matrix and bias terms of the layer; This represents the input feature mapping of the i-th layer.

[0084] By introducing the aforementioned improved CBAM module at the skip connections of layers 2 to 5 of the U-Net network in generator A, enhanced feature representation and detail preservation are achieved.

[0085] To enhance global and local context awareness, an RFB module is inserted into the intermediate bottleneck layer.

[0086] Let the input features be ,but:

[0087]

[0088] in, This represents the total number of branches in a multi-branch structure. The expansion rate in the Kth branch is... Hollow convolution kernel; The dilation rate of the dilated convolution (taken as 3, 5, 7, etc.); For weighting; Represents the nonlinear activation function ReLU; This represents the multi-scale fusion feature map generated by the RFB module.

[0089] The output features are then subjected to 1×1 convolution to achieve channel compression:

[0090]

[0091] in, This represents a 1×1 convolution operation; This indicates a convolution concatenation operation; This represents the aggregated multi-scale feature map.

[0092] This operation expands the effective receptive field while maintaining spatial resolution, enabling the fusion of local details and global semantic information.

[0093] The decoder performs deconvolution upsampling on the features of each layer to restore them to the original resolution.

[0094]

[0095] in, This represents the output mapping of the i-th layer of the decoder after feature processing; This represents the feature map that has been enhanced by the improved CBAM module and transmitted via a skip connection at the corresponding encoder level. Indicates the upsampling operator; This represents the fused feature map of the (i-1)th layer of the decoder.

[0096] The final clear image output is:

[0097]

[0098] in, This represents a 3×3 convolution operation; This represents the reconstructed feature map output from the final layer of the decoder; This represents the final, clear underwater image after deturbidity enhancement.

[0099] (1.3) Generator B Structure and Calculation

[0100] like Figure 4 As shown, Generator B is mainly used for cycle consistency constraints in unpaired training to ensure stable inter-domain mapping. Generator B is also based on U-Net and consists of a five-layer encoder and a five-layer decoder. Its structure is basically the same as Generator A, but a standard CBAM attention module is embedded at the U-Net skip connections from level 2 to level 5.

[0101] The specific application process is explained below:

[0102] An example of the original input image size is 256×256×3 (RGB), which is first normalized:

[0103]

[0104] in, This represents the pixel value at coordinates (x, y) of the original, sharp image input. This represents the mean intensity of the pixels in the current input image. This represents the standard deviation of the pixel intensity of the current input image. This represents the pixel value after normalization.

[0105] The encoder also uses a five-layer convolutional structure, and the calculation process is the same as that of generator A:

[0106]

[0107] in, Represents the nonlinear activation function ReLU; Indicates the first The convolution kernel weight matrix of the layer; This represents the output feature map of the i-th layer; This represents the input feature map of the i-th layer; Indicates the first Layer bias terms.

[0108] In this invention, a superscript "B" is added to some parameter symbols to distinguish between the two generators. This invention constructs a bidirectional cyclic generative adversarial network, comprising generator A and generator B, which are functionally symmetric but have different task directions. Although they are symmetric in network topology, their internally stored parameters are completely independent and not shared during model training and inference. To rigorously distinguish the computational processes of the two independent entities in the mathematical description of the technical solution, the superscript "B" is used for identification. Symbols without superscripts specifically refer to the corresponding parameters of generator A; while symbols with superscript "B" specifically refer to the corresponding parameters of generator B, and the same applies below.

[0109] In generator B, channel and spatial attention of CBAM are preserved to enhance feature selection, but UDCP physical features are not introduced; computation is based solely on the feature distribution.

[0110]

[0111]

[0112] in, This represents the channel attention weights learned by generator B; This represents the spatial attention weights learned by generator B; This represents the Sigmoid activation function; This represents a 1×1 convolution operation; This represents a 7×7 convolution operation; Indicates global average pooling; This indicates global max pooling; max( ) and avg( ) represent the max pooling feature and the average pooling feature, respectively; This represents the output feature map of the i-th layer.

[0113] The calculation formula corresponds to the channel attention in CBAM, which uses global average pooling and global max pooling to extract channel descriptors, and then generates channel weights through learnable mapping and sigmoid; where Convolution can be viewed as an equivalent convolutional implementation of the fully connected mapping in CBAM. The calculation formula corresponds to the spatial attention of CBAM, which is to obtain the spatial response map by performing max pooling and average pooling along the channel dimension, and then stitching it together. Convolution and sigmoid functions generate spatial attention weights. In generator B, both formulas employ the standard CBAM attention mechanism.

[0114] The output is fused as follows:

[0115]

[0116] in, This represents the enhanced feature map after dual attention-weighted calibration of channel and spatial dimensions; This represents element-wise multiplication; This represents the output feature map of the i-th layer.

[0117] The calculation formula originates from the standard feature recalibration method of "continuous weighting of channel attention and spatial attention" in the existing CBAM attention mechanism. Specifically, the channel attention weight... Spatial attention weights are used to characterize the importance of different feature channels. Both are used to characterize the importance of different spatial locations. After being normalized by the Sigmoid function, they can be used as gating weights, which are applied to the input features through element-wise multiplication. This enhances important channels and key spatial regions while suppressing redundant features. This process differs from CBAM, which first performs channel weighting. Then perform spatial weighting. The calculation methods are consistent, and after merging, they can be expressed as: Therefore, this formula has the theoretical basis of the CBAM attention mechanism and the principle of attention-gated feature recalibration.

[0118] The bottleneck layer employs a multi-scale convolutional structure to enhance the detailed simulation of domain transformations:

[0119]

[0120] in, This represents the total number of branches in a multi-branch structure. The expansion rate in the Kth branch is... Hollow convolution kernel; The dilation rate of the dilated convolution (taken as 3, 5, 7, etc.); For weighting; Represents the nonlinear activation function ReLU; This represents the multi-scale fused feature map generated by the RFB module; This indicates a convolution concatenation operation; This represents a 1×1 convolution operation; This represents the input features.

[0121] Decoding process:

[0122]

[0123] in, This represents the output mapping of the i-th layer of the decoder after feature processing; This represents the feature map that has been enhanced by the improved CBAM module and transmitted via a skip connection at the corresponding encoder level. Indicates the upsampling operator; This represents the fused feature map of the (i-1)th layer of the decoder.

[0124] Generate turbidity region image output:

[0125]

[0126] in, This represents a 3×3 convolution operation; This represents the reconstructed feature map output from the final layer of the decoder; This represents the final, clear underwater image after deturbidity enhancement.

[0127] (2) Discriminator

[0128] To improve the stability of the adversarial process and the realism of the generated results, this invention designs an innovative dual discriminator for a multi-scale PatchGAN network. The two discriminators correspond to two generators, used to distinguish between real images and the fake images output by the corresponding generators. Each discriminator consists of three sub-discriminators, which determine the authenticity of samples in parallel at the original size, downsampled by 1 / 2, and downsampled by 1 / 4 scales, achieving joint supervision from both local texture and global degradation perspectives. Each sub-discriminator employs a five-layer convolutional structure, and spectral normalization is introduced in each convolutional layer to replace the traditional normalization method, thereby mitigating gradient oscillations, improving model convergence stability, and enhancing the sensitivity of authenticity discrimination.

[0129] The quality of various distorted underwater images processed by the generator is perceived and judged by the discriminator, which in turn motivates the generator to produce high-quality images. The discriminator model adopts a three-scale structure: D1 (original size), D2 (downsampled by 1 / 2), and D3 (downsampled by 1 / 4). The input images are as follows: Each sub-discriminator contains five convolutional units, and the convolutional layers are processed using spectral normalization, as shown in the following formula:

[0130]

[0131] in, It is a random unit vector used to estimate the maximum singular value of the weights; This represents an approximate estimate of the spectral norm of the weight matrix W; This represents the original weight matrix of the convolutional layer; This represents the weight matrix after spectral normalization.

[0132] After this normalization, the single-layer output of the discriminator is:

[0133]

[0134] in, This represents the output prediction probability map of the j-th sub-discriminator; Represents a non-linear activation function; This represents a convolution operation unit with spectral normalization. Let I1 represent the input image at the j-th scale, where I1 is the original image, and I2 and I3 are the images after 1 / 2 downsampling and 1 / 4 downsampling, respectively.

[0135] The final discrimination score is:

[0136]

[0137] in, This represents the final comprehensive evaluation score of the discriminator for the input image I; This means summing up the predictions from the three sub-discriminators at different scales.

[0138] This multi-scale structure can both assess the realism of local textures and ensure the consistency of the global structure.

[0139] (3) Composite loss function

[0140] The composite loss function is the core control module that drives the training of the entire CycleGAN network and achieves parameter optimization. It has a one-to-one constraint relationship with each part of the network structure.

[0141] This invention employs a composite multi-objective optimization mechanism in its loss function system, specifically consisting of four parts: adversarial loss, cycle consistency loss, identity mapping loss, and physical model-assisted loss. The physical model-assisted loss utilizes UDCP to estimate transmittance and dark channel information, constraining the physical consistency of the generated image and making the deep network output more consistent with the optical attenuation characteristics of water.

[0142] The expression for the composite loss function is as follows:

[0143]

[0144] in, To combat the losses; This is a loss due to circular consistency. The loss is the identity mapping loss; To assist the loss in the physical model; - These are the weighting coefficients corresponding to each loss.

[0145] The calculation expressions for each sub-item are as follows:

[0146] (3.1) Countermeasures against losses:

[0147]

[0148] in, For clear domain true images; Input the image for the turbid region; This represents the average loss calculated over all samples in the dataset; This represents the score given by the discriminator to a real, clear image; This indicates that generator A is applied to the turbid image I. t The resulting clear image after processing; This represents the score given by the discriminator for generating a clear image.

[0149] (3.2) Cyclic consistency loss:

[0150]

[0151] in: () represents the murky image output by generator B; For clear domain true images; Input the image for the turbid region; This means that generator A converts a murky image into a clear one; Indicates that generator B will The generated clear image is then converted back to the turbid region, resulting in a reconstructed turbid image. This means that generator B will convert a clear image into a murky image; Indicates that generator A will The generated turbid image is then converted back to the sharp domain, resulting in a reconstructed sharp image. Let L1 norm be denoted as , which represents the pixel-level mean absolute error between two images.

[0152] (3.3) Identity mapping loss:

[0153]

[0154] in, For clear domain true images; Input the image for the turbid region; This means inputting a clear image into the generator used for dehazing. ; This means inputting an already murky image into the generator used for degradation. .

[0155] (3.4) Physical model-aided loss:

[0156]

[0157] Where D(x) is the depth distribution of the dark channel calculated from the input turbidity map; UDCP() represents the prior calculation operator for the underwater dark channel; Input the image for the turbid region; This represents the clear image output by generator A after dehazing.

[0158] Part Two: A Specific Implementation Case

[0159] (1) An image dataset was constructed based on publicly available underwater image data and turbid image samples collected under different turbidity levels, water depths, and lighting conditions. The data was then divided into two domains: a turbid domain and a clear domain, and size normalization and size adjustment were performed on each domain respectively. To ensure training effectiveness and model generalization ability, the dataset was divided into a training set, a validation set, and a test set in a ratio of 70%:15%:15%.

[0160] (2) Referring to the content of Part 1, construct a bidirectional cyclic adversarial system model of CycleGAN network architecture.

[0161] (3) The adversarial system model is trained using the training set and the validation set, and the network parameter weights are updated through backpropagation of the composite loss function to obtain the trained model;

[0162] Data input is loaded in batches, with a standard batch size of 1. Training uses the Adam optimizer (β1=0.5, β2=0.999), and the learning rate is set to... The decay strategy is linear descent. The total number of iterations is 100. It runs on an RTX 4060 GPU platform, using the PyTorch deep learning framework.

[0163] During training, the bidirectional recurrent adversarial system model processes the murky and sharp images in the following manner to achieve mutual conversion between the murky and sharp images:

[0164] Generator A processes a turbid image towards a clearer image. Its input is a turbid image, and its output is a pseudo-clear image, which is the restored image generated from the turbid image. Generator B processes a clear image towards a turbid image. Its input is a clear image, and its output is a pseudo-turbid image, which is the simulated turbid image generated from the clear image.

[0165] Two discriminators perform real and fake image discrimination calculations on the fake clear image and the real clear image output by generator A and the fake murky image and the real murky image output by generator B, respectively, to constrain the consistency between the distribution of the generator output image and the target domain image.

[0166] During training and image processing, generator A and generator B form a bidirectional cyclic consistency mapping closed loop; by calculating the error between the input image and the corresponding reconstructed image, cross-domain restoration of underwater turbid and clear images in unpaired data scenarios is achieved.

[0167] (4) Input the underwater turbidity image to be processed into the trained model, and output the underwater imaging result after processing by generator A. The output result is evaluated and confirmed by PSNR and SSIM indexes. It can quickly output high-definition imaging effect image in situ based on the input turbid water image, and the clarity, color authenticity and detail integrity are significantly improved.

[0168] Therefore, the method of the present invention can efficiently restore the true details of turbid visual images in turbid water environments, solve the problem of blurry and unstable imaging in complex lighting conditions and turbid waters in existing technologies, improve the clarity, real-time performance, reliability and detail fidelity of underwater imaging, and thus provide reliable technical support for visual inspection and operation in turbid waters.

[0169] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for in-situ high-resolution imaging in turbid water environments based on hybrid feature fusion, characterized in that, Includes the following steps: (1) Collect water environment image samples and preprocess them to construct a dataset containing turbidity domain images and clear domain images; (2) Construct a bidirectional recurrent adversarial system model of CycleGAN network architecture, including dual generators, dual discriminators and composite loss function modules; wherein: the dual generators include generator A and generator B with U-Net network structure, both of which have symmetrically arranged multi-level encoder-decoder, and embed RFB modules in the bottleneck layer; at the U-Net skip connection of generator A, an improved CBAM attention module with UDCP physical model is embedded; at the U-Net skip connection of generator B, a standard CBAM attention module is embedded. By incorporating physical turbidity distribution maps into spatial attention calculations, a hybrid fusion of water turbidity information and depth semantic features is achieved. (3) The adversarial system model is trained using the training set and the validation set, and the network parameter weights are updated through backpropagation of the composite loss function to obtain the trained model. (4) Input the underwater turbidity image to be processed into the trained model, and output a high-definition and clear underwater imaging result after processing by generator A.

2. The method according to claim 1, characterized in that, In step (1), the preprocessing of underwater turbid image samples includes at least normalization and resizing; the constructed dataset is further divided into training set, validation set and test set.

3. The method according to claim 1, characterized in that, In the U-Net network structure of the generator, the encoder contains 5 levels of cascaded coding units, each coding unit consisting of sub-units repeated twice; the sub-units include a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function layer connected in sequence; the decoder contains 5 levels of decoding units corresponding to the encoder, each decoding unit including an upconvolutional upsampling layer, a feature concatenation layer, a 3×3 convolutional layer, and a ReLU activation function layer; an improved CBAM attention module is embedded at the U-Net skip connections from level 2 to level 5 in generator A; a standard CBAM attention module is embedded at the U-Net skip connections from level 2 to level 5 in generator B.

4. The method according to claim 1, characterized in that, In the spatial attention calculation of the improved CBAM attention module, the calculation results of the UDCP physical model are introduced. The turbidity distribution map D, which represents the turbidity of the water body and the light attenuation intensity, is concatenated with the maximum pooling feature and the average pooling feature of the input intermediate feature and then convolved to generate spatial attention weights, thereby realizing the fusion of physical prior information and deep learning features. The formula for calculating the spatial attention weight is as follows: ； in, For intermediate features of the input; max( ) and avg( ) represent the maximum pooling feature and the average pooling feature of the intermediate feature, respectively; [ ] represents the channel splicing operation; D is the turbidity distribution map calculated by the UDCP physical model; This is a 7×7 convolution operation; σ is the Sigmoid activation function.

5. The method according to claim 1, characterized in that, The improved CBAM attention module is based on joint attention application computation, which combines channel attention weights and spatial attention weights to apply to the original features, thereby achieving hybrid feature fusion; specifically as shown in the following formula: ； in, It is a hybrid feature that has been fused; These are intermediate features of the input; Channel attention weights; Spatial attention weights; This indicates an element-wise multiplication operation.

6. The method according to claim 1, characterized in that, The dual discriminators are multi-scale PatchGAN discriminators that employ spectral normalization constraints and are used to distinguish between real images and fake images output by the corresponding generators. Each discriminator consists of three levels of sub-discriminators, and each sub-discriminator employs a five-layer convolutional structure.

7. The method according to claim 1, characterized in that, The expression for the composite loss function is as follows: ； in, To combat the losses; This is due to the loss of circular consistency. The loss is the identity mapping loss; To assist the physical model in loss; - These are the weighting coefficients corresponding to each loss.

8. The method according to claim 1, characterized in that, During training, the bidirectional recurrent adversarial system model processes murky and sharp images in the following manner to achieve mutual conversion between murky and sharp images: Generator A performs the processing of the turbid image towards the clear image direction. Its input is the turbid image and its output is the pseudo-clear image, that is, the restored image generated from the turbid image. Generator B performs the processing of the sharp domain image towards turbidity. Its input is the sharp domain image and its output is the pseudo-turbid domain image, that is, the simulated turbid image generated from the sharp image. Two discriminators perform real and fake image discrimination calculations on the fake clear image and the real clear image output by generator A and the fake murky image and the real murky image output by generator B, respectively, to constrain the consistency between the distribution of the generator output image and the target domain image. During training and image processing, generator A and generator B form a bidirectional cyclic consistency mapping closed loop; by calculating the error between the input image and the corresponding reconstructed image, cross-domain restoration of underwater turbid and clear images in unpaired data scenarios is achieved.

9. A computing device, characterized in that, include: At least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions that are executed by the at least one processor to cause the at least one processor to perform the in-situ high-definition imaging method for turbid water environments based on hybrid feature fusion as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the in-situ high-definition imaging method based on hybrid feature fusion in turbid water environments as described in any one of claims 1 to 8.

Images