A multi-sampling rate joint underwater image compressive sensing network

Abstract

This application relates to the fields of underwater image processing and compressed sensing technology, and in particular to a multi-sampling-rate joint underwater image compressed sensing network. It includes: a dual-branch dynamic sampling network that generates an adaptive sampling matrix based on the target sampling rate, underwater image, and depth map to achieve accurate capture of key information; a structure-guided dynamic reconstruction network that receives sampled data, depth map, and target sampling rate, and dynamically adjusts the reconstruction strategy using a structure-adaptive reconstruction module to complete image reconstruction; and a hierarchical loss function used to train the network, dynamically adjusting the weight coefficients of each loss term according to the target sampling rate to balance reconstruction requirements at different sampling rates. Through a unified architecture that adapts a single model to multiple sampling rates, it significantly reduces model training and deployment costs while comprehensively improving the quality and stability of underwater image reconstruction from extremely low to high sampling rates.

Description

Technical Field

[0001] This application relates to the fields of underwater image processing and compressed sensing technology, and in particular to an underwater image compressed sensing network with multiple sampling rates. Background Technology

[0002] With the rapid development of underwater-related industries such as marine resource exploration and ecological monitoring, researchers have placed higher demands on the quality of underwater images, which are one of the core information carriers for underwater research. However, the underwater environment causes image degradation due to light absorption and scattering, and underwater communication relies on low-bandwidth acoustic signals, posing significant challenges to the transmission of high-definition image data.

[0003] Compressed sensing technology, by overcoming the limitations of the Nyquist sampling theorem and utilizing the sparsity of signals in specific domains, achieves accurate signal reconstruction from a small amount of sampled data. This effectively reduces the hardware and bandwidth requirements for data acquisition, storage, and transmission, and has significant application value in the field of underwater image acquisition, providing a new approach for efficient underwater image acquisition and transmission. However, research on underwater compressed sensing algorithms is still in its early stages, with few mature algorithms available; most are simply direct transfers of land-based algorithms to underwater applications. Existing land-based compressed sensing algorithms are mainly divided into two categories: single-model single-sampling-rate algorithms and single-model multi-sampling-rate algorithms. While the former offers a certain level of reconstruction accuracy, it requires separate training and deployment of independent models for different sampling rates, increasing hardware and deployment costs in practical applications. The latter, while enabling a single model to adapt to multiple sampling rates, cannot achieve good performance across all sampling rates due to the inherent differences between low-sampling-rate and high-sampling-rate reconstruction tasks, especially performing poorly at extremely low sampling rates, which are predominantly encountered in underwater operations. At the same time, since it is just a simple transfer of the land-based algorithm, their performance drops sharply at extremely low sampling rates, and the acquired images cannot meet the observation requirements. Summary of the Invention

[0004] This invention aims to provide a multi-sampling-rate joint underwater image compressed sensing network, enabling a single model to adapt to multiple sampling rates and addressing the high training and deployment costs of existing algorithms. Furthermore, this invention incorporates the unique characteristics of underwater scenarios into the algorithm design, resulting in better task scenario adaptation. It leverages the features of underwater images to optimize sampling and reconstruction performance, thereby addressing the pain point of current algorithms performing poorly at extremely low sampling rates.

[0005] In some embodiments of this application, a multi-sampling-rate joint underwater image compressed sensing network is provided, including: A dual-branch dynamic sampling network is used to generate a sampling matrix based on the target sampling rate, the underwater image, and the depth map of the underwater image, and to sample the underwater image to obtain sampling data. A structure-guided dynamic reconstruction network is used to receive the sampled data, the depth map and the target sampling rate, dynamically adjust the reconstruction strategy, perform image reconstruction and output the reconstructed underwater image. The depth map is a scene depth information map corresponding to the underwater image; the dual-branch dynamic sampling network and the structure-guided dynamic reconstruction network are integrated into a single neural network model and are adaptable to underwater image compression sensing tasks at various sampling rates.

[0006] In some embodiments of this application, a dual-branch dynamic sampling network is used to generate a sampling matrix based on a target sampling rate, an underwater image, and a depth map of the underwater image, and to sample the underwater image to obtain sampling data, including: The underwater image and its corresponding depth map are received, and based on the target sampling rate, the underwater image is analyzed for content features to generate an attention weight mask at the current sampling rate, and the depth map is analyzed for structural features to generate a structural attention weight mask. The content attention weight mask and the structure attention weight mask are dynamically weighted and fused to generate an image feature sampling priority mask that matches the target sampling rate. The image feature sampling priority mask is binarized to generate a binary sampling matrix; The underwater image is sampled using the binary sampling matrix to obtain the sampled data.

[0007] In some embodiments of this application, the underwater image and its corresponding depth map are received, and based on the target sampling rate, content feature analysis is performed on the underwater image to generate an attention weight mask at the current sampling rate. When performing structural feature analysis on the depth map to generate a structural attention weight mask, the process includes: The dual-branch dynamic sampling network includes a content feature analysis branch and a structural feature analysis branch, which are used to perform the content feature analysis and structural feature analysis. The dual-branch dynamic sampling network further includes a weight generation network, which is used to convert the target sampling rate into two learnable and adjustable parameters, which are respectively used as the fusion weight coefficients of the content attention weight mask and the structure attention weight mask. The content feature analysis branch performs feature analysis and weight mapping on the underwater image to generate the content attention weight mask, and the structure feature analysis branch performs feature analysis and weight mapping on the depth map to generate the structure attention weight mask.

[0008] In some embodiments of this application, when dynamically weighting and fusing the content attention weight mask and the structure attention weight mask to generate an image feature sampling priority mask that matches the target sampling rate, the following steps are included: The two learnable and adjustable parameters obtained from the weight generation network are used as the fusion weight coefficients of the content attention weight mask and the structure attention weight mask, respectively. Based on the target sampling rate, the proportion of the two fusion weight coefficients is dynamically adjusted. At a low sampling rate, the proportion of the fusion weight coefficient corresponding to the structural attention weight mask is increased, and at a high sampling rate, the proportion of the fusion weight coefficient corresponding to the content attention weight mask is increased. The content attention weight mask and the structure attention weight mask are multiplied by their respective fusion weight coefficients and then superimposed to generate an image feature sampling priority mask that matches the target sampling rate.

[0009] In some embodiments of this application, when binarizing the image feature sampling priority mask to generate a binary sampling matrix, the following steps are included: Based on the sampling ratio indicated by the target sampling rate, a corresponding number of high-priority positions are selected as sampling points from the image feature sampling priority mask; The position corresponding to the sampling point is assigned a value of 1, and the other positions are assigned a value of 0, generating a binary sampling matrix with the same spatial dimension as the input underwater image.

[0010] In some embodiments of this application, the structure-guided dynamic reconstruction network, used to receive the sampled data, the depth map, and the target sampling rate, dynamically adjust the reconstruction strategy, perform image reconstruction, and output the reconstructed underwater image, includes: The structure-guided dynamic reconstruction network includes multiple cascaded structure-adaptive reconstruction modules. Each structure-adaptive reconstruction module receives the current feature map, the depth map, and the target sampling rate, and includes a sampling rate condition block and a dynamic content construction module. The sampling rate conditional block is used to generate weight adjustment coefficient ω and bias adjustment coefficient γ by performing an affine transformation on the target sampling rate, and to embed ω and γ into the input features to obtain a sampling rate conditional feature map; The dynamic content construction module is used to generate a dynamically dilated convolution using the image structure information reflected by the depth map, perform feature modeling on the sampling rate conditional feature map, and obtain an updated feature map as the input of the next module; Through cascaded iterative processing of multiple structure-adaptive reconstruction modules, the reconstructed underwater image is finally output.

[0011] In some embodiments of this application, the sampling rate conditional block, used to generate weight adjustment coefficients ω and bias adjustment coefficients γ by performing an affine transformation on the target sampling rate, and embedding ω and γ into the input features to obtain a sampling rate conditional feature map, includes: The target sampling rate is input into two independent fully connected networks; Apply the Softplus activation function to the output of the first fully connected network to obtain the weight adjustment coefficients ω; Applying the ReLU activation function to the output of the second fully connected network yields the bias adjustment coefficient γ; The input features are convolutionally processed to obtain convolutional features; The product of the convolutional feature and the weight adjustment coefficient ω is calculated, and then the bias adjustment coefficient γ is added to obtain the sampling rate conditional feature map.

[0012] In some embodiments of this application, the dynamic content construction module is used to generate a dynamically dilated convolution using the image structure information reflected by the depth map, to perform feature modeling on the sampling rate conditional feature map, and to obtain an updated feature map, which, when used as input to the next module, includes: Based on the depth map, identify the feature-dense and feature-sparse regions in the sampling rate condition feature map; Feature extraction is performed using small receptive field convolution in the feature-dense regions; Large receptive field convolution is used to extract features and model associations in the sparse feature regions. The features processed by convolution with different receptive fields are aggregated to obtain the updated feature map, which is then used as the input for the next module.

[0013] Some embodiments of this application also include: The neural network model is trained using a hierarchical loss function; The hierarchical loss function is a weighted sum of mean squared error loss, VGG perception loss, and style loss; Furthermore, the weighting coefficients in the weighted sum are dynamically adjusted according to the range of the target sampling rate.

[0014] In some embodiments of this application, when the weighting coefficients in the weighted sum are dynamically adjusted according to the range of the target sampling rate, the following methods are included: The dynamic adjustment of the weighting coefficients follows these rules: As the target sampling rate decreases, the weighting coefficient of the style loss increases accordingly; As the target sampling rate increases, the weighting coefficient of the mean square error loss increases accordingly.

[0015] Compared with existing technologies, the underwater image compressed sensing network with unified sampling rate provided in this application has the following advantages: By adopting a unified architecture design with a single model, it is possible to adapt to underwater image compression sensing tasks under various sampling rates. There is no need to train and store independent models for different sampling rates, which fundamentally solves the problem of high training and deployment costs of existing solutions, significantly reduces the hardware burden, and is more in line with the actual application needs of underwater detection equipment with limited resources and dynamic changes in sampling rates.

[0016] By co-optimizing the dual-branch dynamic sampling mechanism and the hierarchical loss function, the reconstruction objectives under different sampling rates are precisely balanced: at low sampling rates, priority is given to ensuring the overall structure and color consistency of the image, while at high sampling rates, the focus is on repairing texture details and pixel accuracy. This achieves stable and superior reconstruction performance across the entire sampling rate range, effectively solving the pain point of existing methods where performance drops sharply at extremely low sampling rates. Attached Figure Description

[0017] Figure 1 This is a diagram illustrating the overall algorithm structure of a multi-sampling-rate unified underwater image compression sensing algorithm in a preferred embodiment of this application.

[0018] Figure 2 This is a structural diagram of the dual-branch dynamic sampling network in a preferred embodiment of this application.

[0019] Figure 3 This is a structural diagram of the structure-guided dynamic reconstruction network in a preferred embodiment of this application. Detailed Implementation

[0020] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0021] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0022] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0023] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0024] like Figure 1 As shown, a preferred embodiment of this application discloses a multi-sampling-rate joint underwater image compressed sensing network, comprising: A dual-branch dynamic sampling network is used to generate a sampling matrix based on the target sampling rate, the underwater image, and the depth map of the underwater image, and to sample the underwater image to obtain sampling data. A structure-guided dynamic reconstruction network is used to receive the sampled data, the depth map and the target sampling rate, dynamically adjust the reconstruction strategy, perform image reconstruction and output the reconstructed underwater image. The depth map is a scene depth information map corresponding to the underwater image; the dual-branch dynamic sampling network and the structure-guided dynamic reconstruction network are integrated into a single neural network model and are adaptable to underwater image compression sensing tasks at various sampling rates.

[0025] In some embodiments of this application, a dual-branch dynamic sampling network is used to generate a sampling matrix based on a target sampling rate, an underwater image, and a depth map of the underwater image, and to sample the underwater image to obtain sampling data, including: The underwater image and its corresponding depth map are received, and based on the target sampling rate, the underwater image is analyzed for content features to generate a content attention weight mask, and the depth map is analyzed for structural features to generate a structural attention weight mask. The content attention weight mask and the structure attention weight mask are dynamically weighted and fused to generate an image feature sampling priority mask that matches the target sampling rate. The image feature sampling priority mask is binarized to generate a binary sampling matrix; The underwater image is sampled using the binary sampling matrix to obtain the sampled data.

[0026] In some embodiments of this application, receiving the underwater image and its corresponding depth map, and generating a content attention weight mask by performing content feature analysis on the underwater image based on a target sampling rate, and generating a structure attention weight mask by performing structural feature analysis on the depth map, includes: The dual-branch dynamic sampling network includes a content feature analysis branch and a structural feature analysis branch, which are used to perform the content feature analysis and structural feature analysis. The dual-branch dynamic sampling network further includes a weight generation network, which is used to convert the target sampling rate into two learnable and adjustable parameters, which are respectively used as the fusion weight coefficients of the content attention weight mask and the structure attention weight mask. The content feature analysis branch performs feature analysis and weight mapping on the underwater image to generate the content attention weight mask, and the structure feature analysis branch performs feature analysis and weight mapping on the depth map to generate the structure attention weight mask.

[0027] In this embodiment, the content feature analysis branch takes underwater images as input. This branch parses the pixel-level information of the image, extracts and evaluates features related to the importance of visual content. These features include, but are not limited to, texture details of objects, edge contours, and areas of local contrast variation caused by underwater optical effects. Through in-depth analysis of these features, a content attention weight mask is output. In this mask, the numerical value at each position represents the relative importance of the visual content contained at that position during the sampling process; a higher value means that it needs to be sampled more preferentially.

[0028] In this embodiment, the structural feature analysis branch takes the depth map as input. This branch's function is to analyze the three-dimensional spatial structure information of the scene. The depth map reflects the distance from each point in the scene to the imaging plane, directly related to the sharpness attenuation model of underwater imaging; the greater the distance, the more severe the signal attenuation and scattering, and the more blurred the image. This branch evaluates the importance of different spatial locations in maintaining the overall structural integrity of the image by analyzing structural information such as distance distribution and depth steps in the depth map. Finally, it outputs a structural attention weight mask, where higher values ​​correspond to foreground objects, sharp boundaries, or structural regions important for understanding the scene layout.

[0029] In this embodiment, the weight generation network is a lightweight parameter mapping module whose input is a scalar target sampling rate. In the specific workflow, the two learnable and adjustable parameters generated by the weight generation network are respectively input to the content feature analysis branch and the structural feature analysis branch. Each branch incorporates these adjustment parameters during its internal feature extraction and weight mapping calculations. Ultimately, the content feature analysis branch and the structural feature analysis branch can dynamically adjust their internal analysis strategies based on the input target sampling rate. Feature analysis and weight mapping are performed on the underwater image and depth map respectively, thereby generating dynamically changing content attention weight masks and structural attention weight masks that match the current sampling rate requirements.

[0030] In some embodiments of this application, when dynamically weighting and fusing the content attention weight mask and the structure attention weight mask to generate an image feature sampling priority mask that matches the target sampling rate, the following steps are included: The two learnable and adjustable parameters obtained from the weight generation network are used as the fusion weight coefficients of the content attention weight mask and the structure attention weight mask, respectively. Based on the target sampling rate, the proportion of the two fusion weight coefficients is dynamically adjusted. At a low sampling rate, the proportion of the fusion weight coefficient corresponding to the structural attention weight mask is increased, and at a high sampling rate, the proportion of the fusion weight coefficient corresponding to the content attention weight mask is increased. The content attention weight mask and the structure attention weight mask are multiplied by their respective fusion weight coefficients and then superimposed to generate an image feature sampling priority mask that matches the target sampling rate.

[0031] In this embodiment, the weight coefficients required for the fusion process are two learnable and adjustable parameters generated by the weight generation network based on the target sampling rate. These two parameters are directly used as the fusion weight coefficients for the content attention weight mask and the structural attention weight mask, respectively. This ensures that the entire process from branch analysis to mask fusion is controlled by the same set of conditional parameters derived from the target sampling rate, guaranteeing the consistency of the system logic.

[0032] In this embodiment, the fusion weight coefficients are dynamically adjusted based on the target sampling rate. This is a pattern automatically learned by the network during training. At low or even extremely low sampling rates, the available measurement data for reconstruction is very limited. In this case, ensuring the overall structural correctness and spatial layout rationality of the reconstructed image is more critical than restoring subtle textures. Therefore, the network adaptively increases the proportion of fusion weight coefficients corresponding to the structure-focused weight mask, making the fused priority mask more inclined to retain information from regions crucial for maintaining the scene structure. At high sampling rates, there is already relatively sufficient measurement data to support the reconstruction of the overall structure. The focus then shifts to restoring image details, improving texture quality, and correcting color shifts. Therefore, the network adaptively increases the proportion of fusion weight coefficients corresponding to the content-focused weight mask, making the fused priority mask more inclined to retain regions with rich textures, dense details, or critical color information.

[0033] In this embodiment, after determining the dynamic fusion weight coefficients, a specific fusion calculation is performed. The content attention weight mask is multiplied element-wise by its corresponding fusion weight coefficient; the structural attention weight mask is also multiplied element-wise by its corresponding fusion weight coefficient. These two weighted masks are then superimposed to generate the final image feature sampling priority mask.

[0034] In some embodiments of this application, when binarizing the image feature sampling priority mask to generate a binary sampling matrix, the following steps are included: Based on the sampling ratio indicated by the target sampling rate, a corresponding number of high-priority positions are selected as sampling points from the image feature sampling priority mask; The position corresponding to the sampling point is assigned a value of 1, and the other positions are assigned a value of 0, generating a binary sampling matrix with the same spatial dimension as the input underwater image.

[0035] In this embodiment, after obtaining the image feature sampling priority mask, it needs to be converted into a binary sampling matrix. This process is completed by the binarizer built into the dual-branch dynamic sampling network. Specifically, the binarization process includes the following steps: First, perform binarization processing. The core function of the binarizer is to perform a mapping from continuous priority to discrete sampling decisions. Its input is the image feature sampling priority mask generated in the previous step, denoted as M. The value M(i,j) at each position (i,j) in the mask represents the comprehensive sampling priority score at that location. The binarizer converts the continuous priority scores into binary decisions according to a global threshold or selection rule determined by the target sampling rate. Among all H×W positions (H and W are the height and width of the image, respectively), only the top K positions with the highest priority are retained, and the remaining positions are discarded. The value of K is directly calculated from the target sampling rate: K=round(SR×H×W). After this step, an intermediate binary image is generated, where the selected K positions are marked as 1 to indicate "sampling", and the remaining positions are marked as 0 to indicate "no sampling".

[0036] The second step involves generating a spatially consistent binary sampling matrix. Based on the sampling ratio indicated by the target sampling rate, the system determines the number K of pixel locations to be sampled. The binarizer selects the highest priority locations corresponding to the ratio from the image feature sampling priority mask. This ultimately generates a binary sampling matrix with the same spatial dimensions as the input underwater image. This matrix Φ is a matrix of size H×W, where its elements are 1 only at the K selected high-priority locations; and 0 at all other locations.

[0037] In some embodiments of this application, the structure-guided dynamic reconstruction network, used to receive the sampled data, the depth map, and the target sampling rate, dynamically adjust the reconstruction strategy, perform image reconstruction, and output the reconstructed underwater image, includes: The structure-guided dynamic reconstruction network includes multiple cascaded structure-adaptive reconstruction modules. Each structure-adaptive reconstruction module receives the current feature map, the depth map, and the target sampling rate, and includes a sampling rate condition block and a dynamic content construction module. The sampling rate conditional block is used to generate weight adjustment coefficient ω and bias adjustment coefficient γ by performing an affine transformation on the target sampling rate, and to embed ω and γ into the input features to obtain a sampling rate conditional feature map; The dynamic content construction module is used to generate a dynamically dilated convolution using the image structure information reflected by the depth map, perform feature modeling on the sampling rate conditional feature map, and obtain an updated feature map as the input of the next module; Through cascaded iterative processing of multiple structure-adaptive reconstruction modules, the reconstructed underwater image is finally output.

[0038] In some embodiments of this application, the sampling rate conditional block, used to convert the target sampling rate into learnable weight adjustment coefficients ω and bias adjustment coefficients γ, and to embed ω and γ into the input features through affine transformation to obtain the sampling rate conditional feature map, includes: The target sampling rate is input into two independent fully connected networks; Apply the Softplus activation function to the output of the first fully connected network to obtain the weight adjustment coefficients ω; Applying the ReLU activation function to the output of the second fully connected network yields the bias adjustment coefficient γ; The input features are convolutionally processed to obtain convolutional features; The product of the convolutional feature and the weight adjustment coefficient ω is calculated, and then the bias adjustment coefficient γ is added to obtain the sampling rate conditional feature map.

[0039] In this embodiment, the sampling rate conditional block effectively transforms the target sampling rate information into learnable conditional parameters that can modulate the internal features of the network, and embeds them into the calculation of the feature stream, thereby enabling the network to adaptively and dynamically adjust to reconstruction tasks with different sampling rates. Specifically, this transformation and embedding process includes the following steps in sequence: Step 1: Input the target sampling rate in scalar form into two independent fully connected networks. These two networks have simple structures and few parameters, and are specifically designed to map a single sampling rate value into adjustable parameters with rich representational capabilities. Using two independent networks allows the networks to learn independently how to adjust the feature weights and biases according to the sampling rate, providing more flexible and fine-grained control.

[0040] Step 2: Apply the Softplus activation function to the output of the first fully connected network to obtain the weight adjustment coefficients ω. The Softplus function produces a consistently positive and smoothly varying output, ensuring that the weight coefficients ω used for scaling features are positive and avoiding feature confusion that might result from sign flipping. Its smoothness also contributes to training stability. Apply the ReLU activation function to the output of the second fully connected network to obtain the bias adjustment coefficients γ. The ReLU function produces a non-negative output, providing a bias adjustment to the feature map that is always positive or zero. This design allows the network to selectively enhance or leave the feature map unchanged depending on the sampling rate, which helps the model flexibly adjust the activation level of features at different sampling rates.

[0041]

[0042] in, and For activation function, This indicates the sampling rate.

[0043] Step 3: While generating adjustment coefficients in parallel, perform a standard convolution operation on the input feature map. The purpose is to extract and transform spatial and channel-specific feature information from the output of the previous layer, generating an intermediate convolutional feature. This convolution operation contains a learnable convolutional kernel, the parameters of which are determined during training.

[0044] Step 4: Calculate the element-wise product of the intermediate convolutional feature Conv(F) and the weight adjustment coefficient ω, and then add the bias adjustment coefficient γ.

[0045] In some embodiments of this application, the dynamic content construction module is used to generate a dynamically dilated convolution using the image structure information reflected by the depth map, to perform feature modeling on the sampling rate conditional feature map, and to obtain an updated feature map, which, when used as input to the next module, includes: Based on the depth map, identify the feature-dense and feature-sparse regions in the sampling rate condition feature map; Feature extraction is performed using small receptive field convolution in the feature-dense regions; Large receptive field convolution is used to extract features and model associations in the sparse feature regions. The features processed by convolution with different receptive fields are aggregated to obtain the updated feature map, which is then used as the input for the next module.

[0046] In some embodiments of this application, the dynamic content construction module is used to generate a dynamically dilated convolution using the image structure information reflected by the depth map, to perform feature modeling on the sampling rate conditional feature map, and to obtain an updated feature map, which, when used as input to the next module, includes: Based on the depth map, identify the feature-dense and feature-sparse regions in the sampling rate condition feature map; Small receptive field convolution is used to perform local fine feature extraction and detail enhancement in the feature-dense regions; The large receptive field convolution is used to perform wide-area context feature extraction and structural association modeling on the sparse feature regions. The large receptive field convolution includes dilated convolution or large-size convolution kernel. The features processed by convolution with different receptive fields are aggregated to obtain an updated feature map that integrates local details and global structural information, which serves as the input for the next module.

[0047] In this embodiment, the dynamic content building module performs differentiated dynamic convolution modeling on regions with different degrees of degradation in the image based on the scene structure information revealed by the depth map. Specifically, this process includes the following steps in logical order: Step 1: The module receives the sampling rate conditional feature map from the sampling rate conditional block and the input depth map. The depth map reflects the relative distances of pixels in the scene. Based on the depth information, the dynamic content building module first divides the feature map into regions, identifying two types of regions with significant differences in feature distribution and modeling requirements. **Feature-dense regions:** Corresponding to near-field objects in the depth map. These regions are relatively less affected by underwater scattering and attenuation due to their proximity to the imaging device, and details in the original image are relatively well preserved. Therefore, they appear as regions with rich texture, clear edges, and strong feature responses in the feature map. **Feature-sparse regions:** Corresponding to distant or background water bodies in the depth map. These regions are severely affected by underwater optical effects, resulting in significant image information degradation and a large loss of details. Therefore, they appear as regions with smooth textures, weak feature responses, and scarce information in the feature map.

[0048] Step 2: After completing region recognition, dynamic convolution operations are generated and applied based on region characteristics. For feature-dense regions: Small receptive field convolutions are used for local fine feature extraction and detail enhancement. This convolution focuses on capturing local fine texture and detail features, enabling high-precision feature matching and aggregation in information-rich regions, effectively repairing and enhancing the microscopic details of the image, and avoiding overly smooth reconstruction results. For feature-sparse regions: Large receptive field convolutions are used for wide-area contextual feature extraction and structural association modeling. The large receptive field convolution includes dilated convolutions or large-size convolution kernels. Its core purpose is to expand the receptive field, enabling the network to collect information from a broader contextual region, thereby performing structural inference and contextual completion in information-scarce regions.

[0049] Step 3: After the differential convolution processing described above, feature-dense and feature-sparse regions respectively obtain feature representations suitable for their characteristics. The dynamic content construction module reintegrates the outputs of these two processing paths into a unified feature map through feature aggregation operations. This aggregated feature map is the output feature of the dynamic content construction module. It contains both the fine local details preserved and enhanced by small receptive field convolutions and the wide-area structural correlation information captured and constructed by large receptive field convolutions, achieving an updated feature representation that integrates local details and global structural information.

[0050] Some embodiments of this application also include: The neural network model is trained using a hierarchical loss function; The hierarchical loss function is a weighted sum of mean squared error loss, VGG perception loss, and style loss; Furthermore, the weighting coefficients in the weighted sum are dynamically adjusted according to the range of the target sampling rate.

[0051] In some embodiments of this application, when the weighting coefficients in the weighted sum are dynamically adjusted according to the range of the target sampling rate, the following methods are included: The dynamic adjustment of the weighting coefficients follows these rules: As the target sampling rate decreases, the weighting coefficient of the style loss increases accordingly; As the target sampling rate increases, the weighting coefficient of the mean square error loss increases accordingly.

[0052] In this embodiment, the hierarchical loss function is a weighted sum of three complementary loss terms. Mean Squared Error Loss: This calculates the sum of squared differences in corresponding pixel values ​​between the reconstructed image and the original ground truth image. This loss directly constrains the reconstructed image to maintain pixel accuracy consistent with the target, ensuring correct color and basic brightness. VGG Perceptual Loss: Utilizing a VGG network pre-trained on a large-scale image dataset, this extracts deep features from multiple network layers between the reconstructed and ground truth images and calculates the differences between these features. This loss guides the reconstructed image to maintain consistency with the ground truth image in high-level semantic features and visual content, helping to restore correct object shapes and texture structures, and improving the perceptual quality of the image. Style Loss: Also based on a pre-trained VGG network, but this calculates the difference in Gram matrices between feature maps. The Gram matrix captures the correlation between feature channels, reflecting the overall style and statistical characteristics of the image, such as texture and color distribution. This loss aims to make the reconstructed image approximate the ground truth image in overall style, which is crucial for maintaining the overall visual harmony, color consistency, and texture naturalness of the image.

[0053] In this embodiment, to enable a single neural network model to simultaneously handle different tasks from low to high sampling rates, the weight coefficients in the weighted sum of this invention are dynamically adjusted according to the range of the target sampling rate corresponding to the current training samples. Specifically, as the target sampling rate decreases, the weight coefficient of style loss increases accordingly; as the target sampling rate increases, the weight coefficient of mean squared error loss increases accordingly. At low sampling rates, very little measurement data is available for reconstruction. At this time, the primary goal is to ensure that the reconstructed image has a reasonable and complete overall structure and a consistent color style, avoiding content distortion or severe color distortion. Style loss, through strong constraints on the global statistical characteristics of the image, can most effectively guide the network to achieve this goal. At high sampling rates, there is already relatively sufficient data to support the correct restoration of the overall structure. At this time, the optimization focus shifts to further improving the pixel-level accuracy of the image and restoring richer details and textures. As the most direct pixel-level constraint, mean squared error loss has the most direct effect on improving objective indicators such as PSNR and repairing micro-details. Therefore, increasing its weight coefficient ratio at this stage helps the model achieve more refined reconstruction.

[0054] In this embodiment, the training and testing of the network of the present invention specifically includes the following steps: The model is trained and validated using the publicly available underwater image dataset UVTD. This dataset contains 5380 real underwater images and their corresponding depth maps. To facilitate network processing, all images and depth maps are uniformly scaled to a fixed size of 256×256 pixels. Following conventional machine learning data partitioning principles, 3504 images from the dataset are used as the training set to optimize network parameters; the remaining 1876 images are used as an independent test set.

[0055] The network of this invention is built using the PyTorch deep learning framework. Training was performed on a computer equipped with an NVIDIA RTX 4090 GPU. During training, the batch size was set to 8, and a random value in the range [0.01, 0.3] was assigned to each data point in the batch as the corresponding sampling rate input. The Adam optimizer was used, with an initial learning rate set to 0.0002. A total of 200 epochs were trained, employing a learning rate decay strategy, where the learning rate was halved every 50 epochs to promote model convergence and improve final performance.

[0056] To verify the dynamic adjustment mechanism of the hierarchical loss function, in a specific implementation example, different weight coefficients are assigned to the mean squared error loss, VGG perceptual loss, and style loss according to different ranges of the target sampling rate, in order to achieve the optimal balance of the reconstruction target: When the target sampling rate is below 0.05: priority is given to ensuring the overall structure and color consistency of the reconstructed image. In this case, the weight coefficients of the three losses are set to 0.3, 0.2, and 0.5, respectively. When the target sampling rate is between 0.05 and 0.2: it is necessary to balance the structural integrity of the image and the restoration of details. In this case, the weight coefficients of the three losses are adjusted to 0.4, 0.3, and 0.3, respectively. When the target sampling rate is above 0.2: the focus is on improving the pixel-level accuracy and detail richness of the image. In this case, the weight coefficients of the three losses are set to 0.5, 0.4, and 0.2, respectively.

[0057] After the above training, the single model described in this invention demonstrated superior performance on the test set. At various sampling rates from 0.01 to 0.3, its reconstructed images showed significantly better PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) metrics than the comparison algorithms. The reconstructed results exhibited clear textures and sharp edges, effectively mitigating common underwater image blurring and color shift issues, and maintaining the integrity of the main scene structure even at extremely low sampling rates. Furthermore, the total number of parameters for this single model is approximately 4.88M, far lower than some methods that require separate storage for different sampling rates, significantly reducing the storage and computing resource requirements during deployment.

[0058] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of this application, and these improvements and substitutions should also be considered within the scope of protection of this application.

Claims

1. A multi-sampling-rate joint underwater image compressed sensing network, characterized in that, include: A dual-branch dynamic sampling network is used to generate a sampling matrix based on the target sampling rate, the underwater image, and the depth map of the underwater image, and to sample the underwater image to obtain sampling data. A structure-guided dynamic reconstruction network is used to receive the sampled data, the depth map and the target sampling rate, dynamically adjust the reconstruction strategy, perform image reconstruction and output the reconstructed underwater image. The depth map is a scene depth information map corresponding to the underwater image; the dual-branch dynamic sampling network and the structure-guided dynamic reconstruction network are integrated into a single neural network model and are adaptable to underwater image compression sensing tasks at various sampling rates.

2. The underwater image compressed sensing network with multiple sampling rates as described in claim 1, characterized in that, A dual-branch dynamic sampling network is used to generate a sampling matrix based on a target sampling rate, an underwater image, and a depth map of the underwater image, and to sample the underwater image to obtain sampled data, including: The underwater image and its corresponding depth map are received, and based on the target sampling rate, the underwater image is analyzed for content features to generate an attention weight mask at the current sampling rate, and the depth map is analyzed for structural features to generate a structural attention weight mask. The content attention weight mask and the structure attention weight mask are dynamically weighted and fused to generate an image feature sampling priority mask that matches the target sampling rate. The image feature sampling priority mask is binarized to generate a binary sampling matrix; The underwater image is sampled using the binary sampling matrix to obtain the sampled data.

3. The underwater image compressed sensing network with multiple sampling rates as described in claim 2, characterized in that, The underwater image and its corresponding depth map are received, and based on the target sampling rate, content feature analysis is performed on the underwater image to generate an attention weight mask at the current sampling rate. When performing structural feature analysis on the depth map to generate a structural attention weight mask, the process includes: The dual-branch dynamic sampling network includes a content feature analysis branch and a structural feature analysis branch, which are used to perform the content feature analysis and structural feature analysis. The dual-branch dynamic sampling network further includes a weight generation network, which is used to convert the target sampling rate into two learnable and adjustable parameters, which are respectively used as the fusion weight coefficients of the content attention weight mask and the structure attention weight mask. The content feature analysis branch performs feature analysis and weight mapping on the underwater image to generate the content attention weight mask, and the structure feature analysis branch performs feature analysis and weight mapping on the depth map to generate the structure attention weight mask.

4. The underwater image compressed sensing network with multiple sampling rates as described in claim 3, characterized in that, When dynamically weighting and fusing the content attention weight mask and the structure attention weight mask to generate an image feature sampling priority mask that matches the target sampling rate, the process includes: The two learnable and adjustable parameters obtained from the weight generation network are used as the fusion weight coefficients of the content attention weight mask and the structure attention weight mask, respectively. Based on the target sampling rate, the proportion of the two fusion weight coefficients is dynamically adjusted. At a low sampling rate, the proportion of the fusion weight coefficient corresponding to the structural attention weight mask is increased, and at a high sampling rate, the proportion of the fusion weight coefficient corresponding to the content attention weight mask is increased. The content attention weight mask and the structure attention weight mask are multiplied by their respective fusion weight coefficients and then superimposed to generate an image feature sampling priority mask that matches the target sampling rate.

5. The underwater image compressed sensing network with multiple sampling rates as described in claim 4, characterized in that, When performing binarization processing on the image feature sampling priority mask to generate a binary sampling matrix, the following steps are included: Based on the sampling ratio indicated by the target sampling rate, a corresponding number of high-priority positions are selected as sampling points from the image feature sampling priority mask; The position corresponding to the sampling point is assigned a value of 1, and the other positions are assigned a value of 0, generating a binary sampling matrix with the same spatial dimension as the input underwater image.

6. The underwater image compressed sensing network with multiple sampling rates as described in claim 1, characterized in that, The structure-guided dynamic reconstruction network, used to receive the sampled data, the depth map, and the target sampling rate, dynamically adjust the reconstruction strategy, perform image reconstruction, and output the reconstructed underwater image, includes: The structure-guided dynamic reconstruction network includes multiple cascaded structure-adaptive reconstruction modules. Each structure-adaptive reconstruction module receives the current feature map, the depth map, and the target sampling rate, and includes a sampling rate condition block and a dynamic content construction module. The sampling rate conditional block is used to generate a weight adjustment coefficient ω and a bias adjustment coefficient γ by performing an affine transformation on the target sampling rate, and to embed ω and γ into the input features to obtain a sampling rate conditional feature map. The dynamic content construction module is used to generate a dynamically dilated convolution using the image structure information reflected by the depth map, perform feature modeling on the sampling rate conditional feature map, and obtain an updated feature map as the input of the next module; Through cascaded iterative processing of multiple structure-adaptive reconstruction modules, the reconstructed underwater image is finally output.

7. The underwater image compressed sensing network with multiple sampling rates as described in claim 6, characterized in that, The sampling rate conditional block, used to generate weight adjustment coefficients ω and bias adjustment coefficients γ by performing an affine transformation on the target sampling rate, and embedding ω and γ into the input features to obtain the sampling rate conditional feature map, includes: The target sampling rate is input into two independent fully connected networks; Apply the Softplus activation function to the output of the first fully connected network to obtain the weight adjustment coefficients ω; Applying the ReLU activation function to the output of the second fully connected network yields the bias adjustment coefficient γ; The input features are convolutionally processed to obtain convolutional features; The product of the convolutional feature and the weight adjustment coefficient ω is calculated, and then the bias adjustment coefficient γ is added to obtain the sampling rate conditional feature map.

8. The underwater image compressed sensing network with multiple sampling rates as described in claim 7, characterized in that, The dynamic content construction module is used to generate a dynamically dilated convolution using the image structure information reflected in the depth map, to perform feature modeling on the sampling rate conditional feature map, and to obtain an updated feature map, which, when used as input to the next module, includes: Based on the depth map, identify the feature-dense and feature-sparse regions in the sampling rate condition feature map; Feature extraction is performed using small receptive field convolution in the feature-dense regions; Large receptive field convolution is used to extract features and model associations in the sparse feature regions. The features processed by convolution with different receptive fields are aggregated to obtain the updated feature map, which is then used as the input for the next module.

9. The underwater image compressed sensing network with multiple sampling rates as described in claim 7, characterized in that, Also includes: The neural network model is trained using a hierarchical loss function; The hierarchical loss function is a weighted sum of mean squared error loss, VGG perception loss, and style loss; Furthermore, the weighting coefficients in the weighted sum are dynamically adjusted according to the range of the target sampling rate.

10. The underwater image compressed sensing network with multiple sampling rates as described in claim 9, characterized in that, When the weighting coefficients in the weighted sum are dynamically adjusted according to the range of the target sampling rate, the following are included: The dynamic adjustment of the weighting coefficients follows these rules: As the target sampling rate decreases, the weighting coefficient of the style loss increases accordingly; As the target sampling rate increases, the weighting coefficient of the mean square error loss increases accordingly.

Images