Underwater image compressive sensing method and system based on inside-outside joint learning

Abstract

The application relates to the technical field of image processing, and discloses an underwater image compressive sensing method and system based on internal-external joint learning, which comprises the following steps: obtaining an underwater image to be processed; performing inter-image similar feature query and matching on the underwater image based on a pre-trained underwater feature dictionary to generate a global reference feature; processing the underwater image and the corresponding global reference feature through a sampling branch of an intra-image-inter-image joint analysis module to obtain low-dimensional measurement values; reconstructing the low-dimensional measurement values and the corresponding global reference feature through a reconstruction branch of the joint analysis module, wherein the reconstruction branch supplements image information by fusing the global reference feature; and outputting a high-quality reconstructed image. By mining and utilizing the semantic similarity between underwater images and the color and depth degradation characteristics in the images, underwater image reconstruction with higher visual quality is realized at an extremely low sampling rate.

Description

Technical Field

[0001] This invention relates to the field of underwater image processing and compressed sensing technology, and more specifically, to an underwater image compressed sensing method and system based on internal and external joint learning. Background Technology

[0002] With the rapid development of underwater-related industries such as marine resource exploration and ecological monitoring, researchers have placed higher demands on the quality of underwater images, which are one of the core information carriers for underwater research. However, the underwater environment suffers from image degradation (color shift, low contrast, and blurred details) due to light absorption and scattering, and underwater communication relies on low-bandwidth acoustic signals, posing significant challenges to the transmission of high-definition image data.

[0003] Compressed sensing (CS) theory states that if a signal exhibits sparsity in a specific domain, accurate reconstruction can be achieved through sampling at rates far below the Nyquist threshold, providing a new approach for the efficient acquisition and transmission of underwater images. In recent years, deep learning-based compressed sensing methods have made significant progress in land imagery, but the unique characteristics of underwater images make these methods difficult to apply directly. Currently, there is limited research on compressed sensing for underwater images, and existing methods still face performance bottlenecks. Summary of the Invention

[0004] This application provides an underwater image compressed sensing method and system based on intra- and inter-image joint learning. Through an intra- and inter-image joint learning mechanism, it effectively solves the problems of poor performance and insufficient generalization ability of existing underwater image compressed sensing methods at low sampling rates. On the one hand, the underwater feature dictionary fully utilizes the semantic similarity between underwater images to provide rich reference features and compensate for the loss of structural information. On the other hand, the color-aware channel attention block and the depth-aware adaptive dilated convolution block accurately adapt to the color and depth-related degradation characteristics within a single image, achieving differentiated sampling and reconstruction. Experiments demonstrate that this invention significantly outperforms existing methods in quantitative indicators such as PSNR and SSIM, as well as subjective visual quality, especially showing significant advantages at extremely low sampling rates. It also exhibits strong generalization ability and can be adapted to different underwater scenarios, providing an efficient solution for resource-constrained underwater image transmission and reconstruction.

[0005] To achieve the above objectives, this invention provides an underwater image compressed sensing method based on joint internal and external learning, comprising:

[0006] Acquire underwater images to be processed; Based on a pre-trained underwater feature dictionary, the underwater images are used to query and match similar features between images to generate global reference features. The underwater image and corresponding global reference features are processed by the sampling branch of the image-inter-image joint analysis module to obtain low-dimensional measurement values; The low-dimensional measurement value and the corresponding global reference feature are reconstructed through the reconstruction branch of the joint analysis module. The reconstruction branch completes the image information by fusing the global reference feature. Output high-quality reconstructed images.

[0007] Furthermore, the pre-trained underwater feature dictionary specifically includes: Acquire a training set of real underwater images and classify them according to water color to balance the data; Extract semantic region features from images through semantic mapping; A greedy algorithm is used to select multiple typical underwater semantics from various water bodies to construct a training subset; Based on the training subset, a discrete feature codebook for storing common semantic features is trained; After the discrete feature codebook is trained, a multi-scale feature fusion generator is further trained to fuse the multi-scale discrete features obtained from the codebook to generate global reference features that match the size of the input image.

[0008] Furthermore, based on the aforementioned training subset, a discrete feature codebook for storing common semantic features is trained, specifically including: First, the images in the training subset are input into the feature extraction network to extract features at multiple scales, and the features of each semantic at the corresponding scale are learned based on the semantic graph. Then, for each spatial scale, a discrete feature codebook is trained using vector quantization, where each codebook consists of a set of learnable feature vectors. During training, each local feature in the semantic feature map is mapped to the closest feature vector in the codebook at the corresponding scale through nearest neighbor matching. The value of the feature vector is optimized by reconstruction loss and codebook update strategy, so that each codebook can effectively store the common features of the typical underwater semantics at the corresponding scale.

[0009] Furthermore, a greedy algorithm is used to select multiple typical underwater semantics from various water bodies, specifically including: Iterate through each subset of data categorized by water color; In each data subset, the images are semantically segmented based on predefined underwater semantic category labels; With the goal of maximizing semantic category coverage and ensuring a relatively balanced number of samples for each semantic category, images are iteratively selected from the current data subset to be added to the training subset until all preset semantic categories have reached a preset sample number threshold or the data is exhausted.

[0010] Furthermore, the underwater image and corresponding global reference features are processed through the sampling branch of the intra-image and inter-image joint analysis module to obtain low-dimensional measurement values, specifically including: The sampling branch analyzes the color and depth degradation characteristics of the underwater image through the intra-image analysis submodule, and uses the inter-image analysis submodule to align the global reference features with the degradation style and spatial structure of the underwater image. The residual between the global reference feature and the underwater image feature is calculated, and the residual feature is compressed and sampled to generate the low-dimensional measurement value.

[0011] Furthermore, the intra-image analysis submodule includes: The color-aware channel attention block is used to analyze the RGB color channel information of the underwater image. The color-aware channel attention block dynamically generates channel attention weights based on the estimated background light, and prioritizes the enhancement and retention of color channel features with high information entropy. A depth-aware adaptive dilated convolutional block is used to analyze the sharpness attenuation characteristics of the underwater image related to the imaging distance. The depth-aware adaptive dilated convolutional block calculates the dilation rate of each region based on the depth map. For regions with close imaging distance, a small dilation rate convolution is used to extract detailed features, while for regions with far imaging distance, a large dilation rate convolution is used to aggregate contextual information.

[0012] Furthermore, the image analysis submodule includes: The difference-aware dictionary feature fusion module normalizes the global reference features to the underwater image using a degenerate style. This process is based on the underwater imaging model and is completed by inputting the global reference features, the estimated depth map of the global reference features and the background light, and the estimated depth map of the underwater image and the background light into the dictionary feature variant block. After normalization, the spatial offset between the global reference feature and the underwater image feature is calculated and learned through a deformable convolutional block, and the global reference feature is spatially deformed according to the spatial offset to align with the underwater image feature in structure.

[0013] Furthermore, the low-dimensional measurement values ​​and corresponding global reference features are reconstructed through the reconstruction branch of the joint analysis module, specifically including: The reconstruction branch receives the low-dimensional measurement value as input and inputs the corresponding global reference features into the inter-image analysis submodule and intra-image analysis submodule of the reconstruction branch; The image analysis submodule uses color-aware channel attention blocks and depth-aware adaptive dilated convolution blocks to refine the fused features based on underwater degradation characteristics, so as to recover a high-quality reconstructed image with rich details and clear structure. The image analysis submodule aligns the global reference features with the image features represented by the low-dimensional measurement values ​​in terms of degenerate style and spatial structure, and adds the aligned reference features with the low-dimensional measurement values ​​to fill in missing information using known features.

[0014] Furthermore, high-quality reconstructed images specifically include: At extremely low sampling rates, the images reconstructed by the method are superior to the original high-definition underwater images in terms of peak signal-to-noise ratio and structural similarity index under the same conditions, compared with the images reconstructed by an adaptive compressed sensing network based on underwater physical priors. Furthermore, the reconstructed image is visually free of significant blurring and distortion, and is able to preserve the detailed structure and texture features of the target in the underwater scene.

[0015] To achieve the above objectives, the present invention also provides an underwater image compressed sensing system based on joint internal and external learning, comprising: The image acquisition module acquires underwater images to be processed. The query and matching module performs image similarity feature query and matching on the underwater images based on a pre-trained underwater feature dictionary, and generates global reference features. The sampling analysis module processes the underwater image and the corresponding global reference features through the sampling branch of the intra-image and inter-image joint analysis module to obtain low-dimensional measurement values; The reconstruction analysis module reconstructs the low-dimensional measurement values ​​and corresponding global reference features through the reconstruction branch of the joint analysis module. The reconstruction branch completes the image information by fusing the global reference features. The data output module outputs high-quality reconstructed images.

[0016] Compared with existing technologies, the advantages of this invention are as follows: it significantly improves the reconstruction quality of underwater images at extremely low sampling rates; it completes features by constructing an underwater feature dictionary and utilizing semantic similarity between images; and it adaptively handles the unique color monotony and distance-related degradation of underwater images using color perception and depth perception modules, thus achieving efficient modeling of underwater scenes. Simultaneously, the strategy based on multi-scale feature codebooks and multi-body balanced training enhances the algorithm's generalization ability in different underwater environments. Ultimately, while ensuring rich details and clear structure in the reconstructed images, it significantly reduces sampling and transmission overhead, providing a practical solution for high-quality image reconstruction in low-bandwidth underwater communication environments. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating the underwater image compressed sensing method based on joint internal and external learning in an embodiment of the present invention is shown. Figure 2 A schematic diagram of the underwater image compressed sensing system based on internal and external joint learning in an embodiment of the present invention is shown.

[0018] Figure 3 The overall algorithm structure diagram of the compressed sensing algorithm for joint learning between images in this invention is shown.

[0019] Figure 4 A structural diagram of the underwater feature dictionary in this invention is shown.

[0020] Figure 5 The diagram shows the structure of the intra-image and inter-image joint analysis module in this invention. Detailed Implementation

[0021] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0022] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0023] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0024] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0025] The following is a description of preferred embodiments of the present invention in conjunction with the accompanying drawings.

[0026] like Figure 1 As shown, embodiments of the present invention disclose an underwater image compressed sensing method based on joint internal and external learning, including: S110: Acquire the underwater image to be processed; S120: Based on a pre-trained underwater feature dictionary, perform image similarity feature query and matching on the underwater image to generate global reference features; In some embodiments of the present invention, the pre-trained underwater feature dictionary specifically includes: Acquire a training set of real underwater images and classify them according to water color to balance the data; Extract semantic region features from images through semantic mapping; A greedy algorithm is used to select multiple typical underwater semantics from various water bodies to construct a training subset; Based on the training subset, a discrete feature codebook for storing common semantic features is trained; After the discrete feature codebook is trained, a multi-scale feature fusion generator is further trained to fuse the multi-scale discrete features obtained from the codebook to generate global reference features that match the size of the input image.

[0027] In this embodiment, the training process of the underwater feature dictionary includes: Training dataset: UVTD dataset (5380 real underwater images), divided into dictionary construction set (1000 images), network training set (3504 images), and test set (876 images).

[0028] Image preprocessing: All input images are resized to 256×256.

[0029] Network Implementation: All networks were built on the PyTorch framework and trained on an NVIDIA RTX4090 with a batch size of 8. The initial learning rate was 0.0002, and the training lasted for 200 epochs, with the learning rate decreasing by half every 50 epochs.

[0030] Dictionary training phase: First train a 4-scale codebook for 8 semantic categories, and then train MSFG after fixing the codebook parameters.

[0031] Training phase of compressed sensing network: Fix underwater feature dictionary parameters and train the sampling and reconstruction module of the intra-image and inter-image joint analysis module.

[0032] In this embodiment, a discrete feature codebook for storing common semantic features is trained based on the training subset, specifically including: First, the images in the training subset are input into the feature extraction network to extract features at multiple scales, and the features of each semantic at the corresponding scale are learned based on the semantic graph. Then, for each spatial scale, a discrete feature codebook is trained using vector quantization, where each codebook consists of a set of learnable feature vectors. During training, each local feature in the semantic feature map is mapped to the closest feature vector in the codebook at the corresponding scale through nearest neighbor matching. The value of the feature vector is optimized by reconstruction loss and codebook update strategy, so that each codebook can effectively store the common features of the typical underwater semantics at the corresponding scale.

[0033] In this embodiment, a greedy algorithm is used to select multiple typical underwater semantics from various water bodies in a balanced manner, specifically including: Iterate through each subset of data categorized by water color; In each data subset, the images are semantically segmented based on predefined underwater semantic category labels; With the goal of maximizing semantic category coverage and ensuring a relatively balanced number of samples for each semantic category, images are iteratively selected from the current data subset to be added to the training subset until all preset semantic categories have reached a preset sample number threshold or the data is exhausted.

[0034] In this embodiment, as Figure 4As shown, the underwater feature dictionary structure includes a multi-scale feature matcher (MFM) based on a UNet encoder, used to align the scale of the input features with the scale of the features stored in the dictionary to improve matching accuracy; a series of discrete feature codebooks for storing features, providing matching of similar features; and a multi-scale feature fusion generator (MSFG), used to combine the discrete similar features retrieved from the codebooks to output a usable reference image. In the construction process, images from the largest real-world underwater image dataset, UVTD, are first classified according to water color to ensure data balance across various water bodies. Then, a greedy algorithm is used to balance and select eight underwater semantic classes for each water body, resulting in approximately 1000 underwater images used to train the discrete codebooks and MSFG, enabling feature quantization and fusion to generate global reference features.

[0035] The beneficial effects of the above technical solution are as follows: by constructing a pre-trained underwater feature dictionary based on multi-body balance classification and greedy algorithm selection, and combining a multi-scale feature matcher and fusion generator, similar features can be found at all sampling rates, and global reference features with high semantic similarity can be efficiently queried and matched from underwater images; the high degree of semantic repetition of targets between underwater images is fully utilized, providing strong prior information for subsequent compressed sampling and reconstruction processes, effectively compensating for the loss of structural and detailed information caused by extremely low sampling rates, and significantly improving the overall reconstruction quality and visual fidelity of underwater image compressed sensing.

[0036] S130: The underwater image and the corresponding global reference features are processed by the sampling branch of the intra-image-inter-image joint analysis module to obtain low-dimensional measurement values; In some embodiments of the present invention, the underwater image and the corresponding global reference features are processed by the sampling branch of the intra-image-inter-image joint analysis module to obtain low-dimensional measurement values, specifically including: The sampling branch analyzes the color and depth degradation characteristics of the underwater image through the intra-image analysis submodule, and uses the inter-image analysis submodule to align the global reference features with the degradation style and spatial structure of the underwater image. The residual between the global reference feature and the underwater image feature is calculated, and the residual feature is compressed and sampled to generate the low-dimensional measurement value.

[0037] In this embodiment, the in-image analysis submodule includes: The color-aware channel attention block is used to analyze the RGB color channel information of the underwater image. The color-aware channel attention block dynamically generates channel attention weights based on the estimated background light, and prioritizes the enhancement and retention of color channel features with high information entropy. A depth-aware adaptive dilated convolutional block is used to analyze the sharpness attenuation characteristics of the underwater image related to the imaging distance. The depth-aware adaptive dilated convolutional block calculates the dilation rate of each region based on the depth map. For regions with close imaging distance, a small dilation rate convolution is used to extract detailed features, while for regions with far imaging distance, a large dilation rate convolution is used to aggregate contextual information.

[0038] In this embodiment, the image analysis submodule includes: The difference-aware dictionary feature fusion module normalizes the global reference features to the underwater image using a degenerate style. This process is based on the underwater imaging model and is completed by inputting the global reference features, the estimated depth map of the global reference features and the background light, and the estimated depth map of the underwater image and the background light into the dictionary feature variant block. After normalization, the spatial offset between the global reference feature and the underwater image feature is calculated and learned through a deformable convolutional block, and the global reference feature is spatially deformed according to the spatial offset to align with the underwater image feature in structure.

[0039] In this embodiment, as Figure 5 As shown, the Intra-Image and Inter-Image Joint Analysis Module (IIA-Module) is divided into a sampling end (IIA-ModuleS) and a reconstruction end (IIA-ModuleR), consisting of an intra-image analysis submodule and an inter-image analysis submodule. IIA-ModuleS and IIA-ModuleR use the same methods for extracting and utilizing intra-image features, but differ slightly in their utilization of inter-image features. Specifically, IIA-ModuleS primarily subtracts the dictionary features from the image features, focusing only on the difference in sampling, thus allocating more sampling resources to other regions. In contrast, IIA-ModuleR adds the dictionary features to the image features, using known features to fill in missing features, thereby improving reconstruction quality.

[0040] In this example, the image analysis submodule is the Differential Aware Dictionary Feature Fusion (DA-UFFM) module. This module normalizes dictionary features to image features through special transformations, making their style and structure more similar to image features, thereby improving the utilization rate of dictionary features. Among them, the Dictionary Feature Variant Block (DFVB) is designed based on the underwater imaging model (Equation 1), which normalizes the degenerate style of dictionary features and image features, eliminating the influence of style differences on the utilization of dictionary features.

[0041] in It is an underwater image to be sampled, affected by underwater imaging degradation. It contains no degraded structural or detailed information. It's a depth map. It's the background light. In our algorithm, and This is obtained through an underwater prior estimation network. Based on Equation 1, a DFVB (Equation 2) was designed, specifically implementing the following feature changes:

[0042] in This represents the result after normalizing the dictionary features to the input image features. It is obtained from the depth map and a fixed parameter β. After style normalization, the spatial offset between dictionary features and image features is calculated through a deformable convolutional block. By learning the offset adjustment amount, the position and shape of dictionary features are adjusted so that they are structurally closer to image features, thereby improving the matching degree and realizing the utilization of similarity between images.

[0043] In this embodiment, the image analysis module mainly utilizes the underwater image imaging model (Formula 1) to analyze the sampling focus of a single image. Specifically, it uses a Color-Aware Channel Attention Block (CA-CAB) to perform sampling focus analysis of the RGB color channels and a Depth-Aware Adaptive Dilated Convolutional Block (DA-ADB) to perform texture sampling focus analysis. CA-CAB generates a feature-color dual attention map based on the background and through a projection network, dynamically adjusting the weights of different color channels and prioritizing the retention of features from channels with high information entropy. DA-ADB calculates the dilation rate based on the depth map, using a large dilation rate convolution for distant regions and a small dilation rate convolution for nearby regions, combined with global context aggregation, to achieve distance-related feature extraction.

[0044] The beneficial effects of the above technical solution are as follows: By using the sampling branch of the intra-image and inter-image joint analysis module, a residual sampling mechanism is introduced when compressing and sampling underwater images. First, the intra-image analysis submodules (CA-CAB and DA-ADB) are used to adaptively analyze and enhance the key region features in a single image that are affected by color and depth degradation. At the same time, the inter-image analysis submodule (DA-UFFM) is used to accurately align the global reference features with the degradation style and spatial structure under the guidance of the underwater imaging model. Finally, only the difference between the aligned reference features and the original image features is compressed and sampled, so that the limited sampling resources can be concentrated on capturing truly unknown and unique information in the image, which significantly improves the information capture efficiency at extremely low sampling rates.

[0045] S140: The low-dimensional measurement value and the corresponding global reference feature are reconstructed through the reconstruction branch of the joint analysis module. The reconstruction branch completes the image information by fusing the global reference feature. In this embodiment, the low-dimensional measurement value and the corresponding global reference feature are reconstructed through the reconstruction branch of the joint analysis module, specifically including: The reconstruction branch receives the low-dimensional measurement value as input and inputs the corresponding global reference features into the inter-image analysis submodule and intra-image analysis submodule of the reconstruction branch; The image analysis submodule uses color-aware channel attention blocks and depth-aware adaptive dilated convolution blocks to refine the fused features based on underwater degradation characteristics, so as to recover a high-quality reconstructed image with rich details and clear structure. The image analysis submodule aligns the global reference features with the image features represented by the low-dimensional measurement values ​​in terms of degenerate style and spatial structure, and adds the aligned reference features with the low-dimensional measurement values ​​to fill in missing information using known features.

[0046] In this embodiment, the color-perceived channel attention block and the depth-perceived adaptive dilated convolution block specifically include: The color perception channel attention block generates channel weights based on the estimated background light and adaptively weights the RGB channels of the input features to enhance channels with high information entropy. The depth-aware adaptive dilated convolutional block predicts the dilation rate of different regions based on the estimated depth map, and performs adaptive dilated convolution according to the dilation rate to achieve differentiated extraction of near-range details and far-range contextual information.

[0047] The beneficial effects of the above technical solution are as follows: Through the reconstruction branch of the joint analysis module, a progressive processing flow is realized in the image reconstruction stage. First, the intra-image analysis submodule uses color perception and depth perception mechanisms to adaptively refine and enhance the fused features for underwater degradation characteristics. Then, the inter-image analysis submodule uses the global reference features to accurately align the image features implied by the low-dimensional measurement values ​​with degradation style and spatial structure. By adding features, the known semantic information in the reference features is directly used to effectively fill in the details lost due to compressed sampling, which significantly improves the practicality and reliability of the compressed sensing system in complex underwater environments.

[0048] S150: Outputs high-quality reconstructed images.

[0049] In this embodiment, the technical standards for high-quality reconstructed images include: At extremely low sampling rates, the images reconstructed by the method are superior to the original high-definition underwater images in terms of peak signal-to-noise ratio and structural similarity index under the same conditions, compared with the images reconstructed by an adaptive compressed sensing network based on underwater physical priors. Furthermore, the reconstructed image is visually free of significant blurring and distortion, and is able to preserve the detailed structure and texture features of the target in the underwater scene.

[0050] To further illustrate the technical concept of this invention, the technical solution of this invention will now be described in conjunction with specific application scenarios.

[0051] Correspondingly, such as Figure 2 As shown, this application also provides an underwater image compressed sensing system based on internal and external joint learning, comprising: The image acquisition module acquires underwater images to be processed. The query and matching module performs image similarity feature query and matching on the underwater images based on a pre-trained underwater feature dictionary, and generates global reference features. The sampling analysis module processes the underwater image and the corresponding global reference features through the sampling branch of the intra-image and inter-image joint analysis module to obtain low-dimensional measurement values; The reconstruction analysis module reconstructs the low-dimensional measurement values ​​and corresponding global reference features through the reconstruction branch of the joint analysis module. The reconstruction branch completes the image information by fusing the global reference features. The data output module outputs high-quality reconstructed images.

[0052] In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0053] Although the invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the embodiments disclosed in this invention can be combined with each other in any way. The fact that not all of these combinations are described in this specification is merely for the sake of brevity and resource conservation.

[0054] It will be understood by those skilled in the art that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An underwater image compressed sensing method based on joint internal and external learning, characterized in that, include: Acquire underwater images to be processed; Based on a pre-trained underwater feature dictionary, the underwater images are used to query and match similar features between images to generate global reference features. The underwater image and corresponding global reference features are processed by the sampling branch of the image-inter-image joint analysis module to obtain low-dimensional measurement values; The low-dimensional measurement value and the corresponding global reference feature are reconstructed through the reconstruction branch of the joint analysis module. The reconstruction branch completes the image information by fusing the global reference feature. Output high-quality reconstructed images.

2. The underwater image compressed sensing method based on joint internal and external learning according to claim 1, characterized in that, The pre-trained underwater feature dictionary specifically includes: Acquire a training set of real underwater images and classify them according to water color to balance the data; Extract semantic region features from images through semantic mapping; A greedy algorithm is used to select multiple typical underwater semantics from various water bodies to construct a training subset; Based on the training subset, a discrete feature codebook for storing common semantic features is trained; After the discrete feature codebook is trained, a multi-scale feature fusion generator is further trained to fuse the multi-scale discrete features obtained from the codebook to generate global reference features that match the size of the input image.

3. The underwater image compressed sensing method based on joint internal and external learning according to claim 2, characterized in that, Based on the aforementioned training subset, a discrete feature codebook for storing common semantic features is trained, specifically including: The images in the training subset are input into the feature extraction network to extract features at multiple scales, and the features of each semantic at the corresponding scale are learned based on the semantic graph; For each spatial scale, a discrete feature codebook is trained using vector quantization techniques, where each codebook consists of a set of learnable feature vectors. During training, each local feature in the semantic feature map is mapped to the closest feature vector in the codebook at the corresponding scale through nearest neighbor matching. The value of the feature vector is optimized by reconstruction loss and codebook update strategy, so that each codebook can effectively store the common features of the typical underwater semantics at the corresponding scale.

4. The underwater image compressed sensing method based on joint internal and external learning according to claim 2, characterized in that, A greedy algorithm is used to select multiple typical underwater semantics from various water bodies, specifically including: Iterate through each subset of data categorized by water color; In each data subset, the images are semantically segmented based on predefined underwater semantic category labels; With the goal of maximizing semantic category coverage and ensuring a relatively balanced number of samples for each semantic category, images are iteratively selected from the current data subset to be added to the training subset until all preset semantic categories have reached a preset sample number threshold or the data is exhausted.

5. The underwater image compressed sensing method based on joint internal and external learning according to claim 1, characterized in that, The underwater image and corresponding global reference features are processed through the sampling branch of the intra-image and inter-image joint analysis module to obtain low-dimensional measurement values, specifically including: The sampling branch analyzes the color and depth degradation characteristics of the underwater image through the intra-image analysis submodule, and uses the inter-image analysis submodule to align the global reference features with the degradation style and spatial structure of the underwater image. The residual between the global reference feature and the underwater image feature is calculated, and the residual feature is compressed and sampled to generate the low-dimensional measurement value.

6. The underwater image compressed sensing method based on joint internal and external learning according to claim 5, characterized in that, The image intra-analysis submodule includes: The color-aware channel attention block is used to analyze the RGB color channel information of the underwater image. The color-aware channel attention block dynamically generates channel attention weights based on the estimated background light, and prioritizes the enhancement and retention of color channel features with high information entropy. A depth-aware adaptive dilated convolutional block is used to analyze the sharpness attenuation characteristics of the underwater image related to the imaging distance. The depth-aware adaptive dilated convolutional block calculates the dilation rate of each region based on the depth map. For regions with close imaging distance, a small dilation rate convolution is used to extract detailed features, while for regions with far imaging distance, a large dilation rate convolution is used to aggregate contextual information.

7. The underwater image compressed sensing method based on joint internal and external learning according to claim 5, characterized in that, The image analysis submodule includes: The difference-aware dictionary feature fusion module normalizes the global reference features to the underwater image using a degenerate style. This process is based on the underwater imaging model and is completed by inputting the global reference features, the estimated depth map of the global reference features and the background light, and the estimated depth map of the underwater image and the background light into the dictionary feature variant block. After normalization, the spatial offset between the global reference feature and the underwater image feature is calculated and learned through a deformable convolutional block, and the global reference feature is spatially deformed according to the spatial offset to align with the underwater image feature in structure.

8. The underwater image compressed sensing method based on joint internal and external learning according to claim 1, characterized in that, The reconstruction branch of the joint analysis module reconstructs the low-dimensional measurements and corresponding global reference features, specifically including: The reconstruction branch receives the low-dimensional measurement value as input and inputs the corresponding global reference features into the inter-image analysis submodule and intra-image analysis submodule of the reconstruction branch; The image analysis submodule uses color-aware channel attention blocks and depth-aware adaptive dilated convolution blocks to refine the fused features based on underwater degradation characteristics, so as to recover a high-quality reconstructed image with rich details and clear structure. The image analysis submodule aligns the global reference features with the image features represented by the low-dimensional measurement values ​​in terms of degenerate style and spatial structure, and adds the aligned reference features with the low-dimensional measurement values ​​to fill in missing information using known features.

9. The underwater image compressed sensing method based on joint internal and external learning according to claim 8, characterized in that, The color-aware channel attention block and the depth-aware adaptive dilated convolution block specifically include: The color perception channel attention block generates channel weights based on the estimated background light and adaptively weights the RGB channels of the input features to enhance channels with high information entropy. The depth-aware adaptive dilated convolutional block predicts the dilation rate of different regions based on the estimated depth map, and performs adaptive dilated convolution according to the dilation rate to achieve differentiated extraction of near-range details and far-range contextual information.

10. An underwater image compressed sensing system based on internal and external joint learning, applied to the underwater image compressed sensing method based on internal and external joint learning as described in any one of claims 1-9, characterized in that, include: The image acquisition module acquires underwater images to be processed. The query and matching module performs image similarity feature query and matching on the underwater images based on a pre-trained underwater feature dictionary, and generates global reference features. The sampling analysis module processes the underwater image and the corresponding global reference features through the sampling branch of the intra-image and inter-image joint analysis module to obtain low-dimensional measurement values; The reconstruction analysis module reconstructs the low-dimensional measurement values ​​and corresponding global reference features through the reconstruction branch of the joint analysis module. The reconstruction branch completes the image information by fusing the global reference features. The data output module outputs high-quality reconstructed images.

Images