Enhancement methods, apparatus, media, and devices based on underwater degraded images

Abstract

This invention proposes an enhancement method, apparatus, medium, and device based on underwater degraded images. By combining local feature modeling of image patches and global feature modeling of graph neural networks, it solves the problems of excessive local enhancement and global imbalance in underwater image enhancement. Through processing the cosine similarity matrix, the information transmission capability between image patches is further improved, achieving high-quality enhancement of underwater images. The enhancement method based on underwater degraded images can effectively restore details in underwater images, improve color consistency and brightness balance, and is particularly suitable for complex underwater environments.

Description

Technical Field

[0001] This invention relates to the field of underwater image enhancement methods, and more particularly to an underwater image enhancement method, apparatus, medium, and device. Background Technology

[0002] Marine resource exploration, intelligent underwater robot navigation, and marine ecological monitoring all require the collection of underwater images. However, the propagation of light below the water surface is affected by the water body, plankton, and minerals, resulting in underwater images that often suffer from inconsistent colors, low contrast, and blurriness, which in turn affects subsequent detection, identification, and tracking.

[0003] Traditional image enhancement methods can be categorized into spatial domain methods and frequency domain methods based on the processing domain. Spatial domain methods directly manipulate the grayscale of image pixels, enhancing image brightness and contrast through grayscale mapping or transformation to improve brightness distribution and color balance. Frequency domain methods use transformation techniques such as Fourier transform and wavelet transform to map the image to the frequency domain, achieving image sharpening and detail enhancement through filtering techniques including high-pass, low-pass, and homomorphic filtering. While these two methods can handle underwater images simply, the complex and variable lighting conditions and non-uniform medium properties of underwater environments make it difficult for traditional algorithms to accurately describe the underwater imaging mechanism. This results in significant differences in enhancement effects across different scenarios and may even introduce artifacts or color distortion.

[0004] With the development of neural network models, FUGN (underwater image fusion enhancement method based on graph neural networks) is now used in conjunction with neural networks to extract local features from underwater images, thereby solving problems such as color distortion, low contrast and blurred details in underwater images.

[0005] Although FUGN introduces graph neural networks for feature propagation between local regions, its message passing mechanism is still limited to the local neighborhood of the graph. The message passing process of graph neural networks struggles to capture long-distance dependencies across the entire image.

[0006] In underwater imaging, it is often necessary to process long-range global information, such as color consistency in distant regions and overall brightness balance. If long-range global information cannot be captured, the enhancement results will often perform well in local areas, but the global color and brightness will often be inconsistent, resulting in problems of local over-enhancement and overall imbalance.

[0007] Secondly, FUGN employs convolutional neural networks and graph neural networks for staged processing. Features need to be transformed multiple times between the convolutional domain and the graph domain to obtain the result. However, the feature spaces of the convolutional domain and the graph domain differ significantly. During multiple transformations, information loss or redundancy can easily occur, resulting in a large difference between the final image details and the actual image.

[0008] Another approach is to use the UIE-Convformer neural network model (a model based on the combination of convolutional neural networks and self-attention mechanisms). By combining convolutional neural networks and self-attention mechanisms, the convolutional neural network extracts local features, while the self-attention mechanism builds global dependencies, ultimately outputting underwater images with improved visual quality.

[0009] UIE-Convformer reduces some computational complexity through channel self-attention mechanism, thereby improving the efficiency of feature interaction. However, the model still needs to calculate the correlation between different feature channels. When the number of channels in the network is large, the computational load will be very large, especially when processing large-scale images or high-resolution underwater images, the computational bottleneck is particularly prominent, resulting in the model efficiency not being effectively guaranteed.

[0010] Secondly, UIE-Convformer uses a fixed receptive field in its convolutional operations, which limits its ability to flexibly model complex and irregular spatial structures in underwater images. Convolutional neural networks are constrained by their grid-like receptive field when modeling local features, resulting in limitations in capturing spatial structure and semantic relationships. For complex underwater scenes, such as multiple underwater objects and light refraction, traditional convolutional operations struggle to effectively model complex spatial relationships, leading to poor enhancement performance. Summary of the Invention

[0011] In view of this, it is necessary to provide a method, apparatus, medium, and device for enhancing underwater degraded images to solve the above problems.

[0012] Embodiments of this application provide an enhancement method based on underwater degraded images, the enhancement method based on underwater degraded images comprising: Acquire underwater images and perform preliminary feature extraction on the underwater images to obtain preliminary extracted feature maps; The initially extracted feature map is segmented to obtain segmented image blocks; A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features; Calculate the cosine similarity matrix based on the extracted features; The cosine similarity matrix is ​​processed to obtain an adjacency matrix with prior information.

[0013] In at least one embodiment of this application, the specific steps of acquiring an underwater image and performing preliminary feature extraction on the underwater image to obtain a preliminary extracted feature map include: The underwater image is convolved and normalized by two shallow feature mapping layers to extract shallow features from the underwater image, thus obtaining a preliminary extracted feature map.

[0014] In at least one embodiment of this application, the specific steps of segmenting the initially extracted feature map to obtain segmented image patches include: The initially extracted feature map is divided into several image blocks to form a segmented image set.

[0015] In at least one embodiment of this application, the specific steps of using a neural network model to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features include: The feature extraction unit in the densely connected convolutional network performs multi-level feature encoding on each image block in the segmented image block to obtain the encoded image set. Multi-level feature extraction is performed on the encoded image set to obtain the extracted features.

[0016] In at least one embodiment of this application, the specific steps of performing multi-level feature extraction on the encoded image set to obtain the extracted features include: The encoded image set is subjected to color and texture extraction to obtain the extracted color and texture features, and composite features are generated. Based on the composite features, the cosine similarity between any two image blocks in the segmented image block is calculated, and a cosine similarity matrix is ​​generated.

[0017] In at least one embodiment of this application, the specific steps of processing the cosine similarity matrix to obtain an edge feature adjacency matrix with prior information include: The cosine similarity matrix is ​​sparsified, symmetricized, normalized, and self-loop removed to obtain the adjacency matrix of the graph mechanism.

[0018] In at least one embodiment of this application, the specific steps of performing sparsification, symmetry, normalization, and self-loop removal on the cosine similarity matrix to obtain the adjacency matrix of the graph mechanism include: The cosine similarity matrix is ​​sparsified to generate a sparser similarity matrix. The sparsified similarity matrix is ​​subjected to self-loop removal processing to generate a similarity matrix after self-loop removal; The similarity matrix after removing self-loops is symmetricized to obtain a symmetric similarity matrix; The symmetric similarity matrix is ​​normalized to generate the adjacency matrix of the graph mechanism, and the adjacency matrix is ​​used as the input of the neural network.

[0019] An underwater image enhancement device, applied in any of the above-described underwater image enhancement methods, the underwater image enhancement device comprising: The image acquisition module is used to acquire underwater images; The preliminary feature extraction module is used to perform preliminary feature extraction on underwater images to obtain a preliminary feature map; The segmentation module is used to segment the initially extracted feature map into image blocks. A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features. The cosine similarity calculation module calculates the cosine similarity matrix based on the extracted features; The processing module processes the cosine similarity matrix to generate an adjacency matrix; The enhancement device based on underwater degraded images is used to perform the following steps: Acquire underwater images and perform preliminary feature extraction on the underwater images to obtain preliminary extracted feature maps; The initially extracted feature map is segmented to obtain segmented image blocks; A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features; Calculate the cosine similarity matrix based on the extracted features; The cosine similarity matrix is ​​processed to obtain an adjacency matrix with prior information.

[0020] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps: Acquire underwater images and perform preliminary feature extraction on the underwater images to obtain preliminary extracted feature maps; The initially extracted feature map is segmented to obtain segmented image blocks; A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features; Calculate the cosine similarity matrix based on the extracted features; The cosine similarity matrix is ​​processed to obtain an adjacency matrix with prior information.

[0021] A computer device, characterized in that it includes a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the following steps: Acquire underwater images and perform preliminary feature extraction on the underwater images to obtain preliminary extracted feature maps; The initially extracted feature map is segmented to obtain segmented image blocks; A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features; Calculate the cosine similarity matrix based on the extracted features; The cosine similarity matrix is ​​processed to obtain an adjacency matrix with prior information.

[0022] The method, apparatus, medium, and device for enhancing underwater degraded images according to this embodiment will have at least the following beneficial effects: The aforementioned enhancement methods, apparatus, media, and devices based on underwater degraded images address the problems of excessive local enhancement and global imbalance in underwater image enhancement by combining local feature modeling of image patches with global feature modeling of graph neural networks. Further processing of the cosine similarity matrix enhances the information transfer capability between image patches, achieving high-quality underwater image enhancement. The enhancement methods based on underwater degraded images effectively restore details, improve color consistency and brightness balance in underwater images, and are particularly suitable for complex underwater environments. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] in: Figure 1 This is a flowchart of an enhancement method based on underwater degraded images in one embodiment; Figure 2 A flowchart of an enhancement method based on underwater degraded images in another embodiment; Figure 3 This is a structural block diagram of an apparatus for enhancing underwater degraded images in one embodiment; Figure 4 This is a structural block diagram of a computer device in one embodiment.

[0025] Explanation of key component symbols: 10. Enhancement device based on underwater degraded images; 11. Image acquisition module; 12. Preliminary feature extraction module; 13. Segmentation module; 14. Neural network model; 15. Cosine similarity calculation module; 16. Processing module. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Embodiments of this application provide an enhancement method based on underwater degraded images, the enhancement method based on underwater degraded images comprising: S101. Acquire an underwater image and perform preliminary feature extraction on the underwater image to obtain a preliminary extracted feature map; S102. The preliminary extracted feature map is segmented to obtain segmented image blocks; S103. A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image block to obtain the extracted features; S104. Calculate the cosine similarity matrix based on the extracted features; S105. Process the cosine similarity matrix to obtain an adjacency matrix with prior information.

[0028] Please refer to Figure 1 In this embodiment, by combining local feature modeling of image patches and global feature modeling of graph neural networks, the problems of excessive local enhancement and global imbalance in underwater image enhancement are solved. By processing the cosine similarity matrix, the information transmission capability between image patches is further improved, achieving high-quality enhancement of underwater images. The enhancement method based on degraded underwater images can effectively restore details in underwater images, improve color consistency and brightness balance, and is particularly suitable for complex underwater environments.

[0029] First, underwater images are acquired from the underwater environment. Since underwater images typically suffer from color distortion, low contrast, and blurred details, a deep learning model is needed to perform preliminary feature extraction.

[0030] Preliminary feature extraction is performed using the shallow feature map (SFM) layer of a convolutional neural network. This process extracts low-level features (such as edges and textures) from the image through convolutional layers, standardizes the features using batch normalization, and introduces non-linearity using the ReLU activation function, which can enhance local features such as brightness and contrast of the image.

[0031] This provides a stable feature base for subsequent image enhancement, especially avoiding information loss due to poor image quality (e.g., low contrast and color distortion in underwater images).

[0032] After initial feature extraction, in order to better capture details in the image and reduce problems such as water scattering and uneven lighting, the image is segmented into several small image patches.

[0033] Image segmentation techniques divide the initial feature map of an image into multiple smaller regions (e.g., 16×16 pixels). Each image block represents a part of the image and contains the local features of that region.

[0034] These image patches, serving as nodes in a graph neural network, enable subsequent graph neural networks to process local features and dependencies between regions, avoiding the disconnect between global features and local details.

[0035] To further extract depth features from the image, a neural network model is used to perform multi-level feature encoding and extraction on each segmented image block.

[0036] The feature extraction units (e.g., DenseBlock3) in a densely connected convolutional network (DenseNet121) are used to encode features at multiple levels for each image patch. Through densely connected convolutional layers, the network can connect the output of each layer with the outputs of all previous layers, effectively capturing detailed information of the image patch.

[0037] Multi-level feature coding methods can extract key information such as color, texture, and edges from images, and enhance the transmission of information flow through the connections between layers.

[0038] By using neural networks to extract features from image patches at multiple levels, the details and global structure of underwater images can be better recovered.

[0039] After multi-level feature extraction, the feature vector of each image patch is obtained. Next, based on these features, the cosine similarity matrix between image patches is calculated to measure the similarity between the image patches.

[0040] The cosine similarity matrix helps graph neural networks determine which image patches have stronger similarities during subsequent feature propagation, so as to better convey information.

[0041] By calculating the cosine similarity matrix, the spatial and semantic similarity between image patches can be captured, providing an important basis for subsequent global enhancement.

[0042] Finally, the cosine similarity matrix is ​​processed to generate an adjacency matrix with prior information, which serves as the input to the graph neural network for subsequent image enhancement.

[0043] By sparsifying the Top-k selection, each node is retained to have its k most similar neighbors, while the connection weights between other nodes are reset to zero.

[0044] By removing self-loops, the connection between an image patch and itself is removed, thus preventing information transfer between nodes and themselves.

[0045] By symmetrizing the similarity matrix, we can ensure that the connections between image patches are bidirectional.

[0046] The similarity matrix is ​​normalized to ensure that the numerical range of the matrix is ​​reasonable, so that the graph neural network can propagate features more efficiently.

[0047] By processing the cosine similarity matrix through sparsification, symmetry, and normalization, the information propagation capability of the graph neural network can be effectively enhanced, thereby improving the image enhancement effect.

[0048] An adjacency matrix is ​​used to represent the connection relationships between nodes in a graph. In this embodiment, the adjacency matrix represents the semantic association strength between image blocks.

[0049] Cosine similarity measures the similarity between two vectors by calculating the cosine of the angle between them.

[0050] Top-K selection: a sparsity reduction method that retains only the top K largest values ​​and sets the rest to zero.

[0051] It should be noted that the adjacency matrix can be constructed from, but is not limited to, the following methods: using other backbone networks such as ResNet[1], Vision Transformer[2] to extract node features; using other similarity measurement methods (such as negative exponent of Euclidean distance, Mahalanobis distance) or learning-based measurement networks to calculate inter-block associations; and using superpixel-based segmentation methods to replace rule-based segmentation, with units that better conform to semantic boundaries as graph nodes.

[0052] The state space model module can be replaced with other efficient long sequence modeling modules, such as modern variants of linear attention (LinearAttention[3]), recurrent neural networks (such as LSTM, GRU), or lighter-weight variants of Transformer (such as Performer[4], Linformer[5]). The graph attention network (GAT) layer can be replaced with other graph neural network layers, such as graph convolutional networks (GCN), graph adoption and aggregation (GraphSAGE), etc., to achieve domain information aggregation.

[0053] In at least one embodiment of this application, the specific steps of acquiring an underwater image and performing preliminary feature extraction on the underwater image to obtain a preliminary extracted feature map include: S201. The underwater image is convolved and normalized through two shallow feature mapping layers to extract shallow features from the underwater image and obtain a preliminary extracted feature map.

[0054] Please refer to Figure 2 In this embodiment, firstly, the underwater image is subjected to convolution and normalization operations based on an activation function through two shallow feature mapping layers. The convolution operation is used to extract low-level features in the underwater image, such as edges, textures, and local brightness variations, while the normalization operation helps to eliminate differences in brightness and contrast in the image, ensuring that the network can be trained stably.

[0055] Through the processing of these two shallow feature mapping layers, a preliminary extracted feature map is finally obtained, which contains the basic feature information of the underwater image. These features will be used as input for subsequent image enhancement processing.

[0056] The lighting and contrast of underwater images are often affected by plankton, minerals, and the water itself, leading to distortion of color and texture. Therefore, extracting shallow features first can help identify and capture local details (such as edges, textures, and brightness distribution), laying the foundation for subsequent deeper feature extraction.

[0057] Normalization helps to address brightness differences in different regions of an image, making the feature extraction process more stable. Because the brightness and color in underwater images often vary unevenly due to absorption and scattering by water, normalization helps to eliminate the effects of these lighting and color variations, thereby improving image processing performance.

[0058] The initial feature maps extracted through two shallow feature mapping layers provide crucial information for the subsequent training of the neural network. These feature maps will serve as the foundation for the network to learn more complex deep features (such as global structure, color consistency, and detail restoration), helping to improve the accuracy of image enhancement.

[0059] By performing convolution and normalization operations on underwater images through shallow feature mapping layers, the influence of the underwater environment on image quality can be effectively eliminated, the training process of the network can be stabilized, and the processing instability caused by uneven lighting and color can be avoided.

[0060] Shallow feature extraction is the first step in image enhancement. It can effectively capture the basic structure and local details in an image, providing more accurate and richer input features for subsequent image detail restoration and global feature modeling. This makes the enhanced image more delicate and natural, avoiding artifacts or color distortion.

[0061] Through preliminary feature extraction, neural networks can learn the basic features of underwater images in a relatively short time, laying the foundation for deeper feature learning and complex image enhancement tasks, thereby improving the efficiency and effectiveness of image enhancement.

[0062] In at least one embodiment of this application, the specific steps of segmenting the initially extracted feature map to obtain segmented image patches include: S202. The preliminary extracted feature map is divided into several image blocks to form a segmented image set.

[0063] Please refer to Figure 2 In this embodiment, the initially extracted feature map is divided into several image blocks to form a segmented image set. Each image block represents a small region in the image, usually a sub-image of a fixed size (e.g., 16×16 pixels). These image blocks will be processed as nodes in the graph neural network.

[0064] By breaking down an image into multiple small regions, the network can independently process and enhance each local region, thereby capturing more fine details in the image.

[0065] By dividing the initially extracted feature map into multiple small blocks, the focus can be placed on the detailed processing of local regions. Each image block, as a separate unit, can be processed independently by the graph neural network. This avoids uniform processing of the entire image, allowing the network to better capture detailed features of the image within a local scope, such as textures, edges, and details of small objects in underwater images.

[0066] Underwater images are often affected by factors such as scattering and uneven illumination, leading to quality differences in different regions. By dividing the image into small patches and processing them separately, the problems of over- or under-enhancement that may occur during global processing can be avoided. This allows each region in the image to be enhanced independently, improving the accuracy and effectiveness of image enhancement.

[0067] Dividing an image into multiple smaller blocks allows the neural network to process multiple image blocks in parallel, greatly improving computational efficiency. In processing high-resolution images, segmenting the image into blocks can also significantly reduce the computational load, enabling this method to handle larger-scale images or higher-resolution underwater images.

[0068] In at least one embodiment of this application, the specific steps of using a neural network model to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features include: S203. Based on the feature extraction unit in the densely connected convolutional network, multi-level feature encoding is performed on each image block in the segmented image block to obtain the encoded image set. Multi-level feature extraction is performed on the encoded image set to obtain the extracted features.

[0069] Please refer to Figure 2 In this embodiment, based on the feature extraction unit in the DenseNet121 convolutional network, multi-level feature encoding is performed on each segmented image block to obtain the encoded image set. The DenseNet121 convolutional network uses dense connections to ensure that the output of each layer in the network is connected to the output of all previous layers, thereby better extracting multi-level feature information.

[0070] Specifically, densely connected convolutional networks encode image patches at multiple levels through multiple feature extraction units (such as DenseBlock3), gradually extracting key information such as color, texture, and edges from the image.

[0071] Multi-level feature extraction is performed on the encoded image set to obtain the extracted features. The neural network model can further capture high-level features in image patches, thereby providing richer image features for subsequent image enhancement and restoration.

[0072] Since underwater images often suffer from blurriness and loss of detail, using the feature extraction unit of a densely connected convolutional network to perform multi-level feature encoding on each image block can effectively restore image details, especially for details such as edges and textures in underwater images, thereby improving image clarity and accuracy.

[0073] By extracting features from multiple image patches at multiple levels, the overall image enhancement effect is more balanced, avoiding the problems of excessive or inconsistent local enhancement that occur in traditional image enhancement methods. In underwater images, color consistency and overall brightness are effectively balanced.

[0074] By using the feature extraction units in a densely connected convolutional network (DenseNet) to perform multi-level feature encoding and extraction on segmented image patches, the detailed information of underwater images can be recovered efficiently and accurately, thus improving image quality.

[0075] By using deep feature encoding, information loss is avoided, image details and global consistency are enhanced, and some limitations of traditional image enhancement methods are solved, especially in complex backgrounds and image degradation in underwater environments.

[0076] Underwater images often contain complex phenomena such as various lighting conditions, light refraction, and scattering. This multi-level feature extraction method can effectively enhance the network's adaptability to these complex scenes. Through multi-level feature extraction, the network can capture the complex relationships at different levels in underwater images, thereby improving the robustness of the enhancement effect.

[0077] In at least one embodiment of this application, the specific steps of performing multi-level feature extraction on the encoded image set to obtain the extracted features include: S204. Extract color and texture from the encoded image set to obtain extracted color and texture features and generate composite features. S205. Calculate the cosine similarity between any two image blocks in the segmented image block based on the composite features, and generate a cosine similarity matrix.

[0078] Please refer to Figure 2 In this embodiment, the encoded image set is further processed by a feature extraction module to extract color and texture features specifically for the images. Color features include information such as hue, saturation, and brightness, while texture features focus on the details, surface structure, and patterns of the images.

[0079] The extracted color and texture features are combined to form composite features, which provide richer feature information for subsequent image patch similarity calculations. For each pair of image patches, the cosine similarity between them is calculated based on the composite features of the image patches. The calculated cosine similarity matrix provides the relationship and structural information between the image patches for subsequent graph neural network processing.

[0080] The process of extracting color and texture features allows the network to focus on the key visual features of an image. Color features help restore color consistency in an image, especially the color distortion problem that is common in underwater images; texture features help restore details in an image, especially in areas where details are blurred or have low contrast.

[0081] The generation of composite features enables the network to utilize information from different features (color and texture) when performing image enhancement, thereby enhancing the multi-layered details of the image, especially in complex underwater environments.

[0082] By calculating the cosine similarity of composite features, the similarity between image patches can be quantified. This helps to capture the relationships between image patches and provides a structured adjacency matrix for graph neural networks, thereby improving the efficiency and accuracy of information propagation in the image enhancement process.

[0083] Composite features combine color and texture information, enabling the similarity calculation between image patches to not only rely on a single feature, but also enhance the model's ability to perceive image details through the combination of multiple features. This helps the network provide more accurate and subtle enhancements when capturing global information and local details.

[0084] In at least one embodiment of this application, the specific steps of processing the cosine similarity matrix to obtain an edge feature adjacency matrix with prior information include: The cosine similarity matrix is ​​sparsified, symmetricized, normalized, and self-loop removed to obtain the adjacency matrix of the graph mechanism.

[0085] Please refer to Figure 2 In this implementation, the cosine similarity matrix is ​​sparsified by selecting the k most similar neighbors of each node (image patch), retaining connections with high similarity, and resetting the weights of other connections to zero. This aims to reduce computational complexity and ensure that the graph neural network focuses only on the most important relationships.

[0086] The cosine similarity matrix is ​​symmetricized to ensure consistency between each pair of elements. This is typically achieved by taking smaller values, making the similarity relationship between image patches bidirectionally symmetric, thus enhancing the propagation capability of the graph neural network.

[0087] The cosine similarity matrix is ​​processed to remove self-loops, that is, the elements on the diagonal (the similarity between an image patch and itself) are set to zero. This is because in graph neural networks, nodes should not propagate information to themselves; removing self-loops avoids unnecessary computation and information redundancy.

[0088] The similarity matrix after removing self-loops and symmetry is normalized by normalizing the matrix elements column-wise (or row-wise) so that the sum of each column equals 1. Normalization helps balance the information propagation between image patches, preventing some nodes from excessively influencing other nodes, thus ensuring more stable and efficient training of the graph neural network.

[0089] The original cosine similarity matrix is ​​transformed into an adjacency matrix with prior information, providing an optimized graph structure for the graph neural network as input in the subsequent image enhancement process.

[0090] By sparsifying the adjacency matrix, only the connections between the most relevant image patches are retained, reducing unnecessary computation. Especially when processing large-scale images, sparsification significantly reduces the computational resource requirements, making the entire method more efficient and enabling real-time image enhancement.

[0091] By symmetry processing, the relationships between image patches are ensured to be bidirectional, making information transmission between image patches more efficient. After removing self-loops, the connections between an image patch and itself are cleared, avoiding information redundancy and further optimizing the propagation process of the graph neural network.

[0092] Normalization ensures a balance in information propagation between image patches, preventing any single patch from excessively influencing the final enhancement. This allows graph neural networks to train and infer more stably when processing complex underwater images, improving the consistency and accuracy of the enhancement results.

[0093] This allows the adjacency matrix to better reflect the similarity relationships between image patches, providing more accurate input for subsequent feature transfer and enhancement in the graph neural network. Ultimately, the enhanced underwater image quality is significantly improved, with clearer details and restored color consistency.

[0094] By performing sparsification, self-loop removal, symmetry conversion, and normalization on the cosine similarity matrix, an adjacency matrix with prior information is generated, providing an optimized graph structure for the graph neural network. This effectively improves computational efficiency, enhances the effectiveness of information propagation, optimizes network training stability, and ultimately improves the accuracy and quality of underwater images.

[0095] In at least one embodiment of this application, the specific steps of performing sparsification, symmetry, normalization, and self-loop removal on the cosine similarity matrix to obtain the adjacency matrix of the graph mechanism include: S206. The cosine similarity matrix is ​​sparsified to generate a sparsified similarity matrix. S207. Perform self-loop removal processing on the sparsified similarity matrix to generate a similarity matrix after self-loop removal; S208. The similarity matrix after removing self-loops is symmetricized to obtain a symmetric similarity matrix; S209. Normalize the symmetric similarity matrix to generate the adjacency matrix of the graph mechanism, and use the adjacency matrix as the input of the neural network.

[0096] Please refer to Figure 2 In this embodiment, the cosine similarity matrix is ​​sparsified by retaining only the k most similar neighbors of each image patch. That is, for each image patch, the k connections with the highest similarity to the other image patches are retained, and other similarity values ​​are set to zero. This makes the graph structure sparser and reduces unnecessary computation.

[0097] In the sparsified similarity matrix, the elements on the diagonal (i.e., the similarity between an image patch and itself) are set to zero. Since the connection between an image patch and itself has no practical meaning for information transmission, removing self-loops can effectively avoid useless calculations and redundancy.

[0098] The similarity matrix after removing self-loops is symmetricized to ensure bidirectional relationships between image patches. For each pair of similarity values, a smaller or larger value is selected to ensure the consistency of the graph structure, making the similarity propagation between image patches more symmetrical and enhancing the bidirectional flow of information.

[0099] Normalizing the symmetric similarity matrix typically involves normalizing it column-wise, ensuring that the sum of the elements in each column is 1. This helps balance information propagation between image patches, preventing excessive influence from certain patches and ensuring more stable training and inference for the graph neural network.

[0100] The resulting final matrix is ​​called the adjacency matrix, which represents the similarity and relationships between image patches. This adjacency matrix is ​​then fed as input to the neural network for subsequent image enhancement processes.

[0101] By sparsifying the image, the connections between image patches become sparser, reducing unnecessary computations. This is especially beneficial when processing high-resolution images, as it greatly improves the computational efficiency and speed of the model and reduces the demand for computing resources.

[0102] By removing self-loops, symmetry, and normalization, the effectiveness and stability of information propagation between image patches are ensured. This ensures that the similarity between image patches is accurately transmitted, improving the overall quality of the image.

[0103] This ensures that the graph neural network is not affected by noise and redundant information when propagating information between image blocks, thereby enhancing the stability and consistency of image enhancement. The final enhanced image output is effectively improved in terms of color, contrast and detail.

[0104] By removing self-loops and sparsifying operations, redundant information between image patches is reduced, enabling graph neural networks to focus more effectively on meaningful relationships between image patches and avoiding unnecessary computational complexity.

[0105] Normalization and symmetry operations help the network maintain a balance of information when processing images, making the image enhancement results more natural and realistic, avoiding over-enhancement or imbalance. This is especially important for underwater images, which are prone to color distortion and blurred details.

[0106] An optimized adjacency matrix was generated by sparsifying, removing self-loops, symmetricizing, and normalizing the cosine similarity matrix. This effectively improves the connectivity between image patches, reduces redundant information, enhances the stability and computational efficiency of the image enhancement process, and ultimately improves the quality of underwater image enhancement. It addresses the shortcomings of traditional methods in image patch relationship modeling, can handle more complex underwater environments, and provides more accurate and efficient image enhancement results.

[0107] An underwater image enhancement device 10 is applied in the underwater image enhancement method as described in any of the preceding claims, wherein the underwater image enhancement device 10 comprises: Image acquisition module 11 is used to acquire underwater images; The preliminary feature extraction module 12 is used to perform preliminary feature extraction on underwater images to obtain a preliminary extracted feature map; Segmentation module 13 is used to segment the initially extracted feature map to obtain segmented image blocks; Neural network model 14 is used to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features; The cosine similarity calculation module 15 calculates the cosine similarity matrix based on the extracted features; Processing module 16 processes the cosine similarity matrix to generate an adjacency matrix; The enhancement device based on underwater degraded images is used to perform the following steps: Acquire underwater images and perform preliminary feature extraction on the underwater images to obtain preliminary extracted feature maps; The initially extracted feature map is segmented to obtain segmented image blocks; A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features; Calculate the cosine similarity matrix based on the extracted features; The cosine similarity matrix is ​​processed to obtain an adjacency matrix with prior information.

[0108] Please refer to Figure 3In this embodiment, the image acquisition module 11 first acquires an externally input underwater image, and then the preliminary feature extraction module 12 performs preliminary feature extraction on the underwater image to obtain a preliminary extracted feature map; the segmentation module 13 performs image segmentation on the preliminary extracted feature map to obtain segmented image blocks; the neural network model 14 performs multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain extracted features; the cosine similarity calculation module 15 calculates the cosine similarity matrix based on the extracted features; and the processing module 16 performs sparsification, self-loop removal, symmetry, and normalization processing on the cosine similarity matrix to generate an adjacency matrix.

[0109] By combining local feature modeling of image patches with global feature modeling of graph neural networks, the problems of local over-enhancement and global imbalance in underwater image enhancement are solved. Further processing of the cosine similarity matrix enhances the information transfer capability between image patches, achieving high-quality underwater image enhancement. The enhancement method based on degraded underwater images can effectively restore details, improve color consistency and brightness balance, and is particularly suitable for complex underwater environments.

[0110] A computer device, characterized in that it includes a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the following steps: Acquire underwater images and perform preliminary feature extraction on the underwater images to obtain preliminary extracted feature maps; The initially extracted feature map is segmented to obtain segmented image blocks; A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image block to obtain the extracted features; Calculate the cosine similarity matrix based on the extracted features; The cosine similarity matrix is ​​processed to obtain an adjacency matrix with prior information.

[0111] Please refer to Figure 4 In this embodiment, by combining local feature modeling of image patches and global feature modeling of graph neural networks, the problems of excessive local enhancement and global imbalance in underwater image enhancement are solved. By processing the cosine similarity matrix, the information transmission capability between image patches is further improved, achieving high-quality enhancement of underwater images. The enhancement method based on degraded underwater images can effectively restore details in underwater images, improve color consistency and brightness balance, and is particularly suitable for complex underwater environments.

[0112] Figure 4An internal structural diagram of a computer device in one embodiment is shown. This computer device can specifically be a terminal or a server. Figure 4 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program that, when executed by the processor, enables the processor to implement an enhancement method based on underwater degraded images. The internal memory may also store a computer program that, when executed by the processor, enables the processor to implement an enhancement method based on underwater degraded images. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0113] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps: Acquire underwater images and perform preliminary feature extraction on the underwater images to obtain preliminary extracted feature maps; The initially extracted feature map is segmented to obtain segmented image blocks; A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image block to obtain the extracted features; Calculate the cosine similarity matrix based on the extracted features; The cosine similarity matrix is ​​processed to obtain an adjacency matrix with prior information.

[0114] Please refer to Figure 4 In this embodiment, by combining local feature modeling of image patches and global feature modeling of graph neural networks, the problems of excessive local enhancement and global imbalance in underwater image enhancement are solved. By processing the cosine similarity matrix, the information transmission capability between image patches is further improved, achieving high-quality enhancement of underwater images. The enhancement method based on degraded underwater images can effectively restore details in underwater images, improve color consistency and brightness balance, and is particularly suitable for complex underwater environments.

[0115] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An enhancement method based on underwater degraded images, characterized in that, The enhancement method based on underwater degraded images includes: Acquire underwater images and perform preliminary feature extraction on the underwater images to obtain preliminary extracted feature maps; The initially extracted feature map is segmented to obtain segmented image blocks; A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features; Calculate the cosine similarity matrix based on the extracted features; The cosine similarity matrix is ​​processed to obtain an adjacency matrix with prior information.

2. The enhancement method based on underwater degraded images according to claim 1, characterized in that, The specific steps for acquiring underwater images and performing preliminary feature extraction on the underwater images to obtain preliminary extracted feature maps include: The underwater image is convolved and normalized by two shallow feature mapping layers to extract shallow features from the underwater image, thus obtaining a preliminary extracted feature map.

3. The enhancement method based on underwater degraded images according to claim 1, characterized in that, The specific steps for segmenting the initially extracted feature map to obtain segmented image blocks include: The initially extracted feature map is divided into several image blocks to form a segmented image set.

4. The enhancement method based on underwater degraded images according to claim 1, characterized in that, The specific steps of using a neural network model to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features include: The feature extraction unit in the densely connected convolutional network performs multi-level feature encoding on each image block in the segmented image block to obtain the encoded image set. Multi-level feature extraction is performed on the encoded image set to obtain the extracted features.

5. The enhancement method based on underwater degraded images according to claim 4, characterized in that, The specific steps for performing multi-level feature extraction on the encoded image set to obtain the extracted features include: The encoded image set is subjected to color and texture extraction to obtain the extracted color and texture features, and composite features are generated. Based on the composite features, the cosine similarity between any two image blocks in the segmented image block is calculated, and a cosine similarity matrix is ​​generated.

6. The enhancement method based on underwater degraded images according to claim 1, characterized in that, The specific steps for processing the cosine similarity matrix to obtain an edge feature adjacency matrix with prior information include: The cosine similarity matrix is ​​sparsified, symmetricized, normalized, and self-loop removed to obtain the adjacency matrix of the graph mechanism.

7. The enhancement method based on underwater degraded images according to claim 6, characterized in that, The specific steps for obtaining the adjacency matrix of the graph mechanism by performing sparsification, symmetry, normalization, and self-loop removal on the cosine similarity matrix include: The cosine similarity matrix is ​​sparsified to generate a sparser similarity matrix. The sparsified similarity matrix is ​​subjected to self-loop removal processing to generate a similarity matrix after self-loop removal; The similarity matrix after removing self-loops is symmetricized to obtain a symmetric similarity matrix; The symmetric similarity matrix is ​​normalized to generate the adjacency matrix of the graph mechanism, and the adjacency matrix is ​​used as the input of the neural network.

8. An enhancement device based on underwater degraded images, applied in the underwater degraded image enhancement method as described in any one of claims 1-7, characterized in that, The enhancement device based on underwater degraded images includes: The image acquisition module is used to acquire underwater images; The preliminary feature extraction module is used to perform preliminary feature extraction on underwater images to obtain a preliminary feature map; The segmentation module is used to segment the initially extracted feature map into image blocks. A neural network model is used to perform multi-level feature encoding and extraction on each image block in the segmented image blocks to obtain the extracted features. The cosine similarity calculation module calculates the cosine similarity matrix based on the extracted features; The processing module processes the cosine similarity matrix to generate an adjacency matrix.

9. A computer-readable storage medium, characterized in that, The device stores a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

Images