Lightweight underwater image enhancement method, apparatus and device based on semi-supervised inactive function, and storage medium

Abstract

The invention discloses a lightweight underwater image enhancement method, device and equipment based on a semi-supervised inactive function, and a storage medium, and relates to the technical field of underwater image processing. The lightweight underwater image enhancement method based on the semi-supervised inactive function comprises the following steps: constructing an initial image enhancement model based on a teacher-student framework of semi-supervised comparative learning and a gradient inactive function network; training the initial image enhancement model through label data and label-free data to obtain a target image enhancement model; obtaining an underwater image to be enhanced; and inputting the underwater image into the target image enhancement model to obtain an enhanced image. According to the method, the image can be efficiently processed, the cross-scene adaptive capacity of underwater image enhancement can be improved, the model deployment cost is reduced, and powerful technical support is provided for applications such as smart fishery.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater image processing, in particular to a lightweight underwater image enhancement method and device based on semi-supervised activation function-free. BACKGROUND

[0002] In the marine farming scenario, the images captured by underwater cameras often have problems such as color shift, insufficient overall brightness, low contrast, and background turbidity due to the light absorption and scattering properties of seawater. The complexity of the marine scene, including the influence of ocean currents, the difference in light incidence angle, and the difference in farmed fish and water style, further exacerbates the instability of image quality and the difficulty of adapting to a single model. These factors pose many problems and challenges for intelligent fishing applications, and there is an urgent need for a technology that can effectively improve underwater image quality to support intelligent management and monitoring of marine farming.

[0003] Current underwater image enhancement systems mainly include traditional image processing methods and deep learning-based methods. Traditional image processing methods rely on expert knowledge for manual parameter tuning, and improve image quality by adjusting parameters such as brightness, contrast, and color balance. Deep learning-based methods use convolutional neural networks and other models to learn the features of a large amount of labeled data to achieve image enhancement. These methods can improve underwater image quality to some extent, but still have some limitations.

[0004] The shortcomings of existing methods are as follows: on the one hand, traditional image processing methods require expert manual parameter tuning and have poor adaptability, making it difficult to cope with complex marine farming scenarios. On the other hand, deep learning-based methods perform well in image enhancement, but require a large amount of labeled data and high computational resources, making it difficult to adapt to actual deployment environments. In addition, public underwater datasets cannot be generalized to cover all farming scenarios, as each marine net cage farming area has different water environments, fish species, and biological behavior characteristics, and the depth of camera deployment leads to uncertain illumination. Moreover, most open datasets are only suitable for partial typical environments and have poor generalizability. Therefore, how to improve the cross-scene adaptability of underwater image enhancement and reduce model deployment costs has become a problem to be solved.

[0005] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0006] The present application aims to provide a lightweight underwater image enhancement method and device based on semi-supervised activation function-free, which aims to solve the technical problem of how to improve the cross-scene adaptability of underwater image enhancement and reduce model deployment costs.

[0007] To achieve the above object, the application provides a lightweight underwater image enhancement method based on semi-supervised activation function-free, which comprises the following steps:

[0008] A teacher-student framework based on semi-supervised contrast learning and a gradient activation function-free network are used to construct an initial image enhancement model.

[0009] The initial image enhancement model is trained by using labeled data and unlabeled data to obtain a target image enhancement model.

[0010] An underwater image to be enhanced is obtained.

[0011] The underwater image is input into the target image enhancement model to obtain an enhanced image.

[0012] In an embodiment, the initial image enhancement model comprises a teacher model and a student model; the step of training the initial image enhancement model by using labeled data and unlabeled data to obtain a target image enhancement model comprises the following steps: the parameters of the teacher model are updated by using an exponential moving average strategy based on the parameters of the student model; a supervised loss is calculated according to labeled data, and a contrast loss is calculated according to unlabeled data; the supervised loss, a pseudo-label consistency loss and the contrast loss are combined into a joint loss function; the parameters of the student model are optimized in a reverse direction according to the joint loss function, and the step of updating the parameters of the teacher model by using the exponential moving average strategy based on the parameters of the student model is returned until the number of training iterations reaches a preset iteration threshold or the joint loss function converges, and the student model is taken as the target image enhancement model.

[0013] In an embodiment, the step of calculating a supervised loss according to labeled data and a contrast loss according to unlabeled data comprises the following steps: labeled data is input into the student model to calculate a supervised loss comprising pixel-level error, perception error and gradient error; unlabeled data is input into the teacher model to generate an initial pseudo-label, and a target pseudo-label with a quality score higher than a preset threshold is selected from the initial pseudo-label; a contrast sample pair is constructed based on the target pseudo-label and the unlabeled data, and a contrast loss is calculated according to the contrast sample pair.

[0014] In an embodiment, the teacher-student framework based on semi-supervised contrastive learning includes a teacher model and a student model; and the step of constructing an initial image enhancement model by the teacher-student framework based on semi-supervised contrastive learning and the gradient activation-free network includes: initializing network parameters of the teacher model and the student model, and the initial backbone network of the teacher model and the student model is a gradient activation-free network; defining a gradient feature extraction module, a feature fusion module, a minimalist gating mechanism, and a channel attention module in the gradient activation-free network to obtain a target backbone network; establishing a contrastive learning mechanism between the teacher model and the student model according to a pseudo-label generation module and a contrastive loss calculation module; and combining the teacher model, the student model, the target backbone network, the contrastive learning mechanism, and a data preprocessing module to obtain the initial image enhancement model.

[0015] In an embodiment, the target image enhancement model includes an initial convolution module, a gradient feature extraction module, a minimalist gating mechanism, a feature fusion module, and a channel attention module; and the step of inputting the underwater image into the target image enhancement model to obtain an enhanced image includes: performing convolution processing on the underwater image by the gradient feature extraction module and the initial convolution module to obtain gradient amplitude features and original features; performing feature fusion, feature splitting, and weighting processing on the original features and the gradient amplitude features by the feature fusion module, the minimalist gating mechanism, and the channel attention module to obtain weighted gating features; performing residual connection on the weighted gating features and the original features to obtain enhanced features; and performing deconvolution processing on the enhanced features to obtain an enhanced image.

[0016] In an embodiment, the step of performing feature fusion, feature splitting, and weighting processing on the original features and the gradient amplitude features by the feature fusion module, the minimalist gating mechanism, and the channel attention module to obtain weighted gating features includes: performing convolution processing on the original features and the gradient amplitude features by the feature fusion module, and splicing along the channel dimension to obtain fusion features; performing normalization processing and depth separable convolution on the fusion features to obtain local spatial features; splitting the local spatial features into first sub-features and second sub-features by the minimalist gating mechanism, and performing element-wise multiplication to obtain gating features; and performing weighting processing on the gating features based on attention weights of the channel attention module to obtain weighted gating features.

[0017] In an embodiment, the step of performing convolution processing on the underwater image by the gradient feature extraction module and the initial convolution module to obtain gradient amplitude features and original features comprises: performing feature extraction on the underwater image by horizontal and vertical convolution kernels in the gradient feature extraction module to obtain horizontal and vertical gradient components; performing square and square root operations on the horizontal and vertical gradient components to generate gradient amplitude features; and performing convolution processing on the underwater image by the initial convolution module to obtain original features.

[0018] In addition, to achieve the above object, the present application further provides a lightweight underwater image enhancement device based on semi-supervised non-activation function, which comprises:

[0019] a model construction module, configured to construct an initial image enhancement model based on a teacher-student framework of semi-supervised contrast learning and a gradient non-activation function network;

[0020] a model training module, configured to train the initial image enhancement model by using labeled data and unlabeled data to obtain a target image enhancement model;

[0021] an image acquisition module, configured to acquire an underwater image to be enhanced;

[0022] an image enhancement module, configured to input the underwater image into the target image enhancement model to obtain an enhanced image.

[0023] In addition, to achieve the above object, the present application further provides a lightweight underwater image enhancement device based on semi-supervised non-activation function, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the lightweight underwater image enhancement method based on semi-supervised non-activation function as described above.

[0024] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and a computer program is stored in the storage medium, wherein the computer program is executed by a processor to implement the steps of the lightweight underwater image enhancement method based on semi-supervised non-activation function as described above.

[0025] In addition, to achieve the above object, the present application further provides a computer program product, which comprises a computer program, wherein the computer program is executed by a processor to implement the steps of the lightweight underwater image enhancement method based on semi-supervised non-activation function as described above.

[0026] The one or more technical solutions provided by the present application have at least the following technical effects:

[0027] First, an initial image enhancement model is constructed based on a semi-supervised contrastive learning teacher-student framework and a gradient activation-free network. This architecture combines the advantages of semi-supervised learning and the characteristics of lightweight network structure. The teacher model provides high-quality pseudo labels, and the student model uses these pseudo labels and labeled data for joint training to improve the generalization ability of the model. At the same time, the design of the gradient activation-free network reduces the consumption of computing resources, improves the real-time performance and adaptability of the model. Second, the initial image enhancement model is trained using labeled data and unlabeled data to obtain the target image enhancement model. This training process makes full use of limited labeled data and a large amount of unlabeled data to further improve the generalization performance of the model, enabling it to work effectively in various complex underwater environments. Then, the underwater images to be enhanced are obtained, which are usually directly collected from underwater cameras and may have poor quality. Finally, the underwater images are input into the target image enhancement model to obtain enhanced images. The entire process not only efficiently processes images but also improves the cross-scene adaptability of underwater image enhancement and reduces the deployment cost of the model, providing strong technical support for applications such as smart fisheries. BRIEF DESCRIPTION OF DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0030] Figure 1 A flowchart provided by the first embodiment of the semi-supervised activation-free lightweight underwater image enhancement method of the present application is shown in the figure.

[0031] Figure 2 An underwater image enhancement network architecture under a semi-supervised learning framework provided by the first embodiment of the semi-supervised activation-free lightweight underwater image enhancement method of the present application is shown in the figure.

[0032] Figure 3 A flowchart provided by the second embodiment of the semi-supervised activation-free lightweight underwater image enhancement method of the present application is shown in the figure.

[0033] Figure 4 A module structure diagram of the semi-supervised activation-free lightweight underwater image enhancement device of the present application is shown in the figure.

[0034] Figure 5A device structure schematic diagram of a hardware running environment involved in a semi-supervised activation function-free lightweight underwater image enhancement method in the embodiments of the present application.

[0035] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0036] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and do not limit the present application.

[0037] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below with reference to the drawings in the specification.

[0038] In the marine farming scene, the images captured by underwater cameras often have color deviation, insufficient brightness, low contrast and background turbidity due to the light absorption and scattering properties of seawater and the complexity of marine scenes, which brings many challenges to smart fishery applications, and there is an urgent need for a technology that can effectively improve the quality of underwater images. The current underwater image enhancement system mainly includes traditional image processing methods and deep learning-based methods. However, traditional methods have poor adaptability and are difficult to cope with complex scenes; deep learning methods require a large amount of labeled data and high computing resources, and the public data set has poor generalization, making it difficult to cover all farming scenes.

[0039] The main solution of the embodiments of the present application is to construct an initial image enhancement model based on a semi-supervised contrast learning teacher-student framework and a gradient activation function-free network, train it using labeled and unlabeled data, obtain a target image enhancement model, and then perform enhancement processing on the obtained underwater images. This process can improve the cross-scene adaptability of underwater image enhancement and reduce the model deployment cost, providing efficient support for smart fishery and other applications.

[0040] It should be noted that the execution subject of the embodiments of the present application can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, an image processing device, etc. that can realize the above functions. The embodiments and the following embodiments will be described below with the image processing device as an example.

[0041] Based on this, the embodiments of the present application provide a semi-supervised activation function-free lightweight underwater image enhancement method, which is described in detail with reference to Figure 1 , Figure 1 A flowchart of the first embodiment of the semi-supervised activation function-free lightweight underwater image enhancement method of the present application.

[0042] In this embodiment, the lightweight underwater image enhancement method based on semi-supervised activation function-free includes steps S10-S40.

[0043] Step S10, based on the teacher-student framework of semi-supervised contrast learning and gradient activation function-free network, an initial image enhancement model is constructed.

[0044] It should be noted that the teacher-student framework based on semi-supervised contrast learning is a training architecture for improving model performance. In this framework, the teacher model and the student model cooperate with each other. The teacher model is updated from the student model by EMA (Exponential Moving Average), maintains a stable learning goal, and thus provides guidance for the student model. The student model uses labeled data for supervised learning and combines pseudo-labels generated from unlabeled data for consistency training. This framework combines the advantages of supervised learning and unsupervised learning, and can fully utilize unlabeled data to improve the generalization ability and performance of the model in the case of limited labeled data.

[0045] Pseudo-labels are generated by the teacher model to guide the training of the student model in the semi-supervised learning framework. In the absence of real labels, the teacher model predicts the unlabeled data based on its existing knowledge, and the prediction results are pseudo-labels. These pseudo-labels are used as temporary "real" labels for the student model to help the student model learn from unlabeled data.

[0046] The gradient activation function-free network (Grad-NAFNet) is a lightweight neural network structure specially designed for underwater image enhancement tasks. By optimizing the network structure and calculation path, it reduces the computational power consumption and improves the enhancement effect and real-time performance of the original image. The core feature of Grad-NAFNet is the introduction of gradient feature extraction mechanism, which extracts the horizontal and vertical direction gradients of input features in parallel, and combines the main feature convolution and gradient feature convolution to realize high-fidelity enhancement of image edge structure.

[0047] The initial image enhancement model refers to the underwater image enhancement system constructed based on the above-mentioned teacher-student framework and gradient activation function-free network. This model combines semi-supervised contrast learning and lightweight network structure, and can effectively enhance underwater images without the need for a large amount of labeled data. The model generates high-quality pseudo-labels through the teacher model, and uses these pseudo-labels together with labeled data to optimize the student model, thereby improving the generalization ability and enhancement effect of the model.

[0048] As an example, the semi-supervised contrastive learning-based teacher-student framework includes a teacher model and a student model; the steps of constructing an initial image enhancement model by the semi-supervised contrastive learning-based teacher-student framework and the gradient non-activation function network include: initializing network parameters of the teacher model and the student model, initial backbone networks of the teacher model and the student model are both gradient non-activation function networks; defining a gradient feature extraction module, a feature fusion module, a minimalist gating mechanism, and a channel attention module in the gradient non-activation function network to obtain a target backbone network; establishing a contrastive learning mechanism between the teacher model and the student model according to a pseudo-label generation module and a contrastive loss calculation module; combining the teacher model, the student model, the target backbone network, the contrastive learning mechanism, and a data preprocessing module to obtain an initial image enhancement model.

[0049] The teacher model is a key component in the semi-supervised contrastive learning-based teacher-student framework, which updates from the student model through EMA, maintains a stable learning goal, and provides high-quality pseudo-labels for the student model.

[0050] The student model is another key component in the teacher-student framework, responsible for directly processing input data and learning. It is trained through a combination of supervised learning (using labeled data) and unsupervised learning (using pseudo-labels generated by the teacher model). The network structure of the student model is continuously optimized during training to improve its enhancement effect on underwater images.

[0051] Network parameters refer to trainable weights and biases in neural networks, which determine the mapping relationship between the input and output of the network. In the teacher-student framework, both the teacher model and the student model have their own network parameters, which are updated through optimization algorithms (such as gradient descent) during training to minimize the loss function and improve model performance.

[0052] The initial backbone network refers to the basic network structure adopted by the teacher model and the student model at the beginning of training. In this embodiment, the initial backbone network is a gradient non-activation function network, which provides a lightweight and efficient starting point for the model for subsequent training and optimization.

[0053] The gradient feature extraction module is a key component in Grad-NAFNet, responsible for extracting horizontal and vertical gradient information from input features. Fixed convolution kernels are used to extract horizontal and vertical gradient information from input features, respectively. These gradient information helps to enhance the edge structure of the image, thereby improving the clarity and contrast of the image.

[0054] The feature fusion module is another important component in Grad-NAFNet, responsible for fusing the main features and gradient features. By concatenating the main features and gradient features along the channel dimension, then performing normalization and convolution operations, the feature fusion module can generate more rich feature representations, thus improving the model's ability to capture image details.

[0055] The SimpleGate mechanism is a lightweight module in Grad-NAFNet used to control the flow of features. It divides the input features into two along the channel dimension and then element-wise multiplies them to generate gated features. This mechanism can dynamically adjust the importance of features, enhancing the model's ability to express key features while maintaining computational efficiency.

[0056] The SCA (Simple Channel Attention) module is an attention mechanism module in Grad-NAFNet. It generates channel attention weights through global average pooling and Sigmoid activation functions, then weights the features. This module can highlight important channel features and suppress unimportant features, thus improving the model's ability to understand image content.

[0057] The target backbone network refers to the optimized and improved Grad-NAFNet, which integrates gradient feature extraction module, feature fusion module, SimpleGate mechanism and channel attention module. The synergistic effect of these modules enables the target backbone network to more effectively handle underwater image enhancement tasks under lightweight conditions, improving the fidelity of image edge structure and overall quality.

[0058] The pseudo-label generation module is part of the teacher model, responsible for generating pseudo-labels based on unlabeled data.

[0059] The contrastive loss calculation module is a component used to calculate the contrastive loss. It constructs positive and negative sample pairs between the original image and the pseudo-enhanced image, and calculates the contrastive loss. This loss function can further improve the fidelity of the network structure, ensuring that the enhanced image is consistent with the original image in structure and semantics.

[0060] The contrastive learning mechanism is a core mechanism in the teacher-student framework based on semi-supervised contrastive learning. It uses the difference between the pseudo-labels generated by the teacher model and the output of the student model to optimize the parameters of the student model through the contrastive loss calculation module.

[0061] Firstly, the image processing device initializes the network parameters of the teacher model and the student model, ensuring that the initial backbone networks of both models are Grad-NAFNet, which provides a unified and efficient starting point for subsequent training, while ensuring that the models have the same structure and performance basis in the initial stage. Then, the image processing device defines the gradient feature extraction module, feature fusion module, minimal gate mechanism and channel attention module in the gradient activation function network, and optimizes and improves the network structure through the synergistic effect of these modules, so as to obtain the target backbone network. This process aims to enhance the network's ability to capture image edge structures and detailed features, and improve the effect and quality of image enhancement. Then, the image processing device establishes a contrast learning mechanism between the teacher model and the student model according to the pseudo-label generation module and the contrast loss calculation module. Specifically, the teacher model uses the pseudo-label generation module to generate high-quality pseudo-labels for unlabeled data, and the student model learns by comparing its output with the pseudo-labels generated by the teacher model, and further optimizes its network parameters using contrast loss. This mechanism can effectively improve the generalization ability and learning efficiency of the student model. Finally, the image processing device combines the teacher model, student model, target backbone network, contrast learning mechanism and data preprocessing module to form a complete initial image enhancement model. The data preprocessing module is responsible for performing necessary preprocessing operations on the input original image, such as normalization, cropping, etc., to meet the input requirements of the model, while the teacher model and student model cooperate with each other through the contrast learning mechanism, and use the target backbone network to enhance the image, so as to realize effective enhancement of underwater images, improve the clarity, contrast and color accuracy of the image, and provide high-quality image data for subsequent marine aquaculture monitoring and analysis.

[0062] Step S20, training the initial image enhancement model with labeled data and unlabeled data to obtain the target image enhancement model.

[0063] It should be noted that labeled data refers to image data that has been labeled with correct output during training. These data are usually labeled by professionals, and the labeling content includes the expected enhancement effect of the image or other related features. In the task of underwater image enhancement, labeled data may include high-quality images adjusted by hand or processed by professional software, which are used as "standard answers" for supervised learning to help the model learn how to convert low-quality input images into high-quality output images.

[0064] Unlabeled data refers to image data that has not been labeled with correct output during training. These data are usually directly collected from actual application scenarios, such as original images obtained from underwater cameras.

[0065] The target image enhancement model refers to an image enhancement model that has been trained and optimized. It can effectively enhance the quality of underwater images in different scenarios.

[0066] It can be understood that the image processing device first inputs the labeled data into the student model of the initial image enhancement model. Through supervised learning, the difference between the student model output and the labeled data (such as pixel-level error, perception loss, and gradient loss) is calculated to optimize the model parameters. This process enables the model to learn how to convert low-quality input images into high-quality output images. At the same time, the image processing device inputs the unlabeled data into the teacher model, which generates high-quality pseudo labels for the unlabeled data through the pseudo label generation module. The student model then uses these pseudo labels for consistency training to further optimize the model parameters. After this series of training processes, the image processing device finally obtains the target image enhancement model.

[0067] Step S30: Obtain the underwater image to be enhanced.

[0068] It should be noted that the underwater image refers to the original image captured by the underwater camera in the marine aquaculture scene. These images are usually affected by the light absorption and scattering properties of seawater, resulting in color shift, insufficient overall brightness, low contrast, and background turbidity, etc. These problems result in poor image quality, making it difficult to be directly used for subsequent analysis and monitoring.

[0069] It can be understood that the image processing device obtains the original underwater image data by connecting to the underwater camera, reading from the storage device, or searching from the Internet.

[0070] Step S40: Input the underwater image into the target image enhancement model to obtain an enhanced image.

[0071] It should be noted that the enhanced image refers to the underwater image processed by the target image enhancement model. After the optimization processing of the model, the quality of these images has been significantly improved, including improving color shift, increasing overall brightness, enhancing contrast, and reducing background turbidity, etc. The enhanced images are clearer and closer to the real scene, which can better support various application requirements in the marine aquaculture scene, such as fish behavior monitoring, water quality analysis, etc.

[0072] As an example, the target image enhancement model includes an initial convolution module, a gradient feature extraction module, a minimalist gating mechanism, a feature fusion module, and a channel attention module; the step of inputting the underwater image into the target image enhancement model to obtain an enhanced image includes: performing convolution processing on the underwater image through the gradient feature extraction module and the initial convolution module to obtain gradient amplitude features and original features; performing feature fusion, feature splitting, and weighting processing on the original features and the gradient amplitude features through the feature fusion module, the minimalist gating mechanism, and the channel attention module to obtain weighted gating features; performing residual connection on the weighted gating features and the original features to obtain enhanced features; and performing deconvolution processing on the enhanced features to obtain an enhanced image.

[0073] The initial convolution module is the first processing unit in the target image enhancement model, responsible for performing preliminary convolution operations on the input underwater image. It extracts basic features of the image through convolution kernels, providing a basis for subsequent feature processing and enhancement.

[0074] Gradient amplitude features (gradient map) refer to image gradient information calculated by the gradient feature extraction module, which can highlight the edges and structure information in the image, helping to enhance the details and clarity of the image. Specifically, gradient amplitude features are obtained by calculating the gradient amplitude in the horizontal and vertical directions and performing normalized fusion. The formula for gradient amplitude features is as follows:

[0075]

[0076] where G y is the vertical gradient component, and G x is the horizontal gradient component.

[0077] Original features F raw refer to basic features of the image extracted by the initial convolution module, including edge, texture, and color information of the image, which are the basis for subsequent feature processing and enhancement.

[0078] Weighted gating features refer to features processed by the minimalist gating mechanism and the channel attention module. These features are weighted by the gating mechanism and attention mechanism, which can better highlight important features and suppress unimportant features.

[0079] Enhanced features refer to features obtained by performing residual connection on the weighted gating features and the original features through the feature fusion module. These features fuse gradient information and original image information, and after optimization processing, they can better represent the enhancement effect of the input image.

[0080] Firstly, the image processing device inputs the acquired underwater image into the gradient feature extraction module and the initial convolution module simultaneously. The gradient feature extraction module calculates the gradients of the image in horizontal and vertical directions respectively through specific convolution kernels, and fuses the gradient amplitudes of the two directions by L2 normalization to obtain the gradient amplitude feature; the initial convolution module extracts the basic features of the image through convolution operation to obtain the original features. Secondly, the image processing device inputs the gradient amplitude feature and the original feature into the feature fusion module, splices the two features along the channel dimension, then performs LayerNorm normalization processing to improve numerical stability, and linearly maps the channel number through 1x1 convolution, and then performs 3x3 depth separable convolution to extract local spatial features to obtain the fused features. Then, the fused features are input into the minimal gate mechanism, which divides the features into two parts along the channel dimension, and then element-wise multiplies them to obtain the gated features. Finally, the gated features are input into the channel attention module, which generates channel attention weights through global average pooling and Sigmoid activation function, and weights the features to obtain weighted gated features. The purpose of this series of operations is to enhance the expression ability of the features and pay attention to important information through feature fusion, gating and attention mechanisms, so as to improve the model's ability to capture image details and structures.

[0081] Then, the image processing device performs residual connection on the weighted gated features and the original features, that is, element-wise addition operation on the weighted gated features and the original features obtained by the initial convolution module to form enhanced features. This residual connection method can preserve important information of the original image, while combining the optimized features to further improve the image enhancement effect. Finally, the image processing device performs deconvolution processing on the enhanced features to restore the resolution and details of the image through upsampling operation, thereby obtaining the final enhanced image. The purpose of deconvolution processing is to map the features extracted and optimized back to the image space to generate high-quality enhanced images, so that they are closer to the real scene in visual effect, meeting the requirements of underwater image quality in marine aquaculture scenarios.

[0082] As an example, the step of performing feature fusion, feature splitting and weighting processing on the original features and the gradient amplitude features through the feature fusion module, the minimal gate mechanism and the channel attention module to obtain the weighted gated features includes: performing convolution processing on the original features and the gradient amplitude features through the feature fusion module, and splicing along the channel dimension to obtain fused features; performing normalization processing and depth separable convolution on the fused features to obtain local spatial features; splitting the local spatial features into first and second sub-features through the minimal gate mechanism, and element-wise multiplying them to obtain gated features; weighting the gated features based on the attention weights of the channel attention module to obtain the weighted gated features.

[0083] Channel dimension refers to the channel dimension in the image feature tensor, which is usually used to represent the feature channels of an image. In deep learning, an image is usually represented as a four-dimensional tensor with dimensions [batch_size, channels, height, width], where channels is the channel dimension. The channel dimension is used to store the feature information of the image, such as an RGB image has 3 channels, while a feature map in a deep learning model may have more channels, each representing different feature information.

[0084] Fusion feature refers to the feature obtained by concatenating the original feature and the gradient magnitude feature through the feature fusion module. Specifically, the original feature and the gradient magnitude feature are concatenated along the channel dimension to form a feature tensor containing more information. This fusion can fully utilize the advantages of the original feature and the gradient feature, providing more abundant information for subsequent feature processing.

[0085] Local spatial feature refers to the feature obtained by processing the fusion feature through a depth separable convolution. Depth separable convolution is an efficient convolution operation that can extract local spatial information of features. This feature can capture local structure and texture information in images, helping the model better understand image content.

[0086] First sub-feature refers to one of the two sub-features obtained by splitting the local spatial feature through the minimal gating mechanism. The minimal gating mechanism splits the local spatial feature along the channel dimension into two sub-features, and the first sub-feature is one of them. These two sub-features will be used for subsequent gating operations.

[0087] Second sub-feature refers to the other of the two sub-features obtained by splitting the local spatial feature through the minimal gating mechanism. Similar to the first sub-feature, the second sub-feature is also the result of splitting the local spatial feature along the channel dimension, and is used for subsequent gating operations.

[0088] Gated feature refers to the feature obtained by element-wise multiplication of the first sub-feature and the second sub-feature through the minimal gating mechanism. This operation can dynamically adjust the importance of features, enhancing the model's ability to express key features while suppressing unimportant features.

[0089] Attention weight refers to the weight generated by the channel attention module, which is used for weighted processing of the gated feature. The channel attention module generates these weights through global average pooling and Sigmoid activation function, and each weight corresponds to a feature channel. Attention weights can highlight the features of important channels and suppress unimportant features, thereby improving the model's ability to understand image content.

[0090] First, the image processing device inputs the original feature and the gradient amplitude feature into the feature fusion module respectively, and extracts and converts them through a 3x3 convolution operation to obtain F grad_conv The specific calculation formula is as follows:

[0091] F raw_conv =Conv 3×3 (F raw )

[0092] F grad_conv =Conv 3×3 (G mag )

[0093] Then the two feature tensors are spliced together along the channel dimension to form a fusion feature F fused containing more information, and the calculation formula is as follows:

[0094] F fused =Concat(F raw_conv ,F grad_conv )

[0095] The purpose of this is to combine the basic features of the original image with gradient information to provide richer feature representation for subsequent processing. Secondly, the fusion feature is normalized by LayerNorm to improve numerical stability and avoid gradient explosion or disappearance problem, and then the channel number is linearly mapped through 1x1 convolution, and then 3x3 depth separable convolution operation is performed. This convolution method can extract local spatial information of features while being high in computing efficiency, so as to obtain local spatial features. This step is to capture local structures and texture details in images to provide a basis for subsequent feature optimization. Then, the local spatial features are divided into two along the channel dimension through a minimalist gating mechanism to obtain first sub-feature and second sub-feature , and the two sub-features are multiplied element by element to generate a gating feature , and the formula is as follows: Fgate=F1⊙F2 Where, ⊙ represents element-wise multiplication operation.

[0096] This element-wise multiplication operation can dynamically adjust the importance of features, enhance the expression ability of the model to key features, and suppress unimportant features at the same time, so that the model can better focus on important information in the image. Finally, the attention weight The gating features are weighted, and the attention weight can highlight the important channel features and suppress the unimportant features according to the global information, so as to obtain the weighted gating features. This process further optimizes the feature representation and makes it more suitable for the image enhancement task, providing a better foundation for generating high-quality enhanced images. The formula is as follows:

[0100] where σ is the Sigmoid activation function, F sca is the weighted channel enhancement feature map.

[0101] Please refer to Figure 2 , the underwater image enhancement network architecture under the semi-supervised learning framework, which contains the student model (U-Grad-NAF Student) and the teacher model (U-Grad-NAF Teacher). The student model receives labeled images and unlabeled images, and optimizes the parameters by calculating the supervised loss (using labeled images and real labels) and the contrast loss (using unlabeled images and pseudo labels generated by the teacher model). The teacher model is updated by the EMA strategy using the parameters of the student model to generate more stable learning targets. The Grad-NAF Block is also shown in the figure, which is the basic unit for feature extraction in the network, including the gradient feature extraction module, the feature fusion module, the simple gating mechanism and the channel attention module, which are used to process the input original image and gradient information. The input of the network includes the original image and its gradient information, which is processed by a series of 3x3 convolution, 1x1 convolution, LayerNorm normalization, SimpleGate and SCA module to generate enhanced features. These features are then processed by the deconvolution operation to generate the final enhanced image. In addition, the MUSIQ module is also shown in the figure, which is used to evaluate the image quality output by the teacher model and update the reliable label pool (Reliable Bank) accordingly to ensure the quality of the pseudo labels. The whole architecture integrates the supervised loss, the pseudo label consistency loss and the contrast loss through the joint loss function to realize the underwater image enhancement under the condition of limited labeled data.

[0102] As an example, the step of performing convolution processing on the underwater image through the gradient feature extraction module and the initial convolution module to obtain gradient magnitude features and original features includes: performing feature extraction on the underwater image through horizontal and vertical convolution kernels in the gradient feature extraction module to obtain horizontal and vertical gradient components; performing square and square root operations on the horizontal and vertical gradient components to generate gradient magnitude features; and performing convolution processing on the underwater image through the initial convolution module to obtain original features.

[0103] A horizontal convolution kernel is a specific type of convolution kernel used to extract gradient information in the horizontal direction from an input image. It is designed to highlight edges and changes in the horizontal direction within an image. It is typically a two-dimensional matrix designed to detect changes in the horizontal direction within an image. For example, a common horizontal convolution kernel might be:

[0104]

[0105] A vertical convolution kernel is a specific type of convolution kernel used to extract gradient information in the vertical direction from an input image. It is designed to highlight edges and changes in the vertical direction within an image. It is typically a two-dimensional matrix designed to detect changes in the vertical direction within an image. For example, a common vertical convolution kernel might be:

[0106]

[0107] Horizontal gradient component (horizontal direction gradient) G x is the result obtained by performing convolution operation on the input image through the horizontal convolution kernel, which reflects the gradient change of the image in the horizontal direction, i.e. the edge and structure information in the horizontal direction of the image. The horizontal gradient component can help the model capture the details and changes in the horizontal direction of the image. The formula is as follows:

[0108]

[0109] Vertical gradient component (vertical direction gradient) G y is the result obtained by performing convolution operation on the input image through the vertical convolution kernel, which reflects the gradient change of the image in the vertical direction, i.e. the edge and structure information in the vertical direction of the image. The vertical gradient component can help the model capture the details and changes in the vertical direction of the image. The formula is as follows:

[0110]

[0111] First, the image processing device inputs the underwater image into the gradient feature extraction module, performs convolution operation on the image using a predefined horizontal convolution kernel, calculates the gradient change of the image in the horizontal direction, and obtains the horizontal gradient component. At the same time, the same underwater image is convolved using a vertical convolution kernel to calculate the gradient change in the vertical direction, and the vertical gradient component is obtained. The purpose of these two operations is to extract the edge information of the image in the horizontal and vertical directions respectively, and to provide a basis for subsequent gradient amplitude calculation. Second, the image processing device squares the obtained horizontal gradient component and vertical gradient component respectively, then adds the squared horizontal gradient component and vertical gradient component to obtain a comprehensive gradient square sum. Then, the gradient square sum is square root operation to generate the final gradient amplitude feature. This process combines the gradient information in the horizontal and vertical directions into a single feature through mathematical operations, which can more comprehensively reflect the edge and structure information in the image, and provide more detailed information for subsequent feature fusion and enhancement processing. Finally, the image processing device inputs the underwater image into the initial convolution module to extract the basic features of the image through a series of convolution operations to obtain the original features. These original features include the basic information of the image such as edges, textures and colors, which are important basis for subsequent feature fusion and enhancement processing. By simultaneously extracting gradient amplitude features and original features, the model can fully utilize the structural information and basic features of the image to provide more comprehensive input for generating high-quality enhanced images.

[0112] The embodiment provides a lightweight underwater image enhancement method based on semi-supervised non-activation function,

[0113] First, an initial image enhancement model is constructed based on a teacher-student framework of semi-supervised contrast learning and a gradient non-activation function network. This architecture combines the advantages of semi-supervised learning and the characteristics of lightweight network structure, provides high-quality pseudo labels through the teacher model, and the student model uses these pseudo labels and labeled data for joint training to improve the generalization ability of the model. At the same time, the design of the gradient non-activation function network reduces the consumption of computing resources and improves the real-time performance and adaptability of the model. Second, the initial image enhancement model is trained by labeled data and unlabeled data to obtain a target image enhancement model. This training process makes full use of limited labeled data and a large amount of unlabeled data to further improve the generalization performance of the model, so that it can work effectively in various complex underwater environments. Then, the underwater images to be enhanced are obtained, which are usually directly collected from underwater cameras and may have poor quality. Finally, the underwater image is input into the target image enhancement model to obtain an enhanced image. The whole process not only can efficiently process images, but also can improve the cross-scene adaptability of underwater image enhancement and reduce the deployment cost of the model, providing strong technical support for smart fishing and other applications.

[0114] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above-mentioned first embodiment can be referred to the above introduction, and the subsequent will not be described. On this basis, please refer to Figure 3 , Figure 3 The flowchart of the second embodiment of the lightweight underwater image enhancement method based on semi-supervised and activation function-free of the present application is shown in the figure. The initial image enhancement model includes a teacher model and a student model. The step S20 of the lightweight underwater image enhancement method based on semi-supervised and activation function-free includes steps S21-S24:

[0115] Step S21, based on the parameters of the student model, the parameters of the teacher model are updated by the exponential moving average strategy.

[0116] It should be noted that the EMA strategy is a technique for updating the parameters of the teacher model. In this strategy, the parameters of the teacher model are updated gradually according to the parameters of the student model, rather than directly using the parameters of the student model. The formula for updating the parameters of the teacher network is:

[0117] θ t =ηθ t +(1-η)θ s

[0118] Where η∈(0,1) is the exponential moving average factor, θ t is the weight parameter of the teacher network, and θ s is the current weight parameter of the student network.

[0119] It can be understood that first, the parameter values of the student model in the current training step are obtained, which reflect the learning achievements of the student model in the current training stage. Second, according to the preset exponential moving average factor (for example, 0.999), the update value of the teacher model parameter is calculated. The specific calculation formula is: the new parameter value of the teacher model is equal to the current teacher model parameter value multiplied by the exponential moving average factor, plus the corresponding parameter value of the student model multiplied by (1 minus the exponential moving average factor). Finally, the calculated update value is assigned to the parameters of the teacher model, completing the parameter update this time. In this way, the parameters of the teacher model can smoothly approach the parameters of the student model while maintaining a certain stability, avoiding the instability of the teacher model caused by the violent fluctuations of the student model parameters, thereby providing more reliable guidance for the student model.

[0120] Step S22, calculate the supervised loss according to the labeled data, and calculate the contrast loss according to the unlabeled data.

[0121] It should be noted that the supervised loss refers to a loss function calculated on labeled data, which is used to measure the difference between the model output and the true label. By minimizing the supervised loss, the model can learn how to convert low-quality input images into high-quality target images, thereby improving the performance of the model on labeled data. The supervised loss formula is as follows:

[0122] L sup = L L1 + β1L per + β2L grad

[0123] where L L1 is the pixel-level error; L per is the perceptual loss, which is based on the VGG network to extract the difference in high-order semantic features of the image; L grad is the gradient loss, which ensures that the edges are clear, and β1, β2 are weight coefficients that adjust the importance of the perceptual loss and the gradient loss, respectively.

[0124] The contrast loss refers to a loss function calculated on unlabeled data, which is used to measure the difference between the pseudo-label generated by the model and the output of the student model. In semi-supervised learning, the teacher model generates high-quality pseudo-labels for unlabeled data, and the student model learns how to better match these pseudo-labels through contrast loss. Contrast loss is usually calculated based on the similarity or difference of features, for example, by calculating the loss of positive and negative sample pairs at the slice level. The contrast loss is the slice contrast estimation loss, and the formula is as follows:

[0125]

[0126] where y is the output of the student model; φ j (·) represents the image features extracted from the jth layer of VGG; ω j is the loss weighting factor of each layer.

[0127] As an example, the step of calculating the supervised loss according to the labeled data and the contrast loss according to the unlabeled data includes: inputting the labeled data into the student model to calculate the supervised loss containing the pixel-level error, the perceptual error and the gradient error; inputting the unlabeled data into the teacher model to generate initial pseudo-labels, and selecting target pseudo-labels with a quality score higher than a preset threshold from the initial pseudo-labels; constructing a contrast sample pair based on the target pseudo-labels and the unlabeled data, and calculating the contrast loss according to the contrast sample pair.

[0128] Pixel-level error refers to the difference between the model output image and the true label image at the pixel level, usually calculated using L1 loss or L2 loss, that is, calculating the average of the absolute difference or squared difference between each pixel value of the output image and the target image, this error reflects the model's restoration ability at the pixel level, ensuring that the enhanced image is close to the true image in brightness and color.

[0129] Perceptual error is a loss based on high-level semantic features, usually using a pre-trained convolutional neural network (such as VGG network) to extract the feature representation of the image, then calculating the difference between the model output image and the true label image in the feature space. This error can capture high-level semantic information of the image, such as texture, structure, etc., so as to ensure that the enhanced image is closer to the true image in visual effect.

[0130] Gradient error refers to the difference between the model output image and the true label image in gradient information. Gradient information reflects the edges and structures of the image, by calculating the difference in gradient amplitude between the output image and the target image, it can ensure that the enhanced image maintains clear edges and structures.

[0131] Initial pseudo-label refers to the preliminary prediction result generated by the teacher model for unlabeled data. The teacher model generates high-quality predicted labels for unlabeled data through its stable parameters and learned knowledge, these labels are called initial pseudo-labels. The storage pool B stores the pseudo-labels U is defined as:

[0132]

[0133] where is the unlabeled image, is the output of the teacher network after credibility screening.

[0134] Quality score refers to the score of the quality of the initial pseudo-label, usually using an image quality evaluation model to score the initial pseudo-label generated by the teacher model to judge whether its quality is high enough, the quality score reflects the closeness of the pseudo-label to the true label and its rationality in visual effect.

[0135] The preset threshold is a pre-set quality score standard used to screen high-quality pseudo-labels. Only when the quality score of the initial pseudo-label is higher than this preset threshold, it is considered as a reliable target pseudo-label.

[0136] Target pseudo-label refers to the pseudo-label selected from the initial pseudo-label whose quality score is higher than the preset threshold. These pseudo-labels are considered to be high-quality and can be used for consistency training of the student model.

[0137] The contrast sample pair refers to a positive-negative sample pair constructed based on the target pseudo label and the unlabeled data. The positive sample pair refers to a similarity sample between the unlabeled data and the corresponding target pseudo label, and the negative sample pair refers to a difference sample between the unlabeled data and other irrelevant data.

[0138] First, the labeled data is input into the student model, and the pixel-level error between the model output and the true label is calculated, which is usually achieved by calculating the absolute value or square value of the difference between the two at each pixel position, and then the error of all pixels is averaged to obtain the pixel-level error; then the perceptual error is calculated, which requires using a pre-trained feature extraction network to extract the high-level feature representation of the model output and the true label, and then calculating the difference between the two feature representations, usually using L2 norm to measure; then the gradient error is calculated, the horizontal and vertical gradients of the model output and the true label are calculated respectively, and then the difference between the gradients is calculated, and finally the gradient error is obtained by averaging all the gradient errors. The three kinds of errors are weighted and summed to obtain the comprehensive supervision loss, and by minimizing the supervision loss, the student model can learn how to convert low-quality images into high-quality images.

[0139] Second, the unlabeled data is input into the teacher model to generate initial pseudo labels. For an unlabeled image Generating pseudo labels by the teacher network Then, a preset image quality evaluation model (such as MUSIQ) is used to score the quality of each initial pseudo label:

[0140]

[0141] Where q i is the quality score obtained by MUSIQ evaluation.

[0142] This score reflects the closeness of the pseudo label to the true image. If its score is higher than the threshold τ, it is considered as a high-quality pseudo label Otherwise, it is discarded, and the screening criteria are as follows:

[0143]

[0144] Where the MUSIQ index realizes multi-scale estimation of image quality through the following formula:

[0145]

[0146] Where I is the input image, represents the representation of the image at different scales, CNN is a convolutional network for extracting visual features, and f represents the feature fusion and scoring function.

[0147] The target pseudo-labels with a quality score higher than the threshold are screened according to the preset threshold, and this screening process ensures that the pseudo-labels used for student model training have high quality, avoiding interference of low-quality pseudo-labels on the training process.

[0148] Finally, the target pseudo-labels and the unlabeled data are used to construct contrastive sample pairs. Specifically, for each unlabeled data and its corresponding target pseudo-label, other unlabeled data are randomly selected from the data set as negative samples to construct positive sample pairs (unlabeled data and target pseudo-labels) and negative sample pairs (unlabeled data and other unlabeled data). Then, by calculating the similarity or difference between the positive sample pairs and the negative sample pairs, the contrastive loss is obtained, which is usually calculated using a contrastive loss function (such as InfoNCE loss). By minimizing the contrastive loss, the feature representation capability of the model is further optimized, and the generalization performance of the model on unlabeled data is improved, so that the entire model can better adapt to different scenarios when processing underwater image enhancement tasks.

[0149] Step S23, the supervised loss, the pseudo-label consistency loss, and the contrastive loss are combined into a joint loss function.

[0150] It should be noted that the pseudo-label consistency loss refers to a loss function that measures the consistency between the output of the student model and the high-quality pseudo-label generated by the teacher model. Specifically, it is defined by calculating the difference between the prediction result of the student model on the unlabeled data and the target pseudo-label generated by the teacher model. The pseudo-label consistency loss formula is as follows:

[0151]

[0152] Where M is the number of unlabeled images, is the selected pseudo-label in the reliable unsupervised image pool, is the i-th unlabeled image, and s is the data augmentation transformation corresponding to the student model.

[0153] The joint loss function refers to a comprehensive loss function that combines the supervised loss, the pseudo-label consistency loss, and the contrastive loss. The goal of the joint loss function is to optimize the performance of the student model on labeled data (through the supervised loss), the generalization ability on unlabeled data (through the pseudo-label consistency loss), and the robustness of feature representation (through the contrastive loss). By minimizing the joint loss function, the student model can learn more comprehensive and robust feature representations in limited labeled data and a large amount of unlabeled data, thereby improving its performance in practical applications. The joint loss function formula is:

[0154] L total =L sup +λ·L″un

[0155] wherein L total is the total loss (joint loss), L sup is the supervised loss, λ is the unsupervised loss weighting coefficient, L" un is the overall unsupervised loss (including pseudo-label consistency and contrastive loss):

[0156] L" un = L' un + γL cr

[0157] wherein L' un is the pseudo-label consistency loss, L cr is the contrastive loss, and γ is the weighting factor of the contrastive loss.

[0158] It can be understood that first, the pseudo-label consistency loss is calculated, and the specific method is to compare the prediction results of the student model on the unlabeled data with the target pseudo-label generated by the teacher model pixel by pixel or feature by feature, and usually the mean square error (MSE) or L1 loss is used to measure the difference between the two, so as to obtain the value of the pseudo-label consistency loss. Finally, the supervised loss, the pseudo-label consistency loss and the contrastive loss are weighted and summed according to the preset weight coefficient to form a joint loss function. This joint loss function combines the accuracy of supervised learning, the generalization ability of semi-supervised learning and the feature robustness of contrastive learning. By minimizing the joint loss function, the student model can be optimized under various data and task constraints, thereby improving the overall performance.

[0159] Step S24, according to the joint loss function, the parameters of the student model are optimized in reverse, and the step of updating the parameters of the teacher model based on the parameters of the student model through the exponential moving average strategy is returned until the training iteration number reaches the preset iteration threshold or the joint loss function converges, and the student model is taken as the target image enhancement model.

[0160] It should be noted that the preset iteration threshold refers to a maximum number of iterations set in advance during the training process, which is used to control when the training process stops. In this embodiment, the preset iteration threshold is 1000 times. This threshold is set to balance the training time and the model performance, to avoid the waste of computing resources caused by overtraining, and at the same time to ensure that the model has enough time to learn effective features and parameters.

[0161] It can be understood that firstly, the model parameters of the student are back-propagated using the calculated joint loss function, the gradient of the loss function with respect to the parameters is calculated, and the parameters of the student model are updated according to these gradients by an optimization algorithm (such as Adam or SGD) to reduce the value of the joint loss function, so that the student model can better learn and adapt to both labeled and unlabeled data. Secondly, based on the updated parameters of the student model, the parameters of the teacher model and the parameters of the student model are weighted and averaged according to the exponential moving average formula, and the parameters of the teacher model retain a larger weight to ensure the stability of the teacher model, while absorbing the learning achievements of the student model to provide higher quality pseudo-label guidance for the student model. Finally, check whether the current training iteration number reaches the preset iteration threshold or whether the joint loss function converges (i.e. the change of the loss value is less than the set threshold), if any of the conditions is met, stop training, and use the student model as the target image enhancement model for subsequent underwater image enhancement tasks, so as to ensure that the model reaches the optimal performance within a limited training time.

[0162] The embodiment updates the parameters of the teacher model based on the parameters of the student model through the exponential moving average strategy, which can maintain the stability of the teacher model parameters while enabling them to absorb the learning achievements of the student model, thereby providing higher quality pseudo-label guidance for the student model. Then, the supervised loss is calculated according to the labeled data to ensure that the student model learns the correct features and outputs on the labeled data, and the contrastive loss is calculated according to the unlabeled data to optimize the feature representation ability of the model and improve its generalization performance on the unlabeled data. Then, the supervised loss, the pseudo-label consistency loss and the contrastive loss are combined into a joint loss function, which integrates multiple learning goals to enable the student model to optimize under multiple constraints and learn more robust feature representations. Finally, the parameters of the student model are optimized according to the joint loss function, and the step of updating the parameters of the teacher model is returned until the training iteration number reaches the preset iteration threshold or the joint loss function converges, and the student model is used as the target image enhancement model. This cyclic optimization process ensures that the model is continuously optimized in a limited amount of labeled data and a large amount of unlabeled data, and finally obtains a target image enhancement model with stable performance and strong generalization ability, which can better adapt to underwater image enhancement tasks in different scenarios, while reducing the dependence on a large amount of labeled data and improving the efficiency and cost-effectiveness of model deployment.

[0163] The application also provides a lightweight underwater image enhancement device based on semi-supervised activation function-free, please refer to Figure 4 , the lightweight underwater image enhancement device based on semi-supervised activation function-free comprises:

[0164] The model construction module 10 is configured to construct an initial image enhancement model based on a semi-supervised contrastive learning teacher-student framework and a gradient activation function-free network.

[0165] The model training module 20 is configured to train the initial image enhancement model through labeled data and unlabeled data to obtain a target image enhancement model.

[0166] The image acquisition module 30 is configured to acquire an underwater image to be enhanced.

[0167] The image enhancement module 40 is configured to input the underwater image into the target image enhancement model to obtain an enhanced image.

[0168] In an embodiment, the model training module 20 is further configured to update parameters of the teacher model based on parameters of the student model through an exponential moving average strategy, calculate a supervised loss based on labeled data, calculate a contrastive loss based on unlabeled data, combine the supervised loss, a pseudo label consistency loss and the contrastive loss into a joint loss function, and optimize the parameters of the student model based on the joint loss function, and return the step of updating the parameters of the teacher model based on the parameters of the student model through the exponential moving average strategy until a preset iteration threshold is reached or the joint loss function converges, and take the student model as the target image enhancement model.

[0169] In an embodiment, the model training module 20 is further configured to input labeled data into the student model to calculate a supervised loss containing a pixel-level error, a perception error and a gradient error, input unlabeled data into the teacher model to generate initial pseudo labels, and filter target pseudo labels with a quality score higher than a preset threshold from the initial pseudo labels, and construct a contrastive sample pair based on the target pseudo labels and the unlabeled data, and calculate a contrastive loss based on the contrastive sample pair.

[0170] In an embodiment, the model construction module 10 is further configured to initialize network parameters of the teacher model and the student model, the initial backbone network of the teacher model and the student model is a gradient activation function-free network, define a gradient feature extraction module, a feature fusion module, an ultra-simple gate mechanism and a channel attention module in the gradient activation function-free network to obtain a target backbone network, establish a contrastive learning mechanism between the teacher model and the student model according to a pseudo label generation module and a contrastive loss calculation module, and combine the teacher model, the student model, the target backbone network, the contrastive learning mechanism and a data preprocessing module to obtain an initial image enhancement model.

[0171] In an embodiment, the image enhancement module 40 is further configured to perform convolution processing on the underwater image by the gradient feature extraction module and the initial convolution module to obtain gradient amplitude features and original features; perform feature fusion, feature splitting and weighting processing on the original features and the gradient amplitude features by the feature fusion module, the minimal gating mechanism and the channel attention module to obtain weighted gating features; perform residual connection on the weighted gating features and the original features to obtain enhanced features; and perform deconvolution processing on the enhanced features to obtain an enhanced image.

[0172] In an embodiment, the image enhancement module 40 is further configured to perform convolution processing on the original features and the gradient amplitude features by the feature fusion module, and perform channel dimension splicing to obtain fusion features; perform normalization processing and depth separable convolution on the fusion features to obtain local spatial features; split the local spatial features into first sub-features and second sub-features by the minimal gating mechanism, and perform element-wise multiplication to obtain gating features; and perform weighting processing on the gating features based on attention weights of the channel attention module to obtain weighted gating features.

[0173] In an embodiment, the image enhancement module 40 is further configured to perform feature extraction on the underwater image by horizontal convolution kernels and vertical convolution kernels in the gradient feature extraction module to obtain horizontal gradient components and vertical gradient components; perform square and square root operations on the horizontal gradient components and the vertical gradient components to generate gradient amplitude features; and perform convolution processing on the underwater image by the initial convolution module to obtain original features.

[0174] The lightweight underwater image enhancement device based on semi-supervised non-activation function provided in the application adopts the lightweight underwater image enhancement method based on semi-supervised non-activation function in the above embodiments, and can solve the technical problem of how to improve the cross-scene adaptability of underwater image enhancement and reduce the model deployment cost. Compared with the prior art, the lightweight underwater image enhancement device based on semi-supervised non-activation function provided in the application has the same beneficial effects as the lightweight underwater image enhancement method based on semi-supervised non-activation function provided in the above embodiments, and other technical features in the lightweight underwater image enhancement device based on semi-supervised non-activation function are the same as the features disclosed in the above embodiment method.

[0175] The application provides a lightweight underwater image enhancement device based on semi-supervised activation function-free. The lightweight underwater image enhancement device based on semi-supervised activation function-free comprises at least one processor and a memory in communication connection with the at least one processor. The memory stores instructions executable by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the lightweight underwater image enhancement method based on semi-supervised activation function-free in the above embodiment one.

[0176] Reference will be made to the following Figure 5 which shows a structural schematic diagram of the lightweight underwater image enhancement device based on semi-supervised activation function-free suitable for being used to implement the embodiments of the application. The lightweight underwater image enhancement device based on semi-supervised activation function-free in the embodiments of the application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 5 The lightweight underwater image enhancement device based on semi-supervised activation function-free shown is only an example and should not bring any limitation to the functions and use range of the embodiments of the application.

[0177] As Figure 5As shown, the semi-supervised activation function free lightweight underwater image enhancement device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a ROM (Read Only Memory) 1002 or loaded from a storage device 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the semi-supervised activation function free lightweight underwater image enhancement device to operate are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, an LCD (Liquid Crystal Display), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the semi-supervised activation function free lightweight underwater image enhancement device to communicate wirelessly or wired with other devices to exchange data. Although the semi-supervised activation function free lightweight underwater image enhancement device with various systems is shown in the figure, it should be understood that all the systems shown are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0178] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carrying computer program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0179] The application provides a lightweight underwater image enhancement device based on semi-supervised non-activation function, adopts the lightweight underwater image enhancement method based on semi-supervised non-activation function in the above embodiment, and can solve the technical problem of how to improve the cross-scene adaptability of underwater image enhancement and reduce the model deployment cost. Compared with the prior art, the beneficial effects of the lightweight underwater image enhancement device based on semi-supervised non-activation function provided by the application are the same as those of the lightweight underwater image enhancement method based on semi-supervised non-activation function provided by the above embodiment, and other technical features in the lightweight underwater image enhancement device based on semi-supervised non-activation function are the same as those disclosed in the previous embodiment method.

[0180] The application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon for executing the lightweight underwater image enhancement method based on semi-supervised non-activation function in the above embodiment.

[0181] The computer-readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory or flash memory), an optical fiber, a CD-ROM (CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium can be transmitted in any appropriate medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0182] The above computer-readable storage medium can be included in the lightweight underwater image enhancement device based on semi-supervised non-activation function; or can exist separately.

[0183] The computer readable storage medium described above carries one or more programs, which, when executed by the lightweight underwater image enhancement device based on semi-supervised non-activation function, cause the lightweight underwater image enhancement device based on semi-supervised non-activation function to: construct an initial image enhancement model based on a semi-supervised contrast learning teacher-student framework and a gradient non-activation function network; train the initial image enhancement model through labeled data and unlabeled data to obtain a target image enhancement model; obtain an underwater image to be enhanced; input the underwater image into the target image enhancement model to obtain an enhanced image.

[0184] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0185] The accompanying drawings illustrate possible implementation architectures, functions, and operations of the systems, methods, and computer program products of this application, in which like reference numerals represent like elements throughout the various figures. The functions of the various elements shown in the figures can be carried out by any combination of special-purpose hardware and / or computer instructions. Certain functions will be described as being performed by parts of the systems, for example, the image enhancement device based on semi-supervised non-activation function. It will be appreciated that these functions can be carried out by specialized hardware, computer instructions, or any combination of specialized hardware and computer instructions.

[0186] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the above-mentioned lightweight underwater image enhancement method based on semi-supervised non-activation function, and can solve the technical problem of how to improve the cross-scene adaptability of underwater image enhancement and reduce the model deployment cost. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the lightweight underwater image enhancement method based on semi-supervised non-activation function provided by the above-mentioned embodiments, and will not be described here.

[0187] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the semi-supervised activation function-free lightweight underwater image enhancement method as described above. The computer program product provided by the application can solve the technical problem of how to improve the cross-scene adaptability of underwater image enhancement and reduce the model deployment cost. Compared with the prior art, the computer program product provided by the application has the same beneficial effects as the above-mentioned embodiments.

[0188] The above only describes some embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation, direct / indirect application in other related technical fields, or use of the content of the application within the technical concept of the application is included in the patent protection scope of the application.

Claims

1. A lightweight underwater image enhancement method based on semi-supervised activation function-free methods, characterized in that, The method includes: An initial image augmentation model is constructed based on a teacher-student framework and a gradient-free activation function network using semi-supervised contrastive learning. The initial image enhancement model is trained using labeled and unlabeled data to obtain the target image enhancement model; Acquire underwater images to be enhanced; The underwater image is input into the target image enhancement model to obtain the enhanced image.

2. The method as described in claim 1, characterized in that, The initial image enhancement model includes a teacher model and a student model; The step of training the initial image enhancement model using labeled and unlabeled data to obtain the target image enhancement model includes: Based on the parameters of the student model, the parameters of the teacher model are updated using an exponential moving average strategy. The supervised loss is calculated based on labeled data, and the contrastive loss is calculated based on unlabeled data. The supervision loss, pseudo-label consistency loss, and contrast loss are combined into a joint loss function. The steps of back-optimizing the parameters of the student model based on the joint loss function and returning the parameters of the student model, and updating the parameters of the teacher model using an exponential moving average strategy, continue until the number of training iterations reaches a preset iteration threshold or the joint loss function converges, at which point the student model is used as the target image enhancement model.

3. The method as described in claim 2, characterized in that, The steps of calculating the supervised loss based on labeled data and the contrastive loss based on unlabeled data include: The labeled data is input into the student model, and the supervised loss, which includes pixel-level error, perceptual error and gradient error, is calculated. Unlabeled data is input into the teacher model to generate initial pseudo-labels, and target pseudo-labels with quality scores higher than a preset threshold are selected from the initial pseudo-labels; A comparison sample pair is constructed based on the target pseudo-label and the unlabeled data, and the comparison loss is calculated based on the comparison sample pair.

4. The method as described in claim 1, characterized in that, The teacher-student framework based on semi-supervised contrastive learning includes a teacher model and a student model; The steps for constructing the initial image augmentation model using the teacher-student framework and gradient-free activation function network based on semi-supervised contrastive learning include: Initialize the network parameters of the teacher model and the student model. The initial backbone networks of the teacher model and the student model are gradient-free activation function networks. In the gradient-free activation function network, a gradient feature extraction module, a feature fusion module, a minimal gating mechanism, and a channel attention module are defined to obtain the target backbone network. A comparative learning mechanism between the teacher model and the student model is established based on the pseudo-label generation module and the contrastive loss calculation module. The teacher model, the student model, the target backbone network, the contrastive learning mechanism, and the data preprocessing module are combined to obtain the initial image enhancement model.

5. The method as described in claim 1, characterized in that, The target image enhancement model includes an initial convolution module, a gradient feature extraction module, a minimal gating mechanism, a feature fusion module, and a channel attention module; The step of inputting the underwater image into the target image enhancement model to obtain the enhanced image includes: The underwater image is convolved using the gradient feature extraction module and the initial convolution module to obtain gradient magnitude features and original features. The original features and the gradient magnitude features are fused, split, and weighted by the feature fusion module, the simplified gating mechanism, and the channel attention module to obtain weighted gating features. The weighted gated features are residually concatenated with the original features to obtain the enhanced features; The enhanced features are deconvolved to obtain the enhanced image.

6. The method as described in claim 5, characterized in that, The step of performing feature fusion, feature splitting, and weighting on the original features and gradient magnitude features through the feature fusion module, the simplified gating mechanism, and the channel attention module to obtain weighted gating features includes: The original features and the gradient magnitude features are convolved by the feature fusion module and then concatenated along the channel dimension to obtain the fused features. The fused features are normalized and subjected to depthwise separable convolution to obtain local spatial features; The local spatial features are split into a first sub-feature and a second sub-feature using the simplified gating mechanism, and then multiplied element-wise to obtain the gated features. The gating features are weighted based on the attention weights of the channel attention module to obtain weighted gating features.

7. The method as described in claim 5, characterized in that, The step of performing convolution processing on the underwater image using the gradient feature extraction module and the initial convolution module to obtain gradient magnitude features and original features includes: The underwater image is used to extract features by using the horizontal and vertical convolution kernels in the gradient feature extraction module, resulting in horizontal and vertical gradient components. The gradient magnitude feature is generated by performing a square root operation on the sum of the squares of the horizontal gradient components and the vertical gradient components. The underwater image is convolved using the initial convolution module to obtain the original features.

8. A lightweight underwater image enhancement device based on semi-supervised activation function-free operation, characterized in that, The device includes: The model building module is used to build an initial image augmentation model based on a teacher-student framework and a gradient-free activation function network using semi-supervised contrastive learning. The model training module is used to train the initial image enhancement model using labeled and unlabeled data to obtain the target image enhancement model. The image acquisition module is used to acquire underwater images to be enhanced. The image enhancement module is used to input the underwater image into the target image enhancement model to obtain an enhanced image.

9. A lightweight underwater image enhancement device based on semi-supervised activation function-free operation, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the lightweight underwater image enhancement method based on a semi-supervised activation function-free method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the lightweight underwater image enhancement method based on semi-supervised activation function as described in any one of claims 1 to 7.

Images