Intelligent real-time enhancement method, device and electronic equipment for underwater images

Abstract

This invention relates to an intelligent real-time underwater image enhancement method, apparatus, and electronic device. The method includes: acquiring an underwater image to be processed; performing rapid degradation analysis on the underwater image to extract a low-dimensional statistical feature set, which consists of the mean of the grayscale histogram of the entire frame, the average value of the local contrast space, and the grayscale difference value of the red and blue channels; comparing the low-dimensional statistical feature set with a preset threshold set to determine the degradation scene category of the underwater image; dynamically modulating the internal processing parameters of the lightweight Transformer enhancement engine according to the degradation scene category, including the feature fusion weights in the multi-scale feature fusion decoder; and using the parameter-modulated enhancement engine to enhance the underwater image and output the enhanced image. This invention can achieve real-time processing of high-resolution images under limited computing power constraints, effectively identifying and adaptively processing various underwater image degradation scenes, including composite degradation.

Description

Technical Field

[0001] This invention relates to the field of underwater image enhancement and visual perception technology, and in particular to an intelligent real-time underwater image enhancement method, apparatus, and electronic device. Background Technology

[0002] Deep-sea submersibles rely heavily on their visual perception systems to acquire clear and reliable underwater images when performing navigation, observation, and operational tasks. However, the deep-sea imaging environment is extremely complex, and the absorption and scattering effects of seawater on light lead to various degradation problems in the acquired images. Specifically, the rapid decrease in illumination with depth results in low overall image brightness, loss of detail in dark areas, and significant noise (low-light degradation); light scattering caused by suspended particles in the water leads to decreased image contrast, hazy blurring, and color distortion (turbidity degradation). In actual operations, low-light and high-turbidity conditions often coexist, creating more complex compound degradation scenarios that pose a severe challenge to the usability of the visual system.

[0003] To improve underwater image quality, deep learning-based image enhancement methods have been widely applied. Among them, the visual Transformer model has shown potential in image restoration tasks due to its powerful global context modeling capabilities. However, applying it to real-time image enhancement tasks faced by deep-sea submersibles presents significant technical bottlenecks. First, the computational complexity of the self-attention mechanism in the standard Transformer architecture is proportional to the square of the number of image pixels. For high-resolution underwater images (such as 1080p), its enormous computational overhead exceeds the real-time processing capabilities of the submersible's onboard embedded edge computing platform (such as the JTXavier series) (typically requiring a processing latency of less than 33ms), resulting in an irreconcilable contradiction between model performance and real-time requirements. Second, most existing enhancement models have fixed structures and lack mechanisms for quickly and accurately identifying specific degradation types of input images (especially low light, turbidity, and combinations thereof), thus failing to dynamically adjust internal processing strategies according to the actual degradation scenario. A general model trained on a mixed dataset often lacks a clear direction for enhancement when facing a specific dominant degradation. This may result in poor performance in a single degradation scenario and difficulty in coordinating multiple degradation factors in a complex degradation scenario, thus limiting the model's specificity, adaptability, and the reliability of the final enhancement effect.

[0004] Therefore, there is an urgent need to design an underwater image enhancement scheme suitable for edge computing environments, enabling real-time processing of high-resolution images under limited computing power constraints, while effectively identifying and adaptively processing various underwater image degradation scenarios, including composite degradation, to meet the stringent requirements of deep-sea submersible visual perception systems for high reliability, high real-time performance, and strong environmental adaptability. Summary of the Invention

[0005] Therefore, it is necessary to provide an intelligent real-time underwater image enhancement method, device, and electronic device that can run in real time on edge computing devices, has adaptive recognition capabilities for degraded scenes, and has targeted processing strategies for complex degraded scenes, in order to address the above-mentioned technical problems.

[0006] This invention provides an intelligent real-time underwater image enhancement method, comprising:

[0007] Acquire underwater images to be processed;

[0008] A rapid degradation analysis is performed on the underwater image to extract a low-dimensional statistical feature set, which consists of three statistical features: the mean of the gray-level histogram of the entire frame, the average value of the local contrast space, and the gray-level difference value of the red and blue channels.

[0009] The low-dimensional statistical feature set is compared with a preset threshold set to determine the degradation scene category of the underwater image. The degradation scene category includes at least a low-light-turbidity composite degradation scene. The identification conditions for the low-light-turbidity composite degradation scene are: the mean value of the grayscale histogram of the whole frame is lower than the first threshold and the average value of the local contrast space is lower than the second threshold.

[0010] Based on the determined degradation scenario category, the internal processing parameters of the lightweight Transformer enhancement engine are dynamically modulated, including feature fusion weights in the multi-scale feature fusion decoder;

[0011] The underwater image is enhanced using the parameter-modulated lightweight Transformer enhancement engine, and the enhanced image is output.

[0012] The lightweight Transformer enhancement engine includes: a local sparse attention module, used to divide the input feature map into non-overlapping windows and perform attention calculations independently within each window; a cross-window information interaction module, used to introduce global downsampling and upsampling paths after the local sparse attention module to realize information transmission between windows; and a multi-scale feature fusion decoder, used to fuse multi-level features extracted by the encoder and perform weighted fusion of shallow detail features and deep semantic features from the encoder based on the feature fusion weights dynamically generated according to the degradation scene category.

[0013] In one embodiment, the calculation of the mean of the grayscale histogram of the entire frame includes: converting the underwater image into a grayscale image and calculating the arithmetic mean of the grayscale values ​​of all pixels in the grayscale image;

[0014] The calculation of the local contrast spatial average includes: dividing the underwater image into multiple local regions of equal size, calculating the local contrast of each local region, and taking the arithmetic mean of the local contrast of all local regions.

[0015] The calculation of the grayscale difference value of the red and blue channels includes: extracting the red channel and blue channel of the underwater image respectively, and calculating the difference between the mean of the grayscale values ​​of all pixels in the red channel and the mean of the grayscale values ​​of all pixels in the blue channel.

[0016] In one embodiment, the preset threshold set is calibrated based on massive amounts of real-world deep-sea image data; the degradation scene categories also include severely low-light scenes, moderately turbid scenes, and normal scenes.

[0017] In one embodiment, the identification condition for the severe low-light scene is: the mean value of the grayscale histogram of the entire frame is lower than a first threshold and the average value of the local contrast space is not lower than a second threshold;

[0018] The recognition conditions for the moderately turbid scene are: the average local contrast spatial value is lower than the second threshold and the average grayscale histogram value of the entire frame is not lower than the first threshold;

[0019] The identification conditions for the low-light-turbidity composite degradation scene are: the mean value of the grayscale histogram of the entire frame is lower than the first threshold and the average value of the local contrast space is lower than the second threshold;

[0020] The recognition conditions for the normal scene are: the mean value of the grayscale histogram of the entire frame is not lower than the first threshold and the average value of the local contrast space is not lower than the second threshold.

[0021] In one embodiment, the local sparse attention module divides the input feature map into non-overlapping windows of size M×M, where M is an integer greater than 1, and attention calculation is performed only within each window.

[0022] In one embodiment, the cross-window information interaction module uses a window shifting strategy to achieve information transfer between windows.

[0023] In one embodiment, the cross-window information interaction module includes: a downsampling branch for downsampling the output feature map of the local sparse attention module to extract global context information; an upsampling branch for restoring the global context information to its original resolution; and a fusion unit for fusing the restored global context information with the output feature map of the local sparse attention module.

[0024] In one embodiment, the feature fusion weights dynamically generated by the multi-scale feature fusion decoder adjust the fusion ratio of shallow detail features and deep semantic features from the encoder according to the degradation scene category; wherein, when dealing with highly murky scenes, higher weights are assigned to detail feature channels to promote edge sharpening and texture restoration; when dealing with low-light scenes, higher weights are assigned to semantic feature channels to focus on global brightness stretching and noise suppression.

[0025] In one embodiment, based on the determined degradation scenario category, the internal processing parameters of the lightweight Transformer enhancement engine are dynamically modulated, including:

[0026] For low-light scenes, activate the low-light optimization branch and inject brightness prior mapping during the model input stage;

[0027] For highly turbid scenes, the deturbidity branch is activated to guide the model to simulate the inverse process of physical scattering.

[0028] For scenarios with combined degradation due to low light and turbidity, a collaborative processing pipeline is initiated. First, a basic brightness enhancement is performed to improve the low light level, and then a deturbidity removal operation is performed.

[0029] The present invention also provides an intelligent real-time underwater image enhancement device, comprising:

[0030] The image acquisition module is used to acquire underwater images to be processed.

[0031] The degradation feature extraction module is used to perform rapid degradation analysis on the underwater image and extract a low-dimensional statistical feature set. The low-dimensional statistical feature set consists of three statistical features: the mean of the grayscale histogram of the whole frame, the average value of the local contrast space, and the grayscale difference value of the red and blue channels.

[0032] The degradation scene recognition module is used to compare the low-dimensional statistical feature set with a preset threshold set to determine the degradation scene category of the underwater image. The degradation scene category includes at least a low-light-turbidity composite degradation scene. The recognition conditions for the low-light-turbidity composite degradation scene are: the mean value of the grayscale histogram of the entire frame is lower than a first threshold and the average value of the local contrast space is lower than a second threshold.

[0033] The parameter modulation module is used to dynamically modulate the internal processing parameters of the lightweight Transformer enhancement engine according to the determined degradation scene category. The internal processing parameters include feature fusion weights in the multi-scale feature fusion decoder.

[0034] An image enhancement module is used to enhance the underwater image using the parameter-modulated lightweight Transformer enhancement engine and output the enhanced image. The lightweight Transformer enhancement engine includes: a local sparse attention module, used to divide the input feature map into non-overlapping windows and perform attention calculations independently within each window; a cross-window information interaction module, used to introduce global downsampling and upsampling paths after the local sparse attention module to achieve information transfer between windows; and a multi-scale feature fusion decoder, used to fuse multi-level features extracted by the encoder and, based on dynamically generated feature fusion weights according to the degradation scene category, to perform weighted fusion of shallow detail features and deep semantic features from the encoder.

[0035] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement intelligent real-time underwater image enhancement as described above.

[0036] The aforementioned intelligent real-time underwater image enhancement method, device, and electronic equipment rapidly analyze degradation by introducing a low-dimensional statistical feature set composed of the mean of the whole frame's grayscale histogram, the average value of local contrast space, and the grayscale difference value of the red and blue channels. This feature set is then compared with a preset threshold set to determine degradation scene categories, including low-light-turbidity composite degradation scenes. This achieves millisecond-level rapid identification of input image degradation types with extremely low computational overhead. In particular, it clarifies the independent identification conditions for composite degradation scenes, providing an accurate basis for subsequent targeted processing. This solves the problem of existing solutions lacking a rapid scene recognition mechanism and being unable to distinguish between single and composite degradation. The lightweight Transformer enhancement engine uses a local sparse attention module to restrict global attention calculations to within a non-overlapping window, reducing computational complexity from standard VisionTracker... The O(n²) of the nsformer is reduced to O((H×W)×(K×K)), and the global context information is introduced on the basis of local computation by combining the cross-window information interaction module, thereby overcoming the bottleneck of high computational complexity and difficulty in real-time operation on edge computing platforms of the Transformer model. Finally, by dynamically modulating the internal processing parameters of the enhancement engine (such as the feature fusion weights in the multi-scale feature fusion decoder) according to the identified degradation scene category, a single model can adaptively adjust its enhancement strategy for different degradation scenes. For example, in the case of composite degradation scenes, the brightness enhancement and deblurring operations are coordinated in an orderly manner, realizing the accurate matching and coordination between enhancement processing and specific degradation types. Thus, while ensuring real-time performance, the enhancement effect and adaptability of various underwater degradation scenes, including composite degradation, are comprehensively improved. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1 Here is a flowchart of an example of an intelligent real-time underwater image enhancement method;

[0039] Figure 2 A flowchart illustrating the main steps of an intelligent real-time underwater image enhancement method according to one embodiment;

[0040] Figure 3 A schematic diagram of the network structure for a lightweight Transformer enhancement engine;

[0041] Figure 4 A schematic diagram of an interactive interface for intelligent real-time enhancement of underwater images;

[0042] Figure 5 This is a schematic diagram of an underwater image intelligent real-time enhancement device according to one embodiment;

[0043] Figure 6 This is an internal structural diagram of an electronic device according to one embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] The following is combined Figures 1-6 This invention describes an intelligent real-time underwater image enhancement method, apparatus, and electronic device.

[0046] like Figure 1 and Figure 2 As shown, in one embodiment, an intelligent real-time underwater image enhancement method includes the following steps:

[0047] Step S110: Obtain the underwater image to be processed.

[0048] The image was captured by an underwater high-definition camera carried by a deep-sea submersible and then input into the processing system.

[0049] Step S120: Perform rapid degradation analysis on the underwater image and extract a low-dimensional statistical feature set. The low-dimensional statistical feature set consists of three statistical features: the mean of the grayscale histogram of the entire frame, the average value of the local contrast space, and the grayscale difference value of the red and blue channels.

[0050] Specifically, the calculation of the mean of the whole frame grayscale histogram includes: converting the underwater image into a grayscale image, calculating the arithmetic mean of the grayscale values ​​of all pixels in the grayscale image, which quantitatively assesses the overall illumination level of the image; the calculation of the spatial mean of local contrast includes: dividing the underwater image into multiple equal-sized local regions, calculating the local contrast of each local region, and taking the arithmetic mean of the local contrast of all local regions, which effectively characterizes the overall blurring degree of the image caused by water scattering; the calculation of the grayscale difference value of the red and blue channels includes: extracting the red and blue channels of the underwater image respectively, calculating the difference between the mean of the grayscale values ​​of all pixels in the red channel and the mean of the grayscale values ​​of all pixels in the blue channel, which preliminarily quantifies the degree of color attenuation caused by the strong absorption of red light by seawater. The grayscale conversion can be performed using the formula Y=0.299R+0.587G+0.114B, and the local contrast can be measured by the standard deviation or the difference between the maximum and minimum values ​​of the pixel grayscale values ​​in the region. Compared to traditional multidimensional features, these three low-dimensional statistical features reduce the feature dimension from 18-88 dimensions in traditional methods to 3 dimensions. Through parallel computing, they can be completed in milliseconds, laying the foundation for subsequent real-time processing.

[0051] Step S130: Compare the low-dimensional statistical feature set with the preset threshold set to determine the degradation scene category of the underwater image.

[0052] The preset threshold set is calibrated based on massive amounts of real-world deep-sea image data. The degradation scene categories include at least low-light-turbidity composite degradation scenes, as well as severe low-light scenes, moderate turbidity scenes, and normal scenes. The identification conditions for severe low-light scenes are: the mean of the entire frame's grayscale histogram is below a first threshold and the average local contrast spatial value is not below a second threshold. The identification conditions for moderate turbidity scenes are: the average local contrast spatial value is below a second threshold and the mean of the entire frame's grayscale histogram is not below a first threshold. The identification conditions for low-light-turbidity composite degradation scenes are: the mean of the entire frame's grayscale histogram is below a first threshold and the average local contrast spatial value is below a second threshold. The identification conditions for normal scenes are: the mean of the entire frame's grayscale histogram is not below a first threshold and the average local contrast spatial value is not below a second threshold. In actual deployment, the preset threshold set can also be adaptively learned and dynamically adjusted according to the optical characteristics of different sea areas and depths. This classification method, in particular, identifies composite degradation as an independent category, enabling the enhancement strategy to accurately match complex degradation types and solving the problem that a single model cannot cope with composite degradation.

[0053] Step S140: Dynamically modulate the internal processing parameters of the lightweight Transformer enhancement engine based on the determined degradation scenario category.

[0054] The lightweight Transformer enhancement engine serves as the core processing unit, and its innovative structure references... Figure 3 The system includes a local sparse attention module, a cross-window information interaction module, and a multi-scale feature fusion decoder. The local sparse attention module is used to divide the input feature map into non-overlapping windows and perform attention calculations independently within each window. Specifically, the input feature map is divided into non-overlapping windows of size M×M, where M is an integer greater than 1 (e.g., 16). Attention calculations are performed only within each window. This design reduces the computational complexity of the standard Vision Transformer self-attention mechanism from O((H×W)²) to O((H×W)×(K×K)), where K is the window size. For high-resolution images, the computational cost decreases exponentially, enabling the Transformer model to achieve real-time inference on embedded edge computing platforms. However, focusing solely on local information limits the model's grasp of global information. Therefore, a cross-window information interaction module is introduced to introduce a global information transmission path after the local sparse attention module, enabling information interaction between windows. Specifically, the cross-window information interaction module can employ a shifted window strategy, or introduce a lightweight global downsampling and upsampling path after several local attention layers. This includes a downsampling branch for downsampling the feature map to extract global contextual information, an upsampling branch for restoring the global contextual information to its original resolution, and a fusion unit for fusing the restored global contextual information with local features. The downsampling branch can be implemented using convolutional layers with a stride of 2 or 4, the upsampling branch can be implemented using transposed convolution or interpolated upsampling, and the fusion can be achieved through element-wise addition or channel concatenation. This design effectively compensates for the lack of global perception in local attention while maintaining low computational complexity, ensuring the overall consistency of the enhanced image in terms of brightness, white balance, etc. The multi-scale feature fusion decoder fuses multi-level features extracted by the encoder and dynamically generates feature fusion weights based on the degradation scene category to perform weighted fusion of shallow detail features and deep semantic features from the encoder. Specifically, the multi-scale feature fusion decoder achieves adaptive weighted feature fusion through the following mathematical process:

[0055] Let the features output by the encoder at different scales be shallow detail features. With deep semantic features The output features of the multi-scale feature fusion decoder Calculated using a weighted fusion method:

[0056]

[0057] in, and These represent the fusion weights of shallow detail features and deep semantic features, respectively, and satisfy the normalization constraint:

[0058]

[0059] To dynamically adjust the fusion weights based on the degradation of the input image, a degradation feature vector is first constructed:

[0060]

[0061] in:

[0062] This represents the mean value of the grayscale histogram for the entire frame. For the first grayscale value of each pixel. This represents the total number of pixels in the image.

[0063] Represents the local contrast spatial average, where For the first The standard deviation of gray values ​​in a local area This refers to the number of local regions;

[0064] This represents the difference in grayscale values ​​between the red and blue channels, where and These represent the pixel averages of the red and blue channels, respectively.

[0065] Based on the degenerate feature vector, an intermediate response variable is generated through a linear mapping function:

[0066]

[0067] in For mapping coefficients, This is a bias term.

[0068] The final feature fusion weights are then generated using the Softmax normalization function:

[0069]

[0070] Therefore, the output features of the multi-scale feature fusion decoder can be expressed as:

[0071]

[0072] Through the above dynamic weighting mechanism, when the local contrast... When the weight is low, the system automatically increases the weight of detailed features. To enhance edge and texture recovery capabilities; when the grayscale average When the weights are low, the system automatically increases the semantic feature weights. This enhances global brightness recovery and noise suppression capabilities, thereby enabling adaptive feature fusion for different underwater degradation scenarios.

[0073] Specifically, the feature fusion weights dynamically generated by the multi-scale feature fusion decoder adjust the fusion ratio of shallow detail features and deep semantic features from the encoder based on the degradation scene category. In highly hazy scenes, the system allocates higher weights to detail feature channels that contribute to edge sharpening and texture restoration, while in low-light scenes, it prioritizes semantic feature channels capable of global brightness stretching and noise suppression. The lightweight Transformer enhancement engine employs a multi-scale feature fusion structure, setting local sparse attention modules and cross-window information interaction modules at different resolution levels, achieving effective integration of multi-scale information through upsampling and downsampling. This dynamic fusion mechanism realizes an adaptive enhancement logic of "one structure, multiple strategies." The feedforward network of the entire lightweight Transformer enhancement engine is constructed using depthwise separable convolutions, with the total number of parameters strictly controlled to the million level (e.g., 45M), enabling it to be converted into an efficient engine for inference frameworks such as TensorRT. On embedded platforms such as Jetson Xavier NX, it achieves an end-to-end processing latency of less than 50ms for 1080p input, meeting the 30fps real-time requirement of submersible vision systems.

[0074] Step S150: The underwater image is enhanced using the parameter-modulated lightweight Transformer enhancement engine, and the enhanced image is output.

[0075] Specifically, the dynamic modulation of internal processing parameters based on the determined degradation scene category includes: for low-light scenes, activating a low-light optimization branch, injecting a learnable brightness prior mapping into the model input stage, and strengthening constraints on dark area detail reconstruction and noise suppression during the training stage; for highly turbid scenes, activating a deturbidization branch, guiding the model to strengthen the preservation of mid-to-high frequency information in cross-scale fusion, and using the correlation between color channels to correct color shift, simulating the inverse process of physical scattering; for low-light-turbidity composite degradation scenes, initiating a collaborative processing pipeline, first performing basic brightness enhancement for low-light enhancement to improve overall visibility, and then performing deturbidization on the brightness-corrected image. Internally, this is reflected in a cascade of two-level dynamic weighted fusion, thereby systematically solving the coupling problem of the two degradation factors. For extreme perturbation or resource-constrained scenes, the system also provides a degradable real-time mode. In this mode, the model automatically skips some computationally intensive global interaction layers or reduces the number of iterations to further compress processing time to within 20ms, ensuring uninterrupted visual pipeline operation. Furthermore, referring to... Figure 4 To facilitate monitoring and interaction with the technical solution, the system provides an integrated visual interactive interface that displays the original video stream, identified degradation type, confidence level, processing frame rate, and latency in real time. It also offers one-click switching of preset enhancement schemes based on scene labels and manual fine-tuning of core parameters. Operator adjustments are fed back to the fusion weight generator of the multi-scale feature fusion decoder in real time, enabling human-machine collaborative optimization. All processing results can be stored in real time to form a task dataset. The system may also include an enhancement effect evaluation module, which uses peak signal-to-noise ratio, structural similarity index, and underwater image quality evaluation indicators to evaluate the quality of the output image. The evaluation results can be used to optimize model parameters.

[0076] The aforementioned intelligent real-time underwater image enhancement method, through the synergy of millisecond-level rapid scene diagnosis of low-dimensional features, the fundamental reduction of computational complexity by a lightweight Transformer backbone network, and dynamic feature modulation based on scene labels, constitutes a complete adaptive closed-loop system of "perception-analysis-decision-execution". Under the constraint of embedded computing power, it effectively solves the problem of real-time, accurate, and adaptive enhancement of deep-sea images in low-light, high-turbidity, and complex degradation scenes.

[0077] The underwater image intelligent real-time enhancement device provided by the present invention is described below. The underwater image intelligent real-time enhancement device described below and the underwater image intelligent real-time enhancement method described above can be referred to in correspondence.

[0078] like Figure 5As shown, in one embodiment, an underwater image intelligent real-time enhancement device includes an image acquisition module 510, a degradation feature extraction module 520, a degradation scene recognition module 530, a parameter modulation module 540, and an image enhancement module 550.

[0079] The image acquisition module 510 is used to acquire underwater images to be processed.

[0080] The degradation feature extraction module 520 is used to perform rapid degradation analysis on underwater images and extract a low-dimensional statistical feature set. The low-dimensional statistical feature set consists of three statistical features: the mean of the grayscale histogram of the whole frame, the average value of the local contrast space, and the grayscale difference value of the red and blue channels.

[0081] The degradation scene recognition module 530 is used to compare the low-dimensional statistical feature set with the preset threshold set to determine the degradation scene category of the underwater image. The degradation scene category includes at least the low-light-turbidity composite degradation scene. The recognition conditions for the low-light-turbidity composite degradation scene are: the mean of the grayscale histogram of the whole frame is lower than the first threshold and the average value of the local contrast space is lower than the second threshold.

[0082] The parameter modulation module 540 is used to dynamically modulate the internal processing parameters of the lightweight Transformer enhancement engine according to the determined degradation scene category. The internal processing parameters include the feature fusion weights in the multi-scale feature fusion decoder.

[0083] The image enhancement module 550 is used to enhance the underwater image using a parameter-modulated lightweight Transformer enhancement engine and output the enhanced image.

[0084] The lightweight Transformer enhancement engine includes: a local sparse attention module, which divides the input feature map into non-overlapping windows and performs attention calculations independently within each window; a cross-window information interaction module, which introduces global downsampling and upsampling paths after the local sparse attention module to achieve information transfer between windows; and a multi-scale feature fusion decoder, which fuses multi-level features extracted by the encoder and dynamically generates feature fusion weights based on the degradation scene category to perform weighted fusion of shallow detail features and deep semantic features from the encoder. Figure 6 This example illustrates a schematic diagram of the physical structure of an electronic device, which can be a smart terminal. Its internal structure diagram can be as follows: Figure 6As shown, the electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the underwater image intelligent real-time enhancement method of any of the above embodiments.

[0085] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device to which the present invention is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0086] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the underwater image intelligent real-time enhancement method of any of the above embodiments.

[0087] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, it implements the underwater image intelligent real-time enhancement method of any of the above embodiments.

[0088] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0089] By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for intelligent real-time enhancement of underwater images, characterized in that, The method includes: Acquire underwater images to be processed; A rapid degradation analysis is performed on the underwater image to extract a low-dimensional statistical feature set, which consists of three statistical features: the mean of the gray-level histogram of the entire frame, the average value of the local contrast space, and the gray-level difference value of the red and blue channels. The low-dimensional statistical feature set is compared with a preset threshold set to determine the degradation scene category of the underwater image. The degradation scene category includes at least a low-light-turbidity composite degradation scene. The identification conditions for the low-light-turbidity composite degradation scene are: the mean value of the grayscale histogram of the whole frame is lower than the first threshold and the average value of the local contrast space is lower than the second threshold. Based on the determined degradation scenario category, the internal processing parameters of the lightweight Transformer enhancement engine are dynamically modulated, including feature fusion weights in the multi-scale feature fusion decoder; The underwater image is enhanced using the parameter-modulated lightweight Transformer enhancement engine, and the enhanced image is output. The lightweight Transformer enhancement engine includes: a local sparse attention module, used to divide the input feature map into non-overlapping windows and perform attention calculations independently within each window; a cross-window information interaction module, used to introduce global downsampling and upsampling paths after the local sparse attention module to realize information transmission between windows; and a multi-scale feature fusion decoder, used to fuse multi-level features extracted by the encoder and perform weighted fusion of shallow detail features and deep semantic features from the encoder based on the feature fusion weights dynamically generated according to the degradation scene category.

2. The intelligent real-time underwater image enhancement method according to claim 1, characterized in that, The calculation of the mean of the grayscale histogram of the entire frame includes: converting the underwater image into a grayscale image and calculating the arithmetic mean of the grayscale values ​​of all pixels in the grayscale image; The calculation of the local contrast spatial average includes: dividing the underwater image into multiple local regions of equal size, calculating the local contrast of each local region, and taking the arithmetic mean of the local contrast of all local regions. The calculation of the grayscale difference value of the red and blue channels includes: extracting the red channel and blue channel of the underwater image respectively, and calculating the difference between the mean of the grayscale values ​​of all pixels in the red channel and the mean of the grayscale values ​​of all pixels in the blue channel.

3. The intelligent real-time underwater image enhancement method according to claim 1, characterized in that, The preset threshold set is calibrated based on massive amounts of real-world deep-sea image data; the degradation scene categories also include severely low-light scenes, moderately turbid scenes, and normal scenes.

4. The intelligent real-time underwater image enhancement method according to claim 3, characterized in that, The identification conditions for the severe low-light scene are: the mean value of the grayscale histogram of the entire frame is lower than the first threshold and the average value of the local contrast space is not lower than the second threshold; The recognition conditions for the moderately turbid scene are: the average local contrast spatial value is lower than the second threshold and the average grayscale histogram value of the entire frame is not lower than the first threshold; The identification conditions for the low-light-turbidity composite degradation scene are: the mean value of the grayscale histogram of the entire frame is lower than the first threshold and the average value of the local contrast space is lower than the second threshold; The recognition conditions for the normal scene are: the mean value of the grayscale histogram of the entire frame is not lower than the first threshold and the average value of the local contrast space is not lower than the second threshold.

5. The intelligent real-time underwater image enhancement method according to claim 1, characterized in that, The local sparse attention module divides the input feature map into non-overlapping windows of size M×M, where M is an integer greater than 1, and attention calculation is performed only within each window.

6. The intelligent real-time underwater image enhancement method according to claim 5, characterized in that, The cross-window information interaction module uses a window shifting strategy to achieve information transfer between windows.

7. The intelligent real-time underwater image enhancement method according to claim 5, characterized in that, The cross-window information interaction module includes: a downsampling branch for downsampling the output feature map of the local sparse attention module to extract global context information; an upsampling branch for restoring the global context information to its original resolution; and a fusion unit for fusing the restored global context information with the output feature map of the local sparse attention module.

8. The intelligent real-time underwater image enhancement method according to claim 1, characterized in that, The feature fusion weights dynamically generated by the multi-scale feature fusion decoder adjust the fusion ratio of shallow detail features and deep semantic features from the encoder according to the degradation scene category. Specifically, when dealing with highly murky scenes, higher weights are assigned to the detail feature channels to promote edge sharpening and texture restoration. When dealing with low-light scenes, higher weights are assigned to the semantic feature channels to focus on global brightness stretching and noise suppression.

9. The intelligent real-time underwater image enhancement method according to claim 1, characterized in that, Based on the determined degradation scenario category, the internal processing parameters of the lightweight Transformer enhancement engine are dynamically modulated, including: For low-light scenes, activate the low-light optimization branch and inject brightness prior mapping during the model input stage; For highly turbid scenes, the deturbidity branch is activated to guide the model to simulate the inverse process of physical scattering. For scenarios with combined degradation due to low light and turbidity, a collaborative processing pipeline is initiated. First, a basic brightness enhancement is performed to improve the low light level, and then a deturbidity removal operation is performed.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the underwater image intelligent real-time enhancement method according to any one of claims 1 to 9.

Images