A sparse expert network-based tugboat low-illumination image enhancement method

Abstract

The application discloses a tugboat low-illumination image enhancement method based on a sparse expert network, and relates to the technical field of ship visual perception and image processing. The method acquires a low-illumination image of a driving perspective of a tugboat, and obtains a feature sequence through size normalization, channel standardization, convolution embedding and image block flattening; a structured sparse mask containing global anchor point connection and local neighborhood connection is constructed according to the image region distribution of the water area of the tugboat, and sparse attention is used to encode information such as sea-sky division, port light, distant ship, ship body edge and water surface texture; then, a routing network is used to select Top-k experts for enhancement according to the brightness, texture, edge and contrast features of tokens; and finally, a multi-scale decoder is used to reconstruct the enhanced image. The method can improve the visibility of the low-illumination image of the tugboat and the key target detail retention capability, and reduce the redundant calculation amount, and is suitable for real-time deployment of on-board edge devices.

Description

Technical Field

[0001] This invention relates to the field of ship visual perception and image processing technology, and in particular to a method for enhancing low-light images of tugboats based on sparse expert networks. Background Technology

[0002] With the development of smart ports, intelligent ships, and port-assisted driving technologies, the environmental perception requirements of tugboats in operations such as berthing and unberthing of large ships, towing and escorting, passage through narrow channels, and emergency rescue are constantly increasing. Tugboat driver's perspective images are typically used as a crucial input to shipborne visual perception systems to assist in identifying key objects such as ships, navigation marks, dock facilities, surface obstacles, and distant small targets. However, under complex low-light conditions such as nighttime operations, rain and fog, strong port lighting, sea surface reflection, and backlighting, images captured by shipborne cameras are prone to problems such as insufficient overall brightness, localized overexposure in strong light, unclear outlines of dark hulls, weakened distant small targets, amplified sea surface texture noise, and blurred sea-sky boundaries. These issues affect the effectiveness of manual observation and the reliability of subsequent tasks such as target detection, tracking, and collision avoidance decisions. Therefore, considering the regional differences, lighting variations, and real-time processing requirements of low-light water images from the tugboat driver's perspective, an image enhancement technology solution that can balance enhanced accuracy, structural preservation, robustness, and deployment efficiency is urgently needed.

[0003] To address the poor visibility of low-light images, those skilled in the art typically employ image enhancement techniques such as histogram equalization, gamma correction, Retinex decomposition, frequency domain enhancement, convolutional neural networks, Transformer attention networks, diffusion models, and multi-scale encoding / decoding networks. These methods generally improve the subjective visual quality and objective usability of images by increasing brightness, enhancing contrast, suppressing noise, restoring texture details, or learning the mapping relationship between low-light and normal-light images. In recent years, to reduce the computational cost of global attention and improve the ability to model long-distance dependencies, mechanisms such as sparse attention, window attention, and Top-k attention selection have been introduced. To adapt to different degradation types, some technical solutions also employ multi-branch networks, gated networks, or expert networks to enhance different types of image degradation. However, tugboat driving perspective images are not ordinary low-light images. These images often contain different regions simultaneously, including the sea-sky boundary, port point light sources, sea surface reflections, dark ship hulls, dock backgrounds, water surface textures, and distant small targets. These regions differ significantly in brightness distribution, edge structure, texture features, and enhancement requirements. Existing general enhancement models, lacking a unified modeling of the structure and regional degradation differences in aquatic scenes, still struggle to fundamentally address the coordination issues among highlight suppression, shadow restoration, edge preservation, small target enhancement, and low-latency inference within the same frame.

[0004] A search revealed several existing technologies related to low-light enhancement, sparse attention, or expert enhancement. For example, Chinese invention patent application CN117132500A, entitled "A Low-Light Enhancement Method Based on Sparse Transform Networks," discloses a low-light enhancement method that reduces network redundancy and improves low-light image enhancement results through structures such as feature selection blocks, spatial reconstruction units, channel reconstruction units, NRetinex modules, and hybrid-scale feedforward networks. This method primarily focuses on sparse feature selection, Retinex enhancement, and multi-scale feature supplementation in the low-light image enhancement process. It does not disclose the construction of structured attention connections based on the distribution of regions such as the sea-sky boundary, strong point light sources in the port area, distant ships, ship edges, and water surface textures in images from the tugboat's driving perspective. Nor does it involve a processing mechanism for expert selection based on the degradation types of different token regions within the same frame. Therefore, this method struggles to achieve differentiated enhancement for complex situations where strong light, reflections, dark ship hulls, and distant small targets coexist in low-light water images of tugboats.

[0005] Chinese invention patent application CN117649368A, entitled "A Low-Light Enhancement Method Based on an Adaptive Sparse Attention Diffusion Model," discloses a technical solution that adds noise to normal light images, performs curve fine-tuning on low-light images, performs initial denoising enhancement, and edge detection, and uses a sparse attention diffusion model to iteratively denoise and generate enhanced images during the reverse process. While this solution involves sparse attention and low-light enhancement, its core purpose is to recover image details through noise prediction and iterative denoising using the diffusion model. It does not establish global anchor point connections and local neighborhood connections for the water structure from the tugboat's driving perspective, nor does it disclose the technical means of further token-level expert routing of the sparse attention encoding results according to the region degradation type. Since diffusion models typically involve iterative generation processes, this type of solution may also face problems of high inference latency and resource consumption on shipboard edge devices, making it difficult to directly meet the application requirements of low latency and stable enhancement in tugboat operation scenarios.

[0006] Chinese invention patent CN116342420B, entitled "A Method and System for Enhancing Hybrid Degraded Images," discloses a hybrid degraded image enhancement scheme. This scheme inputs shallow features into a multi-task decoding model, assigns weights to multiple enhancement expert units through a gating network, and obtains deep features through recursive processing. The gating network performs global average pooling on the shallow features and generates weights for the enhancement expert units through a fully connected network and a Softmax function. While this scheme introduces multi-expert processing for general hybrid degraded images, its enhancement target is universal hybrid degraded images. The gating is primarily based on overall feature mapping and expert weight allocation. It does not establish a token-level degradation recognition and Top-k sparse expert selection mechanism for the differences in brightness, texture, edge, and contrast in different spatial regions within the same frame image from a tugboat driver's perspective. Furthermore, it does not disclose using scene structures such as the sea-sky boundary, port lights, and distant ships as attention anchors for sparse connections. Therefore, this scheme is difficult to fully meet the enhancement requirements of low-light images of tugboats, where there are both strong local light and weak dark targets, significant differences in the scale between the background and the target, and dynamic changes in the type of regional degradation.

[0007] Chinese invention patent CN118710537B, entitled "A Low-Light Image Enhancement Method Based on Deep Learning Models," discloses a raw image low-light enhancement technology. It achieves image denoising, color space conversion, and color enhancement through a hybrid feature compensation mechanism, encoder, raw decoder, and sRGB decoder with sparse attention and gated feedforward mechanisms. This solution primarily focuses on noise processing, color rendering, and feature compensation in the raw-to-sRGB enhancement process. While it involves sparse attention and gated feedforward mechanisms, it fails to construct scene constraints around the horizon, port point light sources, hull edges, water surface textures, and distant small targets in tugboat-view water images. It also lacks a sparse expert hybrid enhancement mechanism for different degradation types in the same frame of water images. Therefore, this solution still struggles to address the problems of insufficient utilization of structural context, mismatched local enhancement strategies, and high real-time deployment costs in low-light images of tugboats in complex waterways.

[0008] Furthermore, Chinese invention patent CN110163818B, entitled "A Low-Light Video Image Enhancement Method for Maritime UAVs," discloses a low-light video enhancement scheme for maritime UAVs. This scheme includes steps such as video stabilization, Retinex-based image decomposition, CLAHE enhancement of the illumination component, improved residual convolutional neural network denoising of the reflection component, and Retinex synthesis. While the application scenario of this scheme is related to maritime images, its technical approach is mainly based on video stabilization, Retinex decomposition, histogram enhancement, and convolutional denoising. It does not disclose the construction of a sparse attention mask based on the regional structure of the tugboat's driving perspective, nor does it address token-level expert routing processing for different degradation regions in water images, such as highlights, dark targets, smooth backgrounds, and detailed textures. Therefore, this scheme still lacks adaptability to problems encountered in tugboat onboard vision, such as strong sea surface reflection, weak edges of dark hulls, small scale of distant targets, and limited computational resources.

[0009] In summary, while existing low-light image enhancement techniques can improve brightness, contrast, and detail in general low-light images, mixed degraded images, or some maritime images, they generally suffer from problems such as insufficient utilization of scene priors, lack of structured constraints on contextual connections, inadequate handling of degradation differences between different regions within the same frame, coarse-grained expert enhancement, heavy computational overhead, and insufficient adaptability for real-time shipboard deployment. Especially from the tugboat's driving perspective, information such as the sea-sky boundary, port lights, distant vessels, hull edges, and water surface textures possess significant spatial structural features and directly impact subsequent target identification and safety decision support. Using only general low-light enhancement, ordinary sparse attention, or global gating expert enhancement cannot simultaneously meet the requirements of strong light suppression, dark detail recovery, key contour preservation, small target identification, and low-latency processing. Therefore, overcoming the problems of insufficient utilization of scene structure, inadequate modeling of regional degradation differences, and high real-time deployment costs in existing technologies, and proposing a more accurate, robust, and applicable low-light image enhancement method for tugboats suitable for shipboard edge devices, has become a pressing technical challenge for those skilled in the art. Summary of the Invention

[0010] To address the shortcomings of existing low-light image enhancement techniques in the tugboat driving perspective, such as insufficient utilization of scene structure, inadequate modeling of regional degradation differences, easy over-enhancement of strong port light and sea surface reflections, insufficient restoration of details of dark hulls and distant small targets, and difficulties in real-time deployment of shipborne edge platforms, this paper proposes a tugboat low-light image enhancement method based on sparse expert networks. By constructing a structured sparse attention mask for tugboat water scenes, feature encoding is performed using global anchor point information such as the sea-sky boundary, port lights, and distant ships, as well as local neighborhood information such as hull edges and water surface textures. Combined with a token-level Top-k sparse expert routing mechanism, differentiated enhancement strategies are adopted for different regional degradation types, solving the problem that existing methods cannot simultaneously achieve highlight suppression, dark detail restoration, key target contour preservation, and low-latency inference.

[0011] This invention discloses a low-light image enhancement method for tugboats based on sparse expert networks, specifically comprising: S1: Obtain low-light images from the driving perspective of the tugboat, preprocess the low-light images from the driving perspective of the tugboat to obtain the network input feature sequence; S2: Input the network input feature sequence into the sparse attention encoder, obtain the query matrix, key matrix and value matrix through linear transformation, and construct a structured sparse mask matrix according to the regional distribution characteristics of the low-light image from the perspective of tugboat driving. Use the structured sparse mask matrix to indicate the connection relationship between tokens that allows the calculation of attention, and obtain the encoded features. The structured sparse mask matrix includes global anchor point connections and local neighborhood connections. Global anchor point connections are used to aggregate at least one global semantic information from the sea-sky boundary line, port lights, and distant ships. Local neighborhood connections are used to capture at least one local detail information from the ship hull edge and water surface texture. S3: Input the encoded features into the sparse expert hybrid module, and let the first... The encoding characteristics of each token are: The token's selection probability among multiple experts is obtained through the routing network, and the top-k experts with the highest probabilities are selected to participate in the processing of the current token. Let the set of selected experts be denoted as . ; S4: Let the first An expert subnetwork is Then the first The output representation of each token after processing by the sparse expert hybrid module is as follows: ; in, Indicates the first The output after processing each token by the sparse expert hybrid module Indicates the first The token is allocated to the first The weight of each expert; S5: Input the features processed by the sparse expert hybrid module into the multi-scale decoder, and reconstruct the enhanced tugboat driving perspective image by fusing through step-by-step upsampling and skip connections. The routing network determines the degradation type of the current region based on at least one of the features contained in the token, such as local brightness, texture variation, edge strength, and contrast, so that different regions in the same frame of low-light image of the tugboat can be enhanced with different strategies.

[0012] As a preferred embodiment of the present invention, the low-light image of the tugboat driving perspective in step S1 is represented as follows: ; in, Indicates the image height. 3 represents the image width, and 3 represents the three RGB color channels; Preprocessing includes size normalization, pixel value normalization, channel normalization, convolutional embedding, and image patch division and flattening, resulting in the network input feature sequence. The process is represented as follows: ; in, This means unifying images from different cameras and resolutions to a preset size; dividing by 255 means linearly scaling pixel values ​​from [0,255] to the range [0,1]. This indicates that the mean-variance standardization is performed on each color channel; Indicates convolutional embedding; This represents the image patch division and flattening operation; , Indicates the number of image blocks or tokens. This represents the feature dimension of each token.

[0013] As a preferred embodiment of the present invention, the tugboat low-light image enhancement method is implemented by sequentially processing a sparse attention encoder, a sparse expert hybrid module, and a multi-scale decoder. The overall enhancement network representation is as follows: ; in, For sparse attention encoders, It is a sparse expert hybrid module. For multi-scale decoders, This is an enhanced image of the tugboat's driving perspective.

[0014] As a preferred embodiment of the present invention, the feature is that: in step S2, the input feature of a certain coding layer in the sparse attention encoder is set as follows: ; The query matrix, key matrix, and value matrix are respectively , and The structured sparse mask matrix is The sparse attention output is represented as: ; in, For structured sparse mask matrix, This indicates Hadamard's bitwise multiplication.

[0015] As a preferred embodiment of the present invention, a structured sparse mask matrix is ​​used. Based on the regional distribution characteristics of low-light images from the tugboat driving perspective, three sparse connection modes are comprehensively constructed: global anchor point connection, local neighborhood connection, and causal connection. The global anchor connection is achieved by selecting a global anchor token set. To achieve this, the global anchor token is determined based on the brightness gradient, edge response, spatial location distribution, or equal-interval sampling method of the image patch, covering at least one area among the sea-sky boundary, strong point light sources in the port area, distant ships, and navigation marks. Any token is bidirectionally connected to the global anchor token, i.e.: ; Local neighborhood connections define a local neighborhood for each token. To achieve this, maintain sparse local full connectivity within this local neighborhood, i.e.: ; Causal connections are used in situations where timing or scan order needs to be considered, allowing only past positions to be considered.

[0016] As a preferred embodiment of the present invention, the sparse attention output is further processed by a residual connection and a feedforward network, wherein the feedforward network is represented as follows: ; in, This represents the intermediate features after processing with sparse attention output and residual connections. , , , These are learnable parameters; The encoder output is obtained by cascading multiple sparse attention and feedforward networks and performing multi-scale downsampling, and is represented as follows: ; in, This represents a sparse attention encoder consisting of a cascaded multilayer sparse attention module, residual connections, a feedforward network, and multi-scale downsampling units. This represents the preprocessed network input feature sequence. This represents the encoded features obtained after processing by the sparse attention encoder; the encoded features are used to characterize the global semantic information and local detail information in the low-light image from the tugboat driving perspective, and serve as the input to the subsequent sparse expert hybrid module.

[0017] As a preferred embodiment of the present invention, step S3 is set as follows: The encoding characteristics of each token are: The token is obtained through the routing network. The routing scores of each expert are normalized using the Softmax function to obtain a probability vector. The specific calculation formula is as follows: ; in, and For routing network parameters, Indicates the first The probability of each token being selected by each expert; To balance enhanced capabilities and computational efficiency, only the top-k experts with the highest probabilities are selected to participate in the processing of the current token. The set of selected experts is denoted as . .

[0018] As a preferred embodiment of the present invention, experts include at least two of the following categories: Experts in highlight suppression and noise control for point light sources and reflective areas on the sea surface in port areas; Expert in restoring dark details and enhancing edges in dark hulls, navigation marks, and distant small target areas; Experts in color consistency and texture smoothing for background areas of sky, sea, and dock; Among them, the sparse expert hybrid module also introduces a sparse mask of channel or spatial dimension for the linear transformation within the expert, so that each expert only performs operations on a subset of relevant features, forming a dual sparse structure of Top-k routing sparseness and expert internal sparseness.

[0019] As a preferred embodiment of the present invention, in step S5, the multi-scale decoder is set to... Layer features are The corresponding encoder layer skip connection feature is Then the first The layer decoding process is represented as follows: ; in, For the upsampling operator, This indicates a feature cascading or addition / fusion operation. This represents a reconstruction unit consisting of convolutional layers and nonlinear activations; An enhanced tugboat driving perspective image is obtained by layer-by-layer upsampling and skip connections.

[0020] As a preferred embodiment of the present invention, before performing image enhancement, the overall enhancement network, consisting of a sparse attention encoder, a sparse expert hybrid module, and a multi-scale decoder, is trained using a joint loss function. The joint loss function is expressed as follows: ; in, Indicates the image reconstruction related loss. This indicates the expert equilibrium loss. This indicates a loss of expert diversity. and These are the weighting coefficients for expert equilibrium loss and expert diversity loss, respectively. The image reconstruction loss is represented as: ; in, Indicates pixel-level reconstruction loss. Indicates perceived loss. This represents the total variational loss. , , These are the corresponding weighting coefficients.

[0021] Compared with the relevant prior art, the beneficial effects of the present invention are: Enhancing low-light images from the tugboat's driving perspective can improve the problems of insufficient image brightness, unclear details, and blurred target outlines in nighttime, rain, fog, backlight, and strong light conditions in port areas, thereby improving the visibility of tugboat visual images. By using a structured sparse attention mechanism, global information such as the sea-sky boundary, port lights, and distant ships, as well as local information such as ship edges and water surface textures, can be effectively correlated, thus better preserving key structures and details in tugboat waterway images. By using the sparse expert hybridization module, corresponding experts are selected for enhancement based on the brightness, texture, edge and contrast characteristics of different regions. This allows different enhancement strategies to be applied to highlight areas, shadow areas, background areas and small target areas in the same image, avoiding local overexposure, insufficient shadow recovery or noise amplification caused by uniform enhancement. By employing Top-k expert routing and sparse computation within experts, only some experts and features related to the current region's degradation type are activated, reducing redundant computation and improving model inference efficiency, making it more suitable for deployment on tugboat-borne edge computing platforms. By reconstructing and enhancing images using a multi-scale decoder, both deep semantic information and shallow detail information can be utilized simultaneously to improve the clarity of targets such as ship outlines, navigation marks, dock structures, water surface textures, and distant ships. In summary, this invention can improve the brightness and contrast of low-light images of tugboats while also meeting the needs of highlight suppression, dark detail recovery, key target preservation, and real-time processing, providing more reliable image input for visual perception in low-visibility environments, such as tugboat night navigation, berthing assistance, towing escort, and tugboats. Attached Figure Description

[0022] Figure 1 This is an overall flowchart of a low-light image enhancement method for tugboats based on sparse expert networks according to the present invention. Figure 2 This is a schematic diagram of the sparse attention mechanism in this invention; Figure 3 This is a schematic diagram of the sparse expert hybrid module in this invention; Figure 4 This is a model training diagram for the tugboat low-light image enhancement method based on sparse attention and expert hybrid network of the present invention; Figure 5 This is a comparison of subjective evaluations of typical low-light images of tugboats after enhancement processing using different methods in Embodiment 2 of the present invention. Figure 5 (a) is the original input image. Figure 5 (b) shows the enhanced result of the HAT algorithm. Figure 5 (c) shows the enhanced results of the SwinIR algorithm. Figure 5 (d) shows the enhanced results based on the sparse attention and expert hybrid network algorithm; Figure 6 These are subjective evaluation comparison images of low-light images of the tugboat's far end processed by different methods in Embodiment 3 of the present invention. Figure 6 (a) is the original input image. Figure 6 (b) shows the enhanced result of the HAT algorithm. Figure 6 (c) shows the enhanced results of the SwinIR algorithm. Figure 6 (d) shows the enhanced results based on the sparse attention and expert hybrid network algorithm. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments. However, the present invention can be implemented in many different ways and should not be construed as limited to the embodiments shown; rather, these embodiments provide those skilled in the art with implementation methods that meet applicable legal requirements.

[0024] Example 1: As Figures 1 to 4 As shown, this embodiment provides a low-light image enhancement method for tugboats based on sparse expert networks. This method is applied to the onboard vision system of tugboats to enhance low-light images acquired from the tugboat's driving perspective. It aims to improve issues such as insufficient image brightness, local overexposure, loss of dark details, blurred ship edges, unclear distant small targets, and amplified sea surface noise under complex conditions including nighttime, rain / fog, backlighting, strong point light sources in port areas, and sea surface reflections.

[0025] In this embodiment, the tugboat's onboard vision system includes an image acquisition device installed in the tugboat's bridge, at the front of the bridge, or on top of the hull, and an onboard edge computing unit communicatively connected to the image acquisition device. The image acquisition device can be a visible light camera, a low-light camera, or an onboard surveillance camera with video output capabilities, used to acquire image information of the harbor basin, waterway, wharf front, berthed vessels, water surface obstacles, and distant vessels from the tugboat's driving perspective. The onboard edge computing unit runs the low-light image enhancement model of this embodiment and outputs the enhanced image to the bridge display terminal, or to the target detection, target tracking, obstacle recognition, and collision avoidance decision-making modules.

[0026] This embodiment uses a low-light image captured by a tugboat during nighttime navigation or berthing auxiliary operations in the port area as input. This input image may simultaneously include the sky area, the sea surface area, the ship or dock structure area, the port lights area, and distant ships or navigation marks. Because these areas have different lighting characteristics and image degradation types, using a uniform global enhancement strategy can easily lead to overexposure of port lights, amplification of noise in reflective areas of the sea surface, insufficient restoration of dark ship outlines, and smoothing down of small distant targets. Therefore, this embodiment follows... Figure 1 The method flow shown executes the following steps in sequence: low-light image acquisition and preprocessing of the tugboat, structured sparse attention coding, sparse expert hybrid enhancement, multi-scale decoding and reconstruction, and enhanced image output.

[0027] Step S1: Acquire low-light images from the tugboat driver's perspective and perform preprocessing.

[0028] The low-light image of the tugboat from the driver's perspective, acquired by the image acquisition device, is denoted as: ;in, Indicates the image height. This indicates the image width, and 3 represents the three RGB color channels. This represents a low-light input image from the perspective of a tugboat. In practical applications, images from different cameras or under different acquisition conditions may have different resolutions, such as 640×360, 1280×720, or 1920×1080. This embodiment does not limit the original resolution of the input image.

[0029] To enable images from different sources to be input into the same enhancement network, this embodiment first normalizes the size of the low-light images from the tugboat's driving perspective, uniformly adjusting the input images to a preset size. For example, the image can be adjusted to... The dimensions are determined by H' and W' based on the computing power and real-time requirements of the shipborne edge computing unit. Subsequently, the pixel values ​​are linearly scaled from [0,255] to the [0,1] range, and mean-variance normalization is performed on the three RGB color channels to reduce the impact of differences in exposure parameters, white balance parameters, and sensor response of different cameras on subsequent network processing.

[0030] Next, convolutional embedding layers are used to perform feature mapping on the standardized image. These layers can include convolutional layers with kernel sizes of 3×3 or 5×5, normalization layers, and non-linear activation layers, used to convert the three-channel image into feature maps with a preset feature dimension. The feature maps are then divided and flattened according to a preset image patch size to form the network input feature sequence. This preprocessing can be represented as: ; in, This indicates a size normalization operation. This indicates standardized channel operation. This represents the convolution embedding operation. This represents the image patch division and flattening operation. , Indicates the number of image blocks or tokens. This represents the feature dimension of each token.

[0031] If the input image is adjusted to 640×360 and divided into 8×8 or 16×16 image blocks, a corresponding number of token sequences can be obtained; if a higher resolution input is used, the number of tokens will increase accordingly.

[0032] To preserve spatial location information in the tugboat image, this embodiment can also add positional encoding to the network input feature sequence. Positional encoding can be learnable positional encoding, two-dimensional sine / cosine positional encoding, or relative positional offset. Thus, the network can not only acquire the local image content of each token, but also the spatial location of that token within the entire tugboat image, providing a foundation of positional information for subsequent identification of the sea-sky boundary, port lights, ship edges, and distant targets.

[0033] The overall enhancement network in this embodiment includes a sparse attention encoder, a sparse expert hybrid module, and a multi-scale decoder connected in sequence. The overall enhancement process can be represented as follows: ;in, For sparse attention encoders, It is a sparse expert hybrid module. For multi-scale decoders, This is an enhanced image of the tugboat's driving perspective.

[0034] Step S2: Input the network input feature sequence into the sparse attention encoder for structured sparse attention encoding.

[0035] like Figure 2 As shown, the sparse attention encoder in this embodiment does not perform fully connected attention calculations on all tokens. Instead, it constructs a structured sparse mask matrix based on the regional distribution characteristics of the low-light image from the tugboat driving perspective. This structured sparse mask matrix is ​​used to indicate whether attention calculations are allowed between different tokens, allowing the network to focus on connections in key regions related to tugboat visual enhancement, thereby reducing redundant calculations between irrelevant regions.

[0036] Let the input features of a certain coding layer in a sparse attention encoder be: The query matrix is ​​obtained through linear transformation. Key matrix Sum matrix ,Right now: ;in, , , All of these are learnable parameter matrices.

[0037] The correlation between tokens is calculated based on the query matrix and the key matrix, and a structured sparse mask matrix is ​​used. Controlling the scope of attention calculation. To avoid unconnected locations still participating in Softmax normalization, in a preferred implementation, this embodiment uses an additive masking method for sparse attention calculation.

[0038] set up ,when When, it indicates the first The token allows you to follow the number 1 One token; when When, it indicates the first The token does not care about the first one. Each token. Based on... Constructing an additive mask matrix ,in: ; The output of sparse attention is then represented as: ; M includes global anchor point connections and local neighborhood connections, and may also include causal connections when processing consecutive video frames or enhancing in scan order is required. Global anchor point connections are used to aggregate global semantic information in tugboat driver's view images. In low-light images of tugboats, the sea-sky boundary, strong point light sources in the port area, distant ships, navigation marks, and the outline of the dock typically play a crucial role in determining the overall illumination and restoring the structure of the image. This embodiment selects a set of global anchor point tokens. Construct global anchor point connections. Global anchor point tokens can be determined based on image patch brightness gradients, edge responses, spatial distribution, or equal-interval sampling methods.

[0039] First, the average brightness, brightness gradient, and edge response of the image patch corresponding to each token are calculated. Then, tokens with obvious horizontal edge responses are selected as candidate anchor points for the sea-sky boundary in the upper and middle regions of the image. Candidate anchor points for port area point light sources are selected in areas with high brightness and prominent local contrast, and candidate anchor points for distant ships or navigation marks are selected in small-scale high-response areas in the distance. Finally, a small number of spatial anchor points are added using an equal-interval sampling method to avoid excessive concentration of anchor points in local areas. The selected global anchor point tokens constitute a set. .

[0040] For any token, a bidirectional connection is maintained with the global anchor token, that is: ; Through the aforementioned global anchor point connections, the dark areas of the hull, the water surface, and distant target areas can all be linked to global semantic regions such as the sea-sky boundary, port lights, and distant ships. This allows the augmentation network to adjust brightness and restore details based on the overall lighting structure of the tugboat image. Local neighborhood connections are used to capture local detail information in the tugboat image.

[0041] In low-light images of tugboats, hull edges, deck edges, mooring equipment, water surface textures, dock structures, and small object outlines are often distributed in locally adjacent areas. Relying solely on global anchor points may weaken local textures. Therefore, this embodiment defines a local neighborhood for each token. And maintain sparse local full connectivity within the local neighborhood, that is: ;in, It can be the first The local neighborhood centered on a token can be a fixed window or a directional neighborhood determined by the direction of the image edge, the direction of the water texture, or the direction of the ship's outline. For example, for a token located near the edge of the ship, the local neighborhood can be expanded along the direction of the ship's edge; for a token located in the sea area, the local neighborhood can be expanded along the direction of the water texture; and for a token located near the sea-sky boundary, the local neighborhood can be expanded along an approximate horizontal line.

[0042] Through the aforementioned local neighborhood connections, the sparse attention encoder can better maintain the continuity of hull edges, the naturalness of water surface textures, and the clarity of dock structures. Causal connections are used to ensure inference stability in scenarios where temporal or scanning order needs to be considered. For example, when a tugboat's onboard vision system performs real-time enhancement on consecutive video frames or processes high-resolution image patches in line-by-line scanning order, the current token can focus only on tokens that are in past positions in terms of time or scanning order, thereby reducing the risk of future information leakage or instability in local enhancements.

[0043] In single-frame static image enhancement scenarios, causal connections can be disabled or used as optional sparse connection branches to participate in mask matrix construction. After sparse attention output, this embodiment further employs residual connections, normalization layers, and a feedforward network for feature updating. Specifically, a certain coding layer may include a sparse attention sublayer and a feedforward network sublayer, as shown below: ; ; in, This indicates a normalization operation. This represents a feedforward network.

[0044] Feedforward networks can be constructed using two linear transformations and a nonlinear activation function, as follows: ; in, , , , These are learnable parameters.

[0045] By cascading multiple sparse attention and feedforward networks, and in conjunction with multi-scale downsampling operations, this embodiment progressively extracts brightness structure, sea surface reflection, ship outline, port lights, and distant small target features from low-light tugboat images, obtaining the encoder output: .

[0046] Step S3: Input the encoder output into the sparse expert hybrid module to adaptively enhance the degradation type of different regions.

[0047] like Figure 3As shown, the sparse expert hybrid module in this embodiment includes a routing network and multiple expert sub-networks. Unlike methods that apply the same enhancement strategy to the entire image, this embodiment selects experts for each token based on the degradation differences in different regions of the same frame of low-light image of a trolley, thus applying different enhancement strategies to different regions.

[0048] Let the first The encoding characteristics of each token are: The token is obtained through the routing network. The routing scores on each expert are normalized using the Softmax function to obtain a probability vector: ;in, and For routing network parameters, Indicates the first Each token represents the probability of selection by each expert. To balance enhanced capabilities and computational efficiency, this embodiment does not activate all experts, but only selects the top-k experts with the highest probabilities to participate in the processing of the current token. The set of selected experts is denoted as . Number of experts It can be set to 4 to 8, and Top-k can be set to 1 or 2.

[0049] When Top-k is 1, each token is processed by only the best matching expert, resulting in low computational cost. When Top-k is 2, each token is processed by the two best-matched experts, resulting in better enhancement capabilities and robustness.

[0050] This embodiment preferably uses a Top-k ratio of 2, allowing a single token to simultaneously combine the outputs of both the dominant and auxiliary reinforcement experts. Let the... An expert subnetwork is Then the first The output of each token after processing by the sparse expert hybrid module is represented as follows: ; in, Indicates the first The output after processing each token by the sparse expert hybrid module Indicates the first The token is allocated to the first The weight of each expert.

[0051] In this embodiment, the routing network determines the degradation type of the current region based on at least one feature of the token, including local brightness, texture variation, edge strength, and contrast, and selects the corresponding expert accordingly. Specifically, for port point light sources, light reflections, and areas with strong sea surface reflections, these areas typically have high brightness, strong local contrast, unstable edges, and easily amplified noise. The routing network tends to select experts in highlight suppression and noise control to reduce overexposure and halo artifacts. For dark hulls, navigation marks, distant ships, and small target areas, these areas typically have low brightness, weak texture, fine edges, and are easily submerged by the background. The routing network tends to select experts in dark detail restoration and edge enhancement to improve the visibility of hull outlines, navigation mark edges, and distant ships. For large areas such as the sky, sea surface, and dock background, these areas typically need to maintain smooth color and natural texture. The routing network tends to select experts in color consistency and texture smoothing to avoid obvious color casts, block artifacts, or excessive amplification of sea surface noise after enhancement.

[0052] Multiple experts include at least two or more of the following types: The first category is experts in highlight suppression and noise control, used to handle point light sources, light reflections, and reflective areas on the sea surface in the port area. The second category is experts in dark detail restoration and edge enhancement, used to handle dark hulls, navigation marks, mooring equipment, deck edges, and small distant target areas; The third category is color consistency and texture smoothing experts, used to process background areas such as sky, sea, and dock.

[0053] Each expert subnetwork can be constructed using lightweight convolutional layers, depthwise separable convolutional layers, local window attention layers, gated feedforward layers, or combinations thereof.

[0054] To further reduce computational load, this embodiment can introduce sparse masks of channel or spatial dimensions within each expert, allowing each expert to perform calculations only on a subset of features relevant to its function. For example, a specular highlight suppression expert can focus on brightness channels, local contrast features, and high-frequency noise features; a dark detail recovery expert can focus on edge response features, texture features, and low-brightness area features; and a color consistency expert can focus on color channel relationships and low-frequency background features. Thus, this embodiment forms a dual sparse structure combining Top-k routing sparseness and expert-internal sparseness, reducing redundancy caused by irrelevant experts and lowering the computational load of invalid features within a single expert, making it more suitable for deployment on tugboat-borne edge computing platforms.

[0055] Step S4: Input the features processed by the sparse expert hybrid module into the multi-scale decoder to reconstruct the enhanced tugboat driving perspective image.

[0056] In this embodiment, the multi-scale decoder is used to recover spatial image features from the token features output by the sparse expert hybridization module, and to reconstruct and enhance the image step by step. Let the multi-scale decoder be... Layer features are The corresponding encoder layer skip connection feature is Then the first The layer decoding process is represented as follows: ; in, This indicates an upsampling operator, which can be bilinear interpolation with convolution, deconvolution, or subpixel convolution. This indicates a feature cascading or addition / fusion operation; This represents a reconstruction unit consisting of convolutional layers, normalization layers, and nonlinear activation layers.

[0057] By employing layer-by-layer upsampling and skip connection fusion, the multi-scale decoder can simultaneously utilize global semantic information from the encoder's deep features and local spatial detail information from its shallow features. Deep features are beneficial for recovering the overall image's brightness distribution, sea-sky boundary, port lighting layout, and ship outline relationships; shallow features are beneficial for preserving hull sidewall textures, deck edges, mooring equipment, water surface textures, dock structures, and details of distant small targets. Finally, the multi-scale decoder outputs an enhanced tugboat driving perspective image. .

[0058] Output image With input image It has the same resolution and is used to achieve low-light enhancement; Output image The resolution is higher than the input image, which is used for detail reconstruction or super-resolution restoration while enhancing low-light performance.

[0059] Regardless of the output method used, the basic process of structured sparse attention coding, sparse expert hybrid enhancement, and multi-scale decoding reconstruction described in this embodiment remains unchanged.

[0060] Step S5: Train the overall augmentation network.

[0061] During the model training phase, this embodiment selects low-light images collected by tugboats in various scenarios, including harbor basins, waterways, berthing, escorting, night navigation, rain and fog, backlighting, and strong port lighting, as training samples. Training samples can include actual low-light images and their corresponding reference images, or low-light-normal-light image pairs constructed through exposure adjustment, brightness attenuation, noise superposition, and illumination perturbation. Through training with multi-scenario samples, the network can adapt to image degradation under different lighting conditions, sea conditions, and target distances during actual tugboat operations.

[0062] Let the network output enhancement result be The reference image is This embodiment uses a joint loss function to train the overall augmentation network consisting of a sparse attention encoder, a sparse expert hybrid module, and a multi-scale decoder. The joint loss function is expressed as: ; in, Indicates the image reconstruction related loss. This indicates the expert equilibrium loss. This indicates a loss of expert diversity. and These are the weighting coefficients for expert equilibrium loss and expert diversity loss, respectively. The image reconstruction loss is represented as: ; in, This represents the pixel-level reconstruction loss, used to constrain the consistency between the enhanced image and the reference image in terms of brightness, color, and pixel structure. This represents perceptual loss and is used to improve the texture and contour representation of key targets such as hull edges, navigation marks, dock structures, and distant ships. This represents the total variational loss, used to suppress noise amplification, halo, or local texture breakage caused by excessive enhancement in reflective areas of the sea surface, dark areas, and near point light sources in the port area; , , These are the corresponding weighting coefficients.

[0063] Expert equalization loss is used to prevent the routing network from being biased towards a small number of experts in the long term, which would lead to the overuse of some experts and the undertraining of others. By introducing expert equalization loss, different experts can have the opportunity to participate in the enhancement processing of point light sources in the port area, sea surface reflections, dark ship hulls, distant small targets and background areas, thus improving the stability of the sparse expert hybrid module.

[0064] Expert diversity loss is used to reduce excessive similarity between the output features of different experts, enabling different experts to form differentiated divisions of labor in aspects such as highlight suppression, shadow restoration, noise control, color correction and texture reconstruction, thereby enhancing the adaptive processing capability of different regions within the same frame of tugboat image.

[0065] like Figure 4As shown, during the training process, a low-light tugboat image is input into the network, and forward propagation is completed according to steps S1 to S4 to obtain an enhanced image. Then, the loss value is calculated according to the joint loss function, and the learnable parameters in the sparse attention encoder, routing network, expert sub-network, and multi-scale decoder are updated through backpropagation. After multiple rounds of iterative training, an enhanced model that can adapt to complex low-light images from the tugboat driving perspective is obtained.

[0066] Step S6: Deploy the trained model on the tugboat's onboard edge computing unit and perform real-time augmentation output.

[0067] In practical application, such as Figure 4 As shown, the trained low-light image enhancement model for tugboats is deployed on the tugboat's onboard edge computing unit. During nighttime berthing, port navigation, escort towing, or emergency rescue operations, the image acquisition device captures real-time images from the tugboat's driver's perspective and sends these images to the onboard edge computing unit. The onboard edge computing unit performs forward inference according to steps S1 to S4 in this embodiment, outputting the enhanced tugboat driver's perspective image.

[0068] The enhanced images can be directly displayed on the cockpit display terminal, enabling the driver to observe targets such as ship outlines, navigation marks, dock boundaries, water surface obstacles, mooring equipment, and distant ships more clearly; they can also be used as input for subsequent intelligent vision perception modules for target detection, ship tracking, obstacle recognition, channel area judgment, and collision avoidance decision-making assistance.

[0069] Because this embodiment uses structured sparse attention to reduce full-connection computation between irrelevant tokens and Top-k sparse expert routing to reduce redundancy of computation involving all experts, it can reduce the amount of computation and storage access during model inference while ensuring low-light enhancement, making it more suitable for shipborne edge computing platforms to operate under low-latency conditions.

[0070] Through the above implementation method, this embodiment incorporates global anchor point information such as the sea-sky boundary, port lights, and distant ships in the tugboat's driving perspective image, along with local detail information such as hull edges, water surface textures, and deck structures, into the sparse attention encoding process. Furthermore, it utilizes a sparse expert hybrid module to apply differentiated enhancement strategies to highlight areas, dark target areas, background areas, and textured areas. Compared to global brightness adjustment, ordinary Retinex enhancement, dense self-attention enhancement, or whole-image-level expert enhancement, this embodiment better adapts to the characteristics of multiple regions coexisting in low-light tugboat images, the coexistence of strong and weak light, significant differences in target scale, and high real-time requirements onboard, thereby improving the brightness naturalness, structural clarity, local detail preservation ability, and real-time deployment adaptability of the enhanced image.

[0071] It should be noted that the specific settings for input resolution, image patch size, token feature dimension, number of experts, Top-k values, upsampling method, loss function weights, and number of network layers in this embodiment are merely optional implementations for illustrating the technical solution of this invention and do not constitute a limitation on the scope of protection of this invention. Those skilled in the art can make equivalent adjustments to the above parameters based on the tugboat camera parameters, the computing power of the shipborne edge computing unit, the port area environment, and real-time processing requirements. As long as it still employs structured sparse attention coding based on the tugboat driver's viewpoint area distribution, sparse expert hybrid enhancement based on token degradation type, and a multi-scale decoding and reconstruction process, it should fall within the scope of protection of this invention.

[0072] Example 2: This example illustrates the application effect of the tugboat low-light image enhancement method based on sparse expert networks in low-light image enhancement scenarios for distant tugboat targets. This example selects a distant image of the "Yungang Electric Tug No. 2" tugboat built by Lianyungang Port Barge Hongyun Industrial Co., Ltd. as the image to be enhanced. When tugboats operate at night in the port, navigate in low visibility, or are observed from a distance, the area occupied by distant tugboat targets in the image is relatively small. Affected by low light, sea surface reflection, background interference, and imaging noise, problems such as unclear target edges, weakened local textures, and insufficient hull structure discernibility easily occur, thus affecting the operator's observation of the distant vessel's status and also impacting the input quality of subsequent target detection and recognition modules.

[0073] In this embodiment, the low-light image of the far end of the "Yungang Electric Tug No. 2" tugboat is used as the input image. Following the method of this invention, image preprocessing, sparse attention encoding, sparse expert hybrid enhancement, and multi-scale decoding reconstruction are performed sequentially to obtain the enhanced image of the far end of the tugboat. First, the input image undergoes size normalization, pixel value normalization, channel normalization, convolutional embedding, and image block partitioning and flattening to obtain the network input feature sequence. Then, this network input feature sequence is input into the sparse attention encoder, and a structured sparse mask matrix is ​​constructed based on the spatial distribution relationship of the target region, background region, and water environment in the far end image of the tugboat. This allows the far end tugboat target, hull outline, background structure, and water surface texture to interact within the effective connection range.

[0074] In the sparse attention encoding process, global anchor connections in the structured sparse mask matrix are used to preserve the correlation between the distant tugboat target and the overall lighting structure of the image, enabling the model to enhance the distant target by combining the overall brightness distribution and the background environment. Local neighborhood connections are used to maintain the continuity of edges, textures, and contours near the distant tugboat target, avoiding weakening the target boundary due to excessive smoothing during enhancement. Through the above processing, the sparse attention encoder can retain structural information and local details that are helpful for the recognition of the distant tugboat target while reducing redundant computation in irrelevant areas.

[0075] The encoded features output by the sparse attention encoder are input into the sparse expert mixing module. Since images of distant tugboats typically contain simultaneously dark target areas, background areas, and potentially reflective water areas, the degradation patterns of these different areas are not consistent. Therefore, this invention uses a routing network to assess the local brightness, texture variations, edge strength, and contrast of each token, and selects the top-k experts with higher probabilities to participate in the enhancement processing of the corresponding token. For the hull, outline edges, and local structural areas of the distant tugboat, experts focusing on dark detail recovery and edge enhancement are prioritized; for water or background areas, experts focusing on color consistency and texture smoothing are prioritized; and for areas with local highlights or reflections, experts focusing on highlight suppression and noise control are prioritized. Thus, this embodiment can employ differentiated enhancement strategies for different areas in the same low-light image of a distant tugboat, rather than applying uniform enhancement to the entire image.

[0076] The features processed by the sparse expert hybrid module are then input into the multi-scale decoder, where they are fused through progressive upsampling and skip connections to reconstruct and enhance the image. During the reconstruction process, the multi-scale decoder simultaneously utilizes the overall illumination and semantic structure information from deep features, as well as the edge, texture, and spatial detail information from shallow features, thereby improving the structural clarity of the distant tugboat target and the overall image visibility.

[0077] In the specific experiment, a typical low-light scene image was extracted from the actual navigation video of "Yungang Electric Tug No. 2" as input. This image was then input into the classic low-light enhancement network HAT, SwinIR, and the tugboat low-light image enhancement method based on a sparse attention and expert hybrid network proposed in this invention, respectively. The enhancement results of the comparative algorithms and the method of this invention were obtained. To visually compare key details such as the hull sidewall structure and mooring equipment, local areas of the image were enlarged by red boxes. The subjective visual comparison results are as follows: Figure 5 As shown.

[0078] like Figure 5As can be seen, under the same tugboat perspective and sea conditions, the results of HAT and SwinIR still show a certain degree of blurring or overexposure in the hull sidewalls and mooring equipment areas. However, the method of this invention, while maintaining an overall brightness improvement, can more clearly restore details such as side railings, deck edges and distant background ships, with sharper local textures and effective suppression of noise and halo artifacts.

[0079] To further quantify the enhancement effects of different methods, this embodiment simultaneously uses three objective metrics—Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Mean Squared Error (MSE)—to evaluate the enhancement results. The evaluation results are shown in Table 1. PSNR is a commonly used objective metric to measure the difference between the reconstructed image and the reference image; a higher value indicates higher image quality. SSIM is the structural similarity index, ranging from 0 to 1. A SSIM value closest to 1 indicates less image distortion and better quality. MSE reflects the magnitude of the error between the predicted image and the real image; a lower value indicates better restoration. Experimental results show that in the typical low-light scenario of the tugboat, the method of this invention outperforms the comparison algorithm in both PSNR and SSIM, and is significantly lower in MSE. Figure 5 The subjective visual effect in the image can verify that the present invention can significantly improve the clarity and detail of low-light images from the perspective of tugboats, providing more reliable visual perception support for the safe navigation of tugboats under low visibility conditions.

[0080] Table 1: Comparison of Objective Evaluation Indicators under Different Algorithms

[0081] Subjective evaluation shows that the algorithm of this invention can preserve the details of the original image and can accurately enhance low-light images from the perspective of a tugboat. Objective evaluation metrics show that the algorithm of this invention has the highest PSNR value, indicating high enhanced image quality. The SSIM value is closest to 1, indicating no image distortion and good quality. The lowest MSE value indicates that the difference between the predicted result of this invention and the true value of the original image is minimal, meaning that this invention performs better. This comparative experiment demonstrates that this invention can achieve effective and accurate enhancement of low-light images from the perspective of a tugboat.

[0082] Example 3: A distant image of the "Yungang Electric Tug No. 2" tugboat, built by Lianyungang Port Tugboat Hongyun Industrial Co., Ltd., was selected for verification. Tugboats frequently encounter low-visibility scenarios, such as nighttime, while operating in the port. The distant image of the "Yungang Electric Tug No. 2" tugboat was used for enhancement. First, the distant image of the tugboat was selected as input, and enhancement was performed using the algorithm of this invention. Comparative experiments were conducted using classic low-light enhancement networks HAT and SwinIR, as well as the tugboat low-light image enhancement method based on a sparse attention and expert hybrid network proposed in this invention. The enhancement effect of the algorithm was evaluated through subjective and objective evaluations. The subjective evaluation results are as follows: Figure 6 As shown, where, Figure 6 (a) is the original input image. Figure 6 (b) shows the enhanced result of the HAT algorithm. Figure 6 (c) shows the enhanced results of the SwinIR algorithm. Figure 6 (d) is an enhanced result of the method of the present invention.

[0083] Depend on Figure 6 As can be seen, the original input image of the distant tugboat is affected by low illumination, resulting in poor overall visibility and making it difficult to clearly identify the distant tugboat target and its local structure. While the HAT and SwinIR algorithms can improve image brightness and contrast to some extent, they still fall short in preserving the structure and restoring details of distant targets. The enhanced results may exhibit insufficient local detail prominence or inadequate distinction between the background and the target. In contrast, the method of this invention, while improving overall brightness, focuses more on edge enhancement and detail restoration of the distant tugboat target area. Furthermore, it uses a sparse expert hybridization module to differentiate between different degraded regions, making the enhanced image of the distant tugboat clearer in terms of target contours, local textures, and overall layering. This is beneficial for observing and identifying distant ship targets under low illumination conditions.

[0084] This embodiment uses Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Indicator (SSIM), and Mean Square Error (MSE) to objectively evaluate the enhancement results of different methods. The evaluation results are shown in Table 1. Table 2: Comparison Results Chart

[0085] As shown in Table 2, on the low-light image of the distant tugboat selected in this embodiment, the PSNR of the method of the present invention is 21.5336, which is higher than that of the HAT algorithm (13.8680) and the SwinIR algorithm (17.3437), indicating that the enhanced image obtained by the method of the present invention has higher reconstruction quality than the reference image. Compared with the HAT algorithm, the PSNR of the method of the present invention is improved by 7.6656; compared with the SwinIR algorithm, the PSNR of the method of the present invention is improved by 4.1899, indicating that the method of the present invention has better image restoration capability on the low-light image of the distant tugboat.

[0086] From the perspective of structural similarity metrics, the SSIM of the method of this invention is 0.8920, which is higher than that of the HAT algorithm (0.6331) and the SwinIR algorithm (0.7859). The closer the SSIM is to 1, the closer the enhanced image is to the reference image in terms of structural, brightness, and contrast information. Therefore, the above results show that the method of this invention can better preserve the target structural information and reduce structural distortion caused by enhancement processing during the low-light image enhancement of the distant tugboat, which is beneficial for preserving the contour and local details of the distant tugboat target.

[0087] From the perspective of mean squared error (MSE), the method of this invention has an MSE of 456.80, which is lower than the 2668.60 of the HAT algorithm and the 1198.69 of the SwinIR algorithm. A smaller MSE indicates a smaller pixel error between the enhanced result and the reference image. These results demonstrate that the method of this invention has lower pixel-level restoration error on the image of the distant tugboat, and the enhanced result has better objective consistency. Combined with... Figure 6 The subjective evaluation results shown indicate that the method of the present invention can not only improve the overall visibility of low-light images of distant tugboats, but also improve the clarity of target edges and local structures.

[0088] Experimental results in this embodiment demonstrate that, in low-light image enhancement scenarios for distant tugboats, the method of this invention effectively associates the distant target region with the overall illumination structure, background region, and local neighborhood information through a structured sparse attention mechanism, avoiding the weakening of target details caused by relying solely on global brightness enhancement. Simultaneously, through Top-k sparse expert routing based on token degradation types, different image regions can invoke expert sub-networks matching their degradation features for enhancement, thereby improving the structural clarity of the distant tugboat target and the overall image quality. Compared to the HAT and SwinIR algorithms, the method of this invention achieves higher PSNR and SSIM, and lower MSE in this embodiment, indicating its good applicability and stability in low-light image enhancement tasks for distant tugboats.

[0089] This embodiment uses a distant image of the "Yungang Electric Tractor No. 2" tugboat as an example to illustrate the specific application and experimental effects of the method of the present invention in low-light enhancement scenarios for distant targets. For distant vessel images acquired from other tugboats, other port areas, at different shooting distances, with different camera angles, and under low-light conditions such as nighttime, rain, fog, backlighting, or complex port lighting, those skilled in the art can achieve image enhancement by using the same or equivalent preprocessing methods, structured sparse attention coding methods, sparse expert hybrid enhancement methods, and multi-scale decoding and reconstruction methods without changing the basic process of the present invention, and all of these should fall within the scope of the present invention.

Claims

1. A method for enhancing low-light images of tugboats based on sparse expert networks, characterized in that: Includes the following steps: S1: Obtain low-light images from the driving perspective of the tugboat, preprocess the low-light images from the driving perspective of the tugboat to obtain the network input feature sequence; S2: Input the network input feature sequence into the sparse attention encoder, obtain the query matrix, key matrix and value matrix through linear transformation, and construct a structured sparse mask matrix according to the regional distribution characteristics of the low-light image from the perspective of tugboat driving. Use the structured sparse mask matrix to indicate the connection relationship between tokens that allows the calculation of attention, and obtain the encoded features. The structured sparse mask matrix includes global anchor point connections and local neighborhood connections. Global anchor point connections are used to aggregate at least one global semantic information from the sea-sky boundary line, port lights, and distant ships. Local neighborhood connections are used to capture at least one local detail information from the ship hull edge and water surface texture. S3: Input the encoded features into the sparse expert hybrid module, and let the first... The encoding characteristics of each token are: The token's selection probability among multiple experts is obtained through the routing network, and the top-k experts with the highest probabilities are selected to participate in the processing of the current token. Let the set of selected experts be denoted as . ; S4: Let the first An expert subnetwork is Then the first The output representation of each token after processing by the sparse expert hybrid module is as follows: ； in, Indicates the first The output after processing each token by the sparse expert hybrid module Indicates the first The token is allocated to the first The weight of each expert; S5: Input the features processed by the sparse expert hybrid module into the multi-scale decoder, and reconstruct the enhanced tugboat driving perspective image by fusing through step-by-step upsampling and skip connections. The routing network determines the degradation type of the current region based on at least one of the features contained in the token, such as local brightness, texture variation, edge strength, and contrast, so that different regions in the same frame of low-light image of the tugboat can be enhanced with different strategies.

2. The method for enhancing low-light images of tugboats based on sparse expert networks according to claim 1, characterized in that: The low-light image from the tugboat driving perspective in step S1 is represented as follows: ； in, Indicates the image height. 3 represents the image width, and 3 represents the three RGB color channels; Preprocessing includes size normalization, pixel value normalization, channel normalization, convolutional embedding, and image patch division and flattening, resulting in the network input feature sequence. The process is represented as follows: ； in, This means unifying images from different cameras and resolutions to a preset size; dividing by 255 means linearly scaling pixel values ​​from [0,255] to the range [0,1]. This indicates that the mean-variance standardization is performed on each color channel; Indicates convolutional embedding; This represents the image patch division and flattening operation; , Indicates the number of image blocks or tokens. This represents the feature dimension of each token.

3. The method for enhancing low-light images of tugboats based on sparse expert networks according to claim 2, characterized in that: The low-light image enhancement method for tugboats is implemented by sequentially processing a sparse attention encoder, a sparse expert hybrid module, and a multi-scale decoder. The overall enhancement network representation is as follows: ； in, For sparse attention encoders, For sparse expert hybrid modules For multi-scale decoders, This is an enhanced image of the tugboat's driving perspective.

4. The method for enhancing low-light images of tugboats based on sparse expert networks according to claim 1, characterized in that: In step S2, let the input features of a certain coding layer in the sparse attention encoder be: ； The query matrix, key matrix, and value matrix are respectively , and The structured sparse mask matrix is The sparse attention output is represented as: ； in, For structured sparse mask matrix, This indicates Hadamard's bitwise multiplication.

5. The method for enhancing low-light images of tugboats based on sparse expert networks according to claim 4, characterized in that: Structured sparse mask matrix Based on the regional distribution characteristics of low-light images from the tugboat driving perspective, three sparse connection modes are comprehensively constructed: global anchor point connection, local neighborhood connection, and causal connection. The global anchor connection is achieved by selecting a global anchor token set. To achieve this, the global anchor token is determined based on the brightness gradient, edge response, spatial location distribution, or equal-interval sampling method of the image patch, covering at least one area among the sea-sky boundary, strong point light sources in the port area, distant ships, and navigation marks. Any token is bidirectionally connected to the global anchor token, i.e.: ； Local neighborhood connections define a local neighborhood for each token. To achieve this, maintain sparse local full connectivity within this local neighborhood, i.e.: ； Causal connections are used in situations where timing or scan order needs to be considered, allowing only past positions to be considered.

6. The method for enhancing low-light images of tugboats based on sparse expert networks according to claim 4, characterized in that: The sparse attention output is further processed by a feedforward network via residual connections. The feedforward network is represented as follows: ； in, This represents the intermediate features after processing with sparse attention output and residual connections. , , , These are learnable parameters; The encoder output is obtained by cascading multiple sparse attention and feedforward networks and performing multi-scale downsampling, and is represented as follows: ； in, This represents a sparse attention encoder consisting of a cascaded multilayer sparse attention module, residual connections, a feedforward network, and multi-scale downsampling units. This represents the preprocessed network input feature sequence. This represents the encoded features obtained after processing by the sparse attention encoder. The encoded features are used to characterize the global semantic information and local detail information in the low-light image from the tugboat driving perspective, and serve as the input to the subsequent sparse expert hybrid module.

7. The method for enhancing low-light images of tugboats based on sparse expert networks according to claim 1, characterized in that: In step S3, let the first The encoding characteristics of each token are: The token is obtained through the routing network. The routing scores of each expert are normalized using the Softmax function to obtain a probability vector. The specific calculation formula is as follows: ； in, and For routing network parameters, Indicates the first The probability of each token being selected by each expert; To balance enhanced capabilities and computational efficiency, only the top-k experts with the highest probabilities are selected to participate in the processing of the current token. The set of selected experts is denoted as . .

8. The method for enhancing low-light images of tugboats based on sparse expert networks according to claim 7, characterized in that: Experts include at least two of the following categories: Experts in highlight suppression and noise control for point light sources and reflective areas on the sea surface in port areas; Expert in restoring dark details and enhancing edges in dark hulls, navigation marks, and distant small target areas; Experts in color consistency and texture smoothing for background areas of sky, sea, and dock; Among them, the sparse expert hybrid module also introduces a sparse mask of channel or spatial dimension for the linear transformation within the expert, so that each expert only performs operations on a subset of relevant features, forming a dual sparse structure of Top-k routing sparseness and expert internal sparseness.

9. The method for enhancing low-light images of tugboats based on sparse expert networks according to claim 1, characterized in that: In step S5, let the multi-scale decoder be... Layer features are The corresponding encoder layer skip connection feature is Then the first The layer decoding process is represented as follows: ； in, For upsampling operators, This indicates a feature cascading or addition / fusion operation. This represents a reconstruction unit consisting of convolutional layers and nonlinear activations; An enhanced tugboat driving perspective image is obtained by layer-by-layer upsampling and skip connections.

10. The method for enhancing low-light images of tugboats based on sparse expert networks according to claim 1, characterized in that: Before performing image augmentation, the overall augmentation network, consisting of a sparse attention encoder, a sparse expert hybrid module, and a multi-scale decoder, is trained using a joint loss function, which is expressed as: ； in, Indicates the image reconstruction related loss. This indicates the expert equilibrium loss. This indicates a loss of expert diversity. and These are the weighting coefficients for expert equilibrium loss and expert diversity loss, respectively. The image reconstruction loss is represented as: ； in, Indicates pixel-level reconstruction loss. Indicates perceived loss. This represents the total variational loss. , , These are the corresponding weighting coefficients.

Images