A small target image enhancement and recognition method and system for a weak light environment

Abstract

The application discloses a small target image enhancement and recognition method and system for a weak light environment, and belongs to the field of computer vision, and comprises the following steps: brightness-texture feature decoupling is performed on a weak light environment original image to generate a brightness component graph and a texture detail graph, and then an enhanced brightness graph is generated, and then a preliminary enhanced image is obtained through adaptive fusion; the preliminary enhanced image is input into a small target saliency prediction network and a basic target detection network respectively to generate a small target visual saliency heat map and a preliminary detection result; the small target visual saliency heat map and the preliminary confidence are fused to generate a region attention weight graph; the region attention weight graph is used to perform weighted modulation on a feature map in the basic target detection network and target detection, and finally a target recognition result is output. The application can solve the problem that in the prior art, when the brightness of a weak light image is enhanced, the key detail features of small targets cannot be protected, restored and enhanced, and noise amplification cannot be effectively inhibited.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a method and system for enhancing and recognizing small target images in low-light environments. Background Technology

[0002] In the fields of smart security, industrial inspection, and especially energy safety production such as smart power plants, vision-based intelligent monitoring systems have become an indispensable part. These systems need to accurately identify targets such as safety helmets, insulating gloves, instrument pointers, and tiny equipment leaks (running, dripping, etc.). Currently, traditional image enhancement methods, such as histogram equalization and gamma correction, are mostly global or empirical processing. While improving the overall brightness of low-light images, they often cause the original weak edge and texture features of small targets to be blurred, distorted, or even submerged, and simultaneously amplify background noise, which reduces the signal-to-noise ratio of small targets and negatively impacts subsequent feature extraction and target recognition. In addition, conventional deep learning models, including some small models and general large models, are difficult to learn robust and discriminative feature representations from low-quality raw data when directly processing low-light images, especially for small targets with very little pixel information, which easily leads to missed detections or false detections. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for enhancing and recognizing small targets in low-light environments. This method addresses the problem that existing technologies struggle to protect, restore, and enhance the key details of small targets while effectively suppressing noise amplification when improving the brightness of low-light images. It can provide the target detection model with optimized and enhanced inputs that are more conducive to learning the features of small targets.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, a method for enhancing and recognizing small targets in low-light environments includes the following steps: Acquire the original image of the low-light environment to be processed; The original image of the low-light environment to be processed is subjected to brightness-texture feature decoupling to generate a brightness component map and a texture detail map; Adaptive brightness enhancement is performed based on the brightness component map to generate an enhanced brightness map. The enhanced brightness map and texture detail map are then adaptively fused to reconstruct a preliminary enhanced image. The pre-enhanced image is input into a pre-trained small target saliency prediction network to generate a small target visual saliency heatmap; The pre-enhanced image is input into a pre-trained base object detection network for preliminary inference to obtain preliminary detection results, which include: each candidate box and its preliminary confidence score. By fusing the visual saliency heatmap of small targets and the preliminary confidence level, a region attention weight map is generated. Using the region attention weight map, the feature map in the basic target detection network is weighted and modulated for target detection, and the final target recognition result is output.

[0005] In some implementations, the original image of the low-light environment to be processed is subjected to luminance-texture feature decoupling to generate a luminance component map and a texture detail map, including: The original image of the low-light environment to be processed is input into a pre-trained convolutional neural network for brightness-texture feature decoupling. The pre-trained convolutional neural network includes: An initial convolutional layer is used to extract initial features from the original image of the low-light environment to be processed, and obtain a shared feature map. The first branch includes a dilated convolutional layer and a global average pooling layer connected in sequence, used to extract a brightness feature vector from the shared feature map; The second branch includes a multi-scale convolutional module and a gradient-enhanced convolutional layer connected in sequence, used to extract texture feature vectors from the shared feature map; the multi-scale convolutional module is composed of several convolutional layers of different sizes connected in parallel, and the convolutional kernel of the gradient-enhanced convolutional layer includes the Sobel horizontal edge detection operator and the Sobel vertical edge detection operator. A normalization layer is used to normalize the brightness feature vector and the texture feature vector; The luminance decoding branch is used to reconstruct the normalized luminance feature vector to obtain the luminance component map; The texture decoding branch is used to reconstruct the normalized texture feature vector to obtain the texture detail map; The pre-trained convolutional neural network is obtained through pre-training, specifically by: minimizing the cosine similarity between the brightness feature vector and the texture feature vector, applying orthogonality constraints to the brightness feature vector and the texture feature vector, and then training to obtain the pre-trained convolutional neural network.

[0006] In some implementations, adaptive luminance enhancement is performed based on the luminance component map to generate an enhanced luminance map, including: The brightness component map is segmented into several homogeneous regions by superpixel segmentation. Calculate the mean and standard deviation of brightness for each homogeneous region; Based on a preset first brightness threshold, a second brightness threshold, and a texture threshold, and combined with the mean brightness and standard deviation, the homogeneous region is divided into extremely dark regions, moderately dark regions, and textured dark regions; wherein, the homogeneous region with a mean brightness less than the first brightness threshold is an extremely dark region; the homogeneous region with a mean brightness greater than or equal to the first brightness threshold and less than the second brightness threshold is a moderately dark region; and the homogeneous region with a mean brightness greater than or equal to the second brightness threshold and a standard deviation greater than the texture threshold is a textured dark region. The extremely dark areas are mapped using an S-curve, the medium dark areas are mapped using gamma correction, and the textured dark areas are mapped using linear stretching, resulting in the mapped results for each homogeneous region. The results of mapping the homogeneous regions are fused and smoothed to generate an enhanced brightness map.

[0007] In some implementations, the enhanced luminance map and texture detail map are adaptively fused to reconstruct a preliminary enhanced image, including: The enhanced brightness map and texture detail map are input into a pixel-level fusion network, and the preliminary enhanced image is obtained by reconstruction using the following formula:

[0008] in, These are the coordinates of a pixel in the image. coordinate position The corresponding fusion weights, The enhanced brightness map The pixel brightness value at the coordinate position. For texture detail images The pixel texture value at the coordinate position. For the generated preliminary enhanced image The final pixel value at the coordinate position.

[0009] The pixel-level fusion network includes an adaptive weight generation sub-network, which is used to generate fusion weights for each pixel position in the enhanced luminance map and texture detail map. In some implementations, the small target saliency prediction network is constructed based on a lightweight backbone network. The small target saliency prediction network includes a first feature layer, a second feature layer, and a third feature layer, with a first prediction head, a second prediction head, and a third prediction head respectively derived from the first feature layer, the second feature layer, and the third feature layer. The initially enhanced image is input into a pre-trained small target saliency prediction network to generate a small target visual saliency heatmap, including: The preliminary enhanced image is input into a pre-trained small target saliency prediction network, and different saliency heatmaps are obtained by passing through the first prediction head, the second prediction head, and the third prediction head, respectively. The different saliency heatmaps are weighted and summed according to their corresponding preset weight coefficients to obtain a visual saliency heatmap of small targets.

[0010] In some implementations, a region attention weight map is generated by fusing a small target visual saliency heatmap and preliminary confidence scores, including: Based on each candidate box and its initial confidence level, a confidence density map is generated on the image plane; The region attention weight map is calculated based on the confidence density map and the small target visual saliency heatmap, specifically using the following formula:

[0011] in, This is a region attention weight map. , and For different linear combination coefficients, To multiply pixel by pixel, Heatmap of visual saliency for small targets. This is a confidence density plot.

[0012] In some implementations, the region attention weight map is used to perform weighted modulation and target detection on the feature maps in the basic target detection network, outputting the final target recognition result, including: The region attention weight map is upsampled to the same size as the target feature layer output by the basic target detection network to obtain the upsampled weight map. Based on the upsampling weight map, the target feature layer is spatially modulated by multiplying pixel by pixel to obtain the spatially modulated features; Calculate the global average value of the regional attention weight map, and generate channel attention vectors through a fully connected network; Based on the channel attention vector, the spatially modulated features are subjected to channel modulation by channel-level multiplication to obtain the channel-modulated features. The channel-modulated features are input into the detection head of the basic target detection network to obtain the final target recognition result.

[0013] Secondly, a small target image enhancement and recognition system for low-light environments includes: The image acquisition module is used to acquire the original image of the low-light environment to be processed; A brightness-texture feature decoupling module is used to decouple the brightness-texture features of the original image of the low-light environment to be processed, and generate a brightness component map and a texture detail map. The adaptive enhancement and image reconstruction module is used to perform adaptive brightness enhancement based on the brightness component map, generate an enhanced brightness map, and adaptively fuse the enhanced brightness map and texture detail map to reconstruct a preliminary enhanced image. The small target visual saliency prediction module is used to input the preliminary enhanced image into a pre-trained small target saliency prediction network to generate a small target visual saliency heatmap. The basic object detection module is used to input the preliminary enhanced image into a pre-trained basic object detection network, perform preliminary inference, and obtain preliminary detection results, which include: each candidate box and its preliminary confidence score; The attention weight map generation module is used to fuse the visual saliency heatmap of small targets and the preliminary confidence level to generate a region attention weight map; The attention-guided fine-grained detection module is used to perform weighted modulation and target detection on the feature map in the basic target detection network using the region attention weight map, and output the final target recognition result.

[0014] Thirdly, a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method for enhancing and recognizing small target images in low-light environments.

[0015] Fourthly, a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method for enhancing and recognizing small target images in low-light environments.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a method for small target image enhancement and recognition in low-light environments. First, the original low-light image undergoes brightness-texture feature decoupling, separating the coupled illumination and detail information to lay the foundation for subsequent differential processing. Then, adaptive brightness enhancement is applied to the separated brightness component map, and adaptive fusion is performed with the texture detail map to reconstruct a preliminary enhanced image. This accurately improves the brightness of dark areas while preserving and restoring key details such as edges and textures that are easily obscured in low light, effectively avoiding the detail blurring and noise amplification problems caused by traditional global enhancement methods. Through a small target saliency prediction network and a basic target detection network, the enhanced image is interpreted from two dimensions: low-level visual attractiveness and high-level semantic confidence. The generated saliency heatmap is fused with the preliminary detection confidence to generate a region attention weight map. Finally, this weight map is used to weight and modulate the feature map of the basic detection network, guiding the network's computational resources to focus on the region most likely to be a small target. This provides the detection model with optimized and enhanced high-quality input with attention guidance, significantly improving the recognition accuracy and robustness of small targets in complex low-light environments.

[0017] Furthermore, a convolutional neural network with specific branch structures is employed to decouple luminance and texture features. The first branch utilizes dilated convolutional layers with an expansion rate of 3 and global average pooling layers to effectively expand the receptive field to capture the global illumination distribution, thereby extracting a pure luminance feature vector. The second branch captures multi-scale structural information through multi-scale convolutional modules composed of convolutional layers of different sizes connected in parallel, and uses gradient-enhanced convolutional layers with Sobel operators to strengthen edge responses, thereby extracting texture feature vectors rich in detail. During pre-training, orthogonality constraints are applied by minimizing the cosine similarity between the luminance feature vector and the texture feature vector, forcing the two feature spaces to be independent of each other. This ensures that the luminance component map and the texture detail map reconstructed by the texture decoding branch are completely semantically separated, avoiding information aliasing and providing higher-quality and purer input for subsequent independent and targeted enhancement and fusion processing.

[0018] Furthermore, during adaptive brightness enhancement, the brightness component map is first divided into homogeneous regions with similar attributes through superpixel segmentation, and the mean and standard deviation of brightness in each region are calculated. Then, based on preset thresholds, these regions are intelligently divided into extremely dark areas, medium dark areas, and textured dark areas, achieving fine perception of image content. For different types of regions, the method applies S-curves, gamma correction, and linear stretching to perform differentiated brightness mapping. Extremely dark areas are significantly brightened, medium dark areas are dynamically adjusted, and textured dark areas containing rich details are linearly stretched to protect their original texture contrast, avoiding a one-size-fits-all enhancement. This method can accurately protect and enhance the detail information in different regions while improving the overall image brightness. Finally, the enhanced brightness map is generated through fusion and smoothing, providing a key guarantee for reconstructing a high-quality, high signal-to-noise ratio preliminarily enhanced image.

[0019] Furthermore, when reconstructing the enhanced brightness map and texture detail map, a pixel-level fusion network is used. An adaptive weight generation subnetwork dynamically generates fusion weights for each pixel position. This allows for greater reliance on the brightness-enhanced map in flat areas to obtain good lighting effects. However, in edge and texture-rich areas, especially in potential areas where small targets are located, training constraints bias the fusion weights towards the texture detail map. As a result, in the final generated preliminary enhanced image, the faint edge and structural features of small targets are preferentially preserved and highlighted, achieving an optimal trade-off between brightness enhancement and detail preservation. This provides higher-quality image data for subsequent recognition tasks from the source.

[0020] Furthermore, the small target saliency prediction network, by drawing prediction heads from different depth feature layers of the lightweight backbone network, fully utilizes multi-scale features and can effectively capture small targets with varying sizes. When the saliency heatmaps output by different prediction heads are weighted and summed through the scale-aware fusion module, their corresponding preset weight coefficients are dynamically adjusted based on the global brightness statistics of the original image or the pre-enhanced image. This allows the network to adaptively select the scale features most favorable to the current scene for fusion according to the lighting conditions, thereby generating a small target visual saliency heatmap with stronger response capability and positioning accuracy for low-contrast and blurred small targets in low-light environments.

[0021] Furthermore, when generating the region attention weight map, two heterogeneous information sources, namely the small target visual saliency heatmap and the confidence density map, are fused and nonlinearly fused. This can enhance regions where both respond strongly at the same time and suppress noise responses from a single information source. The resulting region attention weight map can more accurately locate regions that are very likely to be real small targets, providing high-quality attention guidance for subsequent feature modulation.

[0022] Furthermore, by first upsampling the regional attention weight map and multiplying it pixel-by-pixel with the target feature layer output by the backbone network, spatial modulation of the feature map is achieved, directly enhancing the feature response of the potential target region and suppressing background noise. Simultaneously, by calculating the global average value of the regional attention weight map and generating channel attention vectors through a fully connected network, and then performing channel-level multiplication on the spatially modulated features, adaptive recalibration of feature channels is achieved, emphasizing the feature channels most relevant to the current task. This modulation method ensures that the features finally input to the detection head are focused on the correct spatial location and have the most favorable feature representation for classification and regression, thereby significantly improving the final recognition accuracy of small targets in complex low-light backgrounds. Attached Figure Description

[0023] Figure 1 A flowchart illustrating a method for enhancing and recognizing small targets in low-light environments, provided by an embodiment of the present invention; Figure 2 This is a structural diagram of a small target image enhancement and recognition system for low-light environments, provided by an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings. The content described herein is for explanation rather than limitation of the present invention.

[0025] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification of this invention are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, systems, products or devices.

[0026] like Figure 1 As shown, this embodiment provides a method for enhancing and recognizing small targets in low-light environments, including the following steps: S1, Obtain the original image of the low-light environment to be processed; Specifically, raw images of low-light areas within a thermal power plant are captured using industrial cameras, such as nighttime footage of coal conveying corridors or monitoring footage of equipment interiors. S2, decouple the brightness-texture features of the original image of the low-light environment to be processed to generate a brightness component map and a texture detail map; Specifically, the original image is input into a dual encoder network: the first encoder (luminance branch) extracts the global illumination distribution through dilated convolution, and the second encoder (texture branch) extracts edge details through multi-scale convolution. The network outputs two feature maps: a luminance component map and a texture detail map, where the luminance component map reflects the illumination intensity and the texture detail map preserves the edges of small objects. A convolutional neural network with a dual encoder and a single encoder is used to decouple the brightness and texture features of the original image. A shared initial convolutional layer is used to extract initial features from the original image, resulting in a shared feature map. The shared feature map is then input into a first branch and a second branch for parallel processing. The first branch includes a dilated convolutional layer with a dilation rate of 3 and a global average pooling layer to extract global brightness features from the shared feature map. The second branch includes a multi-scale convolutional module and a gradient-enhanced convolutional layer. The multi-scale convolutional module consists of parallel 1x1 and 3x3 convolutional layers, and the gradient-enhanced convolutional layer uses fixed-parameter Sobel horizontal edge detection operators and Sobel vertical edge detection operators as convolutional kernels. The luminance feature vector output from the first branch and the texture feature vector output from the second branch are normalized respectively. When training the convolutional neural network, orthogonality constraints are applied by minimizing the cosine similarity between the luminance feature vector and the texture feature vector. The luminance feature vector and the texture feature vector after normalization and orthogonality constraints are respectively input into the luminance decoding branch and the texture decoding branch corresponding to the decoder. The luminance component map is reconstructed through the luminance decoding branch, and the texture detail map is reconstructed through the texture decoding branch.

[0027] Construct a convolutional neural network consisting of a shared encoder, a dual-branch encoder, and a decoder. The shared encoder uses three standard convolutional layers: the first branch is the luminance branch, containing one dilated convolutional layer with a dilation rate of 3 and one global average pooling layer; the second branch is the texture branch, containing one multi-scale convolutional module and one gradient-enhanced convolutional layer, where the multi-scale convolutional module includes parallel 1×1 and 3×3 convolutional layers; the decoder contains two parallel decoding branches: a luminance decoding branch and a texture decoding branch. The above convolutional neural network is trained end-to-end with a batch of low-light original images as input. The loss function consists of three parts: First, reconstruction loss: ensuring that the sum of the brightness map and texture map output by the decoder can reconstruct the original image; Second, orthogonality loss: calculating the cosine similarity of the feature vectors output by the two branches and minimizing this value to make the brightness and texture features independent of each other; Third, edge preservation loss: constraining the features extracted by the texture branch to accurately reflect the edge information of the image. In practical applications, a single low-light image is input. First, a shared encoder extracts basic features, which are then input into two branches. The brightness branch expands the receptive field through dilated convolutions to capture global illumination, and then pools to obtain a compact brightness feature vector. The texture branch captures detailed structures through multi-scale convolutions, and then enhances edge responses through convolutional layers with fixed Sobel operators to obtain a texture feature vector. Batch normalization is applied to both feature vectors. During training, orthogonality loss is used to ensure that the two feature vectors are orthogonal in the feature space. The processed brightness feature vector is input into the brightness decoding branch, where upsampling and convolution operations reconstruct a brightness component map of the same size as the input. The texture feature vector is input into the texture decoding branch, where the same operations reconstruct a texture detail map. The output consists of two feature maps: a brightness component map and a texture detail map.

[0028] S3, perform adaptive brightness enhancement based on the brightness component map to generate an enhanced brightness map, and adaptively fuse the enhanced brightness map and texture detail map to reconstruct a preliminary enhanced image; Specifically, intelligent enhancement is performed on the luminance component map: the image is divided into multiple superpixel regions, and enhancement strategies are automatically selected according to the luminance values ​​of each region. Extremely dark areas are significantly brightened using an S-curve, medium dark areas are brightened using dynamic gamma correction, and high-texture dark areas are brightened using linear stretching to preserve details. The enhanced luminance map is then fused with the original texture detail map at the pixel level to generate a preliminary enhanced image with improved visual quality. The brightness component map is segmented into several homogeneous regions by superpixel segmentation; the mean brightness and standard deviation of each homogeneous region are calculated; and the region is divided into extremely dark areas, medium dark areas and textured dark areas according to a preset first brightness threshold, a second brightness threshold and a texture threshold. The regions with a mean brightness value less than a first brightness threshold are defined as extremely dark regions; regions with a first brightness threshold ≤ mean brightness value < a second brightness threshold are defined as moderately dark regions; regions with a mean brightness value ≥ the second brightness threshold and a standard deviation > a texture threshold are defined as textured dark regions. An S-curve is applied to the extremely dark regions for brightness mapping; gamma correction is applied to the moderately dark regions for brightness mapping; linear stretching is applied to the textured dark regions for brightness mapping; and the mapping results for each region are then fused and smoothed to generate the enhanced brightness map.

[0029] Specifically, the SLIC superpixel segmentation algorithm is used on the luminance component map to generate approximately 200-300 homogeneous regions. Two key metrics for each superpixel region are calculated: mean luminance. (Average brightness of all pixels within the area) and standard brightness value (Dispersion of pixel brightness values ​​within a region); set three preset thresholds, where the first brightness threshold is usually set to 30, ranging from 0 to 255; the second brightness threshold is usually set to 120, ranging from 0 to 255; and the texture threshold is usually set to 20; based on each region The regions are categorized as follows: for extremely dark areas, if the mean brightness is less than the first brightness threshold, they are classified as category 1; for moderately dark areas, if the first brightness threshold is less than the mean brightness threshold and the second brightness threshold, they are classified as category 2; for textured dark areas, if the mean brightness is greater than or equal to the second brightness threshold and the standard deviation is greater than the texture threshold, they are classified as category 3. Other areas remain unchanged or are lightly processed. Different mapping functions are applied to the three categories of regions: S-curve mapping is used. ,in, This represents the original brightness value of the currently processed pixel in the extremely dark area. This refers to the enhanced brightness value of the pixels in the extremely dark area after being mapped using an S-curve. This represents the maximum pixel brightness. It is a sigmoid activation function. The slope control parameter for the S-curve. The center offset parameter of the S-curve significantly enhances midtone brightness and effectively improves visibility in dark areas; gamma correction is applied to the medium dark areas for brightness mapping; linear stretching is applied to the textured dark areas. Or keep the original value: ; All enhanced regions are stitched together in their original positions to form a complete image. Bilinear interpolation is then performed at the region boundaries to smooth out blockiness. If necessary, a light Gaussian filter (σ=0.5) can be applied to the entire image to reduce noise. The final output is the processed enhanced brightness image.

[0030] The process of applying gamma correction to the moderately dark area for luminance mapping includes the following steps: The gamma correction is specifically as follows:

[0031] in, Gamma value, This represents the average brightness of the current area. This represents the global average brightness value of the brightness component map. A scaling factor to control the range of gamma values. To control the decay rate, the decay coefficient, This is the reference value for gamma correction.

[0032] Specifically, In this embodiment, A scaling factor used to control the range of gamma values. The range of change is 0.5; The attenuation coefficient, used to control the attenuation rate, is used to control... Follow The rate at which the speed increases and decreases is set to 1.2; This is the reference value for gamma correction, and the value is 0.8.

[0033] The enhanced luminance map is adaptively fused with the texture detail map, including the following steps: The enhanced luminance map and the texture detail map are input into a pixel-level fusion network, which includes an adaptive weight generation subnetwork for each pixel location. Generate fusion weights ; During training, the adaptive weight generation subnetwork constrains the fusion weights using an additional loss function for known small target labeled regions. The value in the region is less than a preset threshold; Through the formula: The preliminary enhanced image is calculated. ,in, These are the coordinates of a pixel in the image. The fusion weights corresponding to the coordinate positions are: The enhanced brightness map The pixel brightness value at the coordinate position. For texture detail images The pixel texture value at the stated coordinates. For the generated preliminary enhanced image The final pixel value at the stated coordinates.

[0034] Specifically, a lightweight convolutional neural network is constructed as an adaptive weight generation sub-network, which includes: network input: the enhanced brightness map (single channel) and texture detail map (single channel) are concatenated along the channel dimension to form a 2-channel input; the network structure contains 4 convolutional layers, each followed by ReLU activation, and finally outputs a single-channel weight map through the Sigmoid activation function; the network output is a fused weight map with the same size as the input. Each pixel value The main loss function is the reconstruction loss, which ensures the similarity between the fused image and a high-quality reference image (if available). The additional constraint loss is a weighted constraint loss applied to known small target labeled regions (determined by the bounding boxes). ,in, The threshold is set to 0.3; this constraint forces the network to allocate lower values ​​in small target regions. A value < 0.3 indicates a greater reliance on texture detail maps during fusion. The enhanced brightness map and texture detail map are input into a trained weighted generation network. The network automatically analyzes the characteristics of each local region; for flat, low-texture regions, the network tends to output higher values. Value (close to 1); for regions with rich edges and high texture: the network tends to output lower values. Value (close to 0); particularly, in regions where small targets may exist, subject to training constraints, The value is suppressed to a low level; for each pixel location in the image. calculate ; Then, the blending effect is analyzed, specifically in the flat background area: It primarily presents enhanced smooth brightness; in small target areas: It primarily preserves the original texture details; in general texture areas: This achieves a balanced fusion of brightness and texture. After performing the above calculations on all pixels, a complete preliminary enhanced image is obtained, which simultaneously possesses: improved overall brightness combined with protected detail texture. In particular, the edge and structural features of small target areas are preserved to the greatest extent.

[0035] S4, the preliminary enhanced image is input into a pre-trained small target saliency prediction network to generate a small target visual saliency heatmap; Specifically, the saliency network pre-trained for image input will be initially enhanced: the network makes predictions on three feature layers at different scales, and the multi-scale prediction results are combined to generate a heatmap, where the highlighted areas in the heatmap represent possible locations of small targets; The small target saliency prediction network is constructed based on a lightweight backbone network; A first prediction head, a second prediction head, and a third prediction head are respectively drawn out from the first feature layer, the second feature layer, and the third feature layer of the backbone network; The first prediction head fuses features from the first feature layer and the second feature layer; the second prediction head fuses features from the second feature layer and the third feature layer; the third prediction head fuses features from the first feature layer and the third feature layer. The saliency heatmaps output by the first prediction head, the second prediction head, and the third prediction head are weighted and summed by the scale-aware fusion module to generate the visual saliency heatmap of the small target. The weight coefficients corresponding to the first, second, and third prediction heads in the scale-aware fusion module are dynamically adjusted based on the global brightness statistics of the original image or the preliminarily enhanced image.

[0036] Specifically, the first prediction head is constructed as follows: the input consists of features from the first and second feature layers; during the process, the second feature layer is upsampled to the same size as the first feature layer and then concatenated along the channel dimension; the specific processing is carried out through three convolutional layers, and the output is a single-channel saliency heatmap H1. Second prediction head construction: The input is the features of the second feature layer and the third feature layer; during the process, the third feature layer is upsampled to the same size as the second feature layer and concatenated in the channel dimension; the specific processing includes processing through 3 convolutional layers, and outputting a single-channel saliency heatmap H2; Third prediction head construction: The input is the features of the first feature layer and the third feature layer; during the process, the third feature layer is upsampled to the same size as the first feature layer and concatenated in the channel dimension; the specific processing includes processing through 3 convolutional layers, and outputting a single-channel saliency heatmap H3; The outputs of the three prediction heads are uniformly upsampled to the original image size; the weights of the three prediction heads are calculated based on the global brightness mean of the input image, specifically as follows: The weight is higher when corresponding to dark images; It has a higher weight when the brightness is medium; The corresponding bright image has a higher weight; among them, The average brightness of all pixels in the input image is used. After normalizing the weights, they are weighted and fused with the saliency heatmaps output by the three prediction heads. A low-light image dataset containing small target annotations is used. The small target annotation boxes are converted into Gaussian heatmaps as training targets. Weighted mean square error loss is used to give higher weights to small target regions. The backbone network is frozen first, and the prediction heads are trained. Then, the entire network is fine-tuned. Then, saliency prediction forward inference is performed, specifically: inputting the initially enhanced image; the backbone network extracts three feature layers through MobileNetV3; multi-scale prediction is processed by three prediction heads and outputs heatmaps; dynamic fusion automatically adjusts the weights of each prediction head according to the overall brightness of the image; outputting a visual saliency heatmap of small targets; performing saliency heatmap analysis, specifically: high-value regions (close to 1) are very likely to be the location of small targets; medium-value regions (0.3-0.7) may be background textures or suspected targets; low-value regions (close to 0) are flat backgrounds or areas without targets; in particular, the heatmap has a strong response capability to small, low-contrast targets.

[0037] S5, the pre-enhanced image is input into the pre-trained basic object detection network for preliminary inference to obtain preliminary detection results, which include: each candidate box and its preliminary confidence score; Specifically, the initial enhanced image is input into the basic detection network (e.g., YOLOv5) to obtain the initial detection boxes and corresponding confidence scores; S6, integrates the visual saliency heatmap of small targets and the preliminary confidence level to generate a region attention weight map; Specifically, the saliency heatmap and confidence map are fused: the two are linearly combined using learnable parameters, and a non-linear interaction term is added to improve the fusion effect, outputting a region attention weight map (the higher the value, the more likely the location is to be an important small target). Based on the position and size of each candidate box in the preliminary detection results, and the corresponding preliminary confidence level, a confidence density map is generated on the image plane, where the size is... The variance of the Gaussian diffusion kernel used in the candidate boxes and Proportional; Calculate the region attention weight map Specifically:

[0038] in, , and The coefficients are linear combination coefficients. To multiply pixel by pixel, Heatmap of visual saliency for small targets. This is a confidence density plot.

[0039] Specifically, a zero-matrix with the same size as the input image is created, i.e., a confidence density map. For each candidate box in the preliminary detection results, obtain its position, size, and confidence score; construct a two-dimensional Gaussian kernel for the candidate box, specifically as follows: ,in, The standard deviation of the Gaussian kernel. The proportionality constant is usually taken as 10-20 (e.g., k=15); the coordinates of the center point are used as the reference. Centered on, using standard deviation Constructing a Gaussian kernel, the Gaussian kernel size is usually taken as... Multiply the Gaussian kernel by the confidence score of the candidate box to generate a Gaussian response map; then superimpose the Gaussian response map onto... At the corresponding positions, the values ​​of the overlapping areas of multiple candidate boxes are accumulated; normalization is performed; and three initialization methods are initialized. , and Let be the coefficients of the linear combination, where The base weight for the significance heatmap is 0.6. The base weight for the confidence density plot is 0.3. These are the interaction term weights, with a value of 0.4; these parameters will be optimized during training using gradient descent. The training data uses image data with small target annotations; an ideal attention map is generated based on the ground truth bounding boxes, assigning high weights to areas near the target location and low weights to the background region to generate the ground truth attention map; the loss function... ,in, Let the mean squared error loss function be . This is a truth-based attention map, which is a target weight map generated based on the annotations of real small targets. The regularization coefficient is typically between 0.01 and 0.1. The first term in the loss function is the mean squared error loss, which makes the predicted weight map close to the true value. The second term is the regularization term, which encourages the sum of the three coefficients to be close to 1, avoiding weight explosion. Other network parameters are fixed, and only training is performed. , and Using the Adam optimizer with a learning rate of 0.01; the generated It has the following characteristics: high-weight region ( >0.7), and At the same time, areas with strong responses are most likely to be the locations of real small targets; areas with medium weights (0.3≤ ≤0.7), with only one feature ( or A strong response may indicate a suspected target or background texture; low-weight regions ( <0.3), both feature responses are weak, which can basically be identified as background.

[0040] S7. Using the region attention weight map, the feature map in the basic target detection network is weighted and modulated for target detection, and the final target recognition result is output.

[0041] Specifically, the attention weight map is applied to the detection network: spatial and channel dual modulation is performed on the intermediate feature map of the network to enhance the feature response of small target areas, suppress the features of background noise areas, and finally output high-precision target recognition results.

[0042] The region attention weight map Upsampled to the same size as the target feature layer output by the backbone of the basic target detection network, resulting in an upsampled weight map. ; The target feature layer is processed by pixel-by-pixel multiplication. Spatial modulation is performed, specifically as follows: ,in, Features after spatial modulation To upsample the weighted graph Broadcast to target feature layer Multiply all channels together; Calculate the global average value of the region attention weight map, and generate channel attention vectors through a fully connected network. ; The spatially modulated features are obtained through channel-level multiplication. Channel modulation is performed, specifically as follows: ,in, The characteristics after channel modulation To convert the channel attention vector Broadcast to Each spatial location, and with The scalars of the corresponding channels are multiplied element by element; The channel-modulated features are input into the detection head of the basic target detection network to obtain the final target recognition result.

[0043] Specifically, the modulated features The input is fed into subsequent layers of the object detection network for detection head processing: the classification branch outputs the class probability of each anchor box; the regression branch outputs the bounding box coordinate adjustment value; the confidence branch outputs the probability of the object's presence; further post-processing is performed, including: non-maximum suppression (NMS) to remove duplicate detections, confidence threshold filtering, and outputting the final detection box and class label; the training strategy is phased training: Phase 1: freeze the attention module and train the basic detection network; Phase 2: unfreeze the attention module and fine-tune the network; the loss function combination is: , The classification loss is cross-entropy. For bounding box regression loss, As an auxiliary loss for the attention module, is the regularization coefficient.

[0044] This embodiment designs a dual-encoder network to decompose an image into a luminance component representing illumination distribution and a texture component containing details. For the luminance component, an adaptive enhancement strategy based on superpixel region classification is adopted, applying differentiated mapping curves to regions with different luminance and texture richness. In the fusion stage, a learnable weight generation network is used to assign higher weights to the texture component in potential small target regions, ensuring that its detailed features are preferentially preserved in the final enhanced image. This achieves intelligent brightening of dark areas while protecting and enhancing texture details, providing higher quality and higher signal-to-noise ratio input images for small target recognition from the source. In addition, by constructing an attention generation path jointly driven by visual saliency (low-level features) and detection confidence (high-level semantics), the heatmap output by the small target saliency prediction network is nonlinearly fused with the confidence density map of the preliminary results of the basic detection network to generate an accurate region attention weight map. This concentrates computational resources on the most suspicious regions, effectively reducing the false negative and false positive rates in complex low-light backgrounds, especially improving the recall rate of small and blurry targets.

[0045] like Figure 2 As shown, this embodiment provides a small target image enhancement and recognition system for low-light environments, including: The image acquisition module is used to acquire the original image of the low-light environment to be processed; A brightness-texture feature decoupling module is used to decouple the brightness-texture features of the original image of the low-light environment to be processed, and generate a brightness component map and a texture detail map. The adaptive enhancement and image reconstruction module is used to perform adaptive brightness enhancement based on the brightness component map, generate an enhanced brightness map, and adaptively fuse the enhanced brightness map and texture detail map to reconstruct a preliminary enhanced image. The small target visual saliency prediction module is used to input the preliminary enhanced image into a pre-trained small target saliency prediction network to generate a small target visual saliency heatmap. The basic object detection module is used to input the preliminary enhanced image into a pre-trained basic object detection network, perform preliminary inference, and obtain preliminary detection results, which include: each candidate box and its preliminary confidence score; The attention weight map generation module is used to fuse the visual saliency heatmap of small targets and the preliminary confidence level to generate a region attention weight map; The attention-guided fine-grained detection module is used to perform weighted modulation and target detection on the feature map in the basic target detection network using the region attention weight map, and output the final target recognition result.

[0046] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0047] This embodiment also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the small target image enhancement and recognition method for low-light environments described in the above embodiment.

[0048] This embodiment also provides a computer program product, which includes a computer program that, when executed by a processor, implements the corresponding steps of the small target image enhancement and recognition method for low-light environments described in the above embodiment.

[0049] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0050] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0051] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0052] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for image enhancement and recognition of small targets in low-light environments, characterized in that, Includes the following steps: Acquire the original image of the low-light environment to be processed; The original image of the low-light environment to be processed is subjected to brightness-texture feature decoupling to generate a brightness component map and a texture detail map; Adaptive brightness enhancement is performed based on the brightness component map to generate an enhanced brightness map. The enhanced brightness map and texture detail map are then adaptively fused to reconstruct a preliminary enhanced image. The pre-enhanced image is input into a pre-trained small target saliency prediction network to generate a small target visual saliency heatmap; The pre-enhanced image is input into a pre-trained base object detection network for preliminary inference to obtain preliminary detection results, which include: each candidate box and its preliminary confidence score. By fusing the visual saliency heatmap of small targets and the preliminary confidence level, a region attention weight map is generated. Using the region attention weight map, the feature map in the basic target detection network is weighted and modulated for target detection, and the final target recognition result is output.

2. The method for image enhancement and recognition of small targets in low-light environments according to claim 1, characterized in that, The original image of the low-light environment to be processed is subjected to luminance-texture feature decoupling to generate a luminance component map and a texture detail map, including: The original image of the low-light environment to be processed is input into a pre-trained convolutional neural network for brightness-texture feature decoupling. The pre-trained convolutional neural network includes: An initial convolutional layer is used to extract initial features from the original image of the low-light environment to be processed, and obtain a shared feature map. The first branch includes a dilated convolutional layer and a global average pooling layer connected in sequence, used to extract a brightness feature vector from the shared feature map; The second branch includes a multi-scale convolutional module and a gradient-enhanced convolutional layer connected in sequence, used to extract texture feature vectors from the shared feature map; the multi-scale convolutional module is composed of several convolutional layers of different sizes connected in parallel, and the convolutional kernel of the gradient-enhanced convolutional layer includes the Sobel horizontal edge detection operator and the Sobel vertical edge detection operator. A normalization layer is used to normalize the brightness feature vector and the texture feature vector; The luminance decoding branch is used to reconstruct the normalized luminance feature vector to obtain the luminance component map; The texture decoding branch is used to reconstruct the normalized texture feature vector to obtain the texture detail map; The pre-trained convolutional neural network is obtained through pre-training, specifically by: minimizing the cosine similarity between the brightness feature vector and the texture feature vector, applying orthogonality constraints to the brightness feature vector and the texture feature vector, and then training to obtain the pre-trained convolutional neural network.

3. The method for image enhancement and recognition of small targets in low-light environments according to claim 1, characterized in that, Adaptive brightness enhancement is performed based on the brightness component map to generate an enhanced brightness map, including: The brightness component map is segmented into several homogeneous regions by superpixel segmentation. Calculate the mean and standard deviation of brightness for each homogeneous region; Based on a preset first brightness threshold, a second brightness threshold, and a texture threshold, and combined with the mean brightness and standard deviation, the homogeneous region is divided into extremely dark regions, moderately dark regions, and textured dark regions; wherein, the homogeneous region with a mean brightness less than the first brightness threshold is an extremely dark region; the homogeneous region with a mean brightness greater than or equal to the first brightness threshold and less than the second brightness threshold is a moderately dark region; and the homogeneous region with a mean brightness greater than or equal to the second brightness threshold and a standard deviation greater than the texture threshold is a textured dark region. The extremely dark areas are mapped using an S-curve, the medium dark areas are mapped using gamma correction, and the textured dark areas are mapped using linear stretching, resulting in the mapped results for each homogeneous region. The results of mapping the homogeneous regions are fused and smoothed to generate an enhanced brightness map.

4. The method for image enhancement and recognition of small targets in low-light environments according to claim 1, characterized in that, The enhanced brightness map and texture detail map are adaptively fused to reconstruct a preliminary enhanced image, including: The enhanced brightness map and texture detail map are input into a pixel-level fusion network, and the preliminary enhanced image is obtained by reconstruction using the following formula: in, These are the coordinates of a pixel in the image. coordinate position The corresponding fusion weights, The enhanced brightness map The pixel brightness value at the coordinate position. For texture detail images The pixel texture value at the coordinate position. For the generated preliminary enhanced image The final pixel value at the coordinate position; The pixel-level fusion network includes an adaptive weight generation sub-network, which is used to generate fusion weights for each pixel position in the enhanced luminance map and texture detail map.

5. The method for image enhancement and recognition of small targets in low-light environments according to claim 1, characterized in that, The small target saliency prediction network is constructed based on a lightweight backbone network. The small target saliency prediction network includes a first feature layer, a second feature layer and a third feature layer, and a first prediction head, a second prediction head and a third prediction head are respectively derived from the first feature layer, the second feature layer and the third feature layer. The initially enhanced image is input into a pre-trained small target saliency prediction network to generate a small target visual saliency heatmap, including: The preliminary enhanced image is input into a pre-trained small target saliency prediction network, and different saliency heatmaps are obtained by passing through the first prediction head, the second prediction head, and the third prediction head, respectively. The different saliency heatmaps are weighted and summed according to their corresponding preset weight coefficients to obtain a visual saliency heatmap of small targets.

6. The method for image enhancement and recognition of small targets in low-light environments according to claim 1, characterized in that, By fusing the visual saliency heatmap of small targets and preliminary confidence scores, a region attention weight map is generated, including: Based on each candidate box and its initial confidence level, a confidence density map is generated on the image plane; The region attention weight map is calculated based on the confidence density map and the small target visual saliency heatmap, specifically using the following formula: in, This is a region attention weight map. , and For different linear combination coefficients, To multiply pixel by pixel, Heatmap of visual saliency for small targets. This is a confidence density plot.

7. The method for image enhancement and recognition of small targets in low-light environments according to claim 1, characterized in that, Using the region attention weight map, the feature maps in the basic object detection network are weighted and modulated for object detection, and the final object recognition result is output, including: The region attention weight map is upsampled to the same size as the target feature layer output by the basic target detection network to obtain the upsampled weight map. Based on the upsampling weight map, the target feature layer is spatially modulated by multiplying pixel by pixel to obtain the spatially modulated features; Calculate the global average value of the regional attention weight map, and generate channel attention vectors through a fully connected network; Based on the channel attention vector, the spatially modulated features are subjected to channel modulation through channel-level multiplication to obtain the channel-modulated features. The channel-modulated features are input into the detection head of the basic target detection network to obtain the final target recognition result.

8. A system for enhancing and recognizing small targets in low-light environments, characterized in that, include: The image acquisition module is used to acquire the original image of the low-light environment to be processed; A brightness-texture feature decoupling module is used to decouple the brightness-texture features of the original image of the low-light environment to be processed, and generate a brightness component map and a texture detail map. The adaptive enhancement and image reconstruction module is used to perform adaptive brightness enhancement based on the brightness component map, generate an enhanced brightness map, and adaptively fuse the enhanced brightness map and texture detail map to reconstruct a preliminary enhanced image. The small target visual saliency prediction module is used to input the preliminary enhanced image into a pre-trained small target saliency prediction network to generate a small target visual saliency heatmap. The basic object detection module is used to input the preliminary enhanced image into a pre-trained basic object detection network, perform preliminary inference, and obtain preliminary detection results, which include: each candidate box and its preliminary confidence score; The attention weight map generation module is used to fuse the visual saliency heatmap of small targets and the preliminary confidence level to generate a region attention weight map; The attention-guided fine-grained detection module is used to perform weighted modulation and target detection on the feature map in the basic target detection network using the region attention weight map, and output the final target recognition result.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for enhancing and recognizing small targets in low-light environments as described in any one of claims 1 to 6.

10. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for small target image enhancement and recognition in low-light environments as described in any one of claims 1 to 6.

Images