A low-light small target image recognition method based on deep learning

Abstract

The present invention provides a method for low-light small target image recognition based on deep learning, which belongs to the field of image recognition. The method includes: establishing a low-light small target data set, training a multi-scale enhancement model and a small target semantic segmentation model, enhancing the low-light image, feeding the enhanced image into the semantic segmentation model to achieve segmentation and recognition of the small target, and locating the target trajectory based on the segmentation map of the small target. The present invention improves the accuracy of network recognition for low-light small target images, effectively reducing the probability of false detection and missed detection; while improving the brightness and contrast of low-light images, it suppresses noise amplification and loss of details, retains detail information, and thus achieves accurate and stable detection and recognition of the enhanced image.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and specifically relates to a low-illumination small target image recognition method based on deep learning. Background Art

[0002] Image enhancement and recognition tasks are important areas in the field of image recognition. With the development of society, image enhancement and recognition tasks for different environments, such as low-light environments, are becoming increasingly important. Low-light small target recognition is a branch of target recognition tasks. Due to the complexity and diversity of the real environment in which images are acquired, images often have low contrast, color distortion, and small targets, resulting in loss of image details, blurred outlines, and difficulty in detection and recognition. Therefore, low-light image enhancement and recognition tasks remain very challenging. While improving the brightness and contrast of low-light images, it is necessary to suppress noise amplification and detail loss during the enhancement process, retain detailed information, and achieve accurate and stable detection and recognition of the enhanced images.

[0003] To date, researchers have proposed various networks that combine neural networks with object recognition. For example, in 2018, Zhou et al. proposed a novel encoder-decoder architecture, U-Net++. The encoder and decoder subnetworks are connected by a series of nested dense skip paths, reducing the semantic gap between the feature maps of the encoder and decoder subnetworks. However, in low-light scenarios with small objects, U-Net++ may lose some feature information during multiple downsampling and upsampling operations. This deficiency is particularly pronounced when processing objects that occupy a small proportion of the entire image, resulting in reduced segmentation performance. Wang et al. proposed the HCF-Net network to address the problems of small object detection loss and low background discrimination. This network improves feature information at different scales by introducing a parallelized patch-aware attention module, thereby improving the recognition accuracy of small object images. However, the overall network's generalization ability is insufficient. Effective recognition of small objects in low-light scenarios remains problematic due to image detail loss, increased noise, and reduced contrast between the object and the background. Therefore, it is crucial to develop a recognition algorithm suitable for low-light small object images to improve their segmentation accuracy. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention proposes a low-light small target recognition method based on deep learning, which enhances the perception ability of low-light small targets, improves the recognition ability and segmentation accuracy of small targets, and realizes the positioning of small targets.

[0005] The technical solutions of the present invention are as follows:

[0006] A low-light small target image recognition method based on deep learning includes the following steps:

[0007] S1: Collect low-illumination small target images and corresponding normal-illumination images, and annotate and preprocess the collected images to obtain a low-illumination small target dataset;

[0008] S2: Construct a multi-scale enhancement model, and use the low-illumination images and corresponding normal-illumination images in the low-illumination small target dataset to train the multi-scale enhancement model to obtain an MSEM model, thereby achieving illumination enhancement of the low-illumination small target image;

[0009] S3: Constructing a small object semantic segmentation model, using the low-light small object image enhanced by the MSEM model and the label map in the low-light small object dataset to train the small object semantic segmentation model, obtaining a DR-UNet model, and realizing segmentation and recognition of the enhanced image;

[0010] S4: Perform contour detection on the segmented image to extract the target edge features of the segmented image;

[0011] S5: Target center positioning is performed based on the target edge features to obtain the position of the small target.

[0012] According to a method for recognizing small targets in low illumination based on deep learning in some embodiments of the present application, in step S2, the multi-scale enhancement model includes a decomposition module and an enhancement module; the decomposition module includes three submodules: shallow feature extraction, activation layer sequence and final reconstruction layer; the shallow feature extraction submodule includes a convolution layer, the number of channels of the convolution layer is 64, the size of the convolution kernel is 3, and it is used to perform preliminary feature extraction on the input low illumination image and the corresponding normal illumination image; the activation sequence layer submodule performs deep feature extraction on the feature information extracted initially, which is performed through five convolution layers. The convolution blocks consisting of a convolution layer and a Leaky-ReLu activation function are connected in sequence, wherein the number of channels of the convolution layer is 64 and the size of the convolution kernel is 3; the final reconstruction layer submodule remaps the feature map extracted by the activation sequence layer submodule back to the original image space to generate a reconstructed image or feature map, and the final reconstruction layer submodule includes a convolution layer and a sigmoid function, wherein the convolution layer converts the feature map output by the activation sequence layer submodule from 64 channels back to 4 channels, and the sigmoid function maps the output to the range of 0-1, and the feature map is divided into a reflection component and an illumination component;

[0013] The enhancement module includes three submodules: feature extraction, multi-scale feature fusion and output generation; the feature extraction submodule includes a convolution layer and three deep feature extraction layers; the convolution layer has 64 channels and a convolution kernel size of 3, and receives the reflection component and the illumination component as input and splices them along the channel dimension. Through this layer, the network can capture the basic texture and edge information in the input image and complete the preliminary extraction of feature information; the output of the convolution layer is used as the input of the first deep feature extraction layer, and is downsampled through a convolution operation with a step size of 2 to reduce the spatial resolution of the feature map and increase its receptive field; the second deep feature extraction layer receives the output of the first deep feature extraction layer as input, and again passes through a Downsampling is performed with a convolution operation with a stride of 2. Compared with the first layer, this layer can capture more abstract and higher-level feature information; the third deep feature extraction layer receives the output of the second deep feature extraction layer as input, and downsamples through the third convolution operation with a stride of 2. Through this layer, the network can further extract and refine feature information, providing strong support for the final enhancement effect; the multi-scale feature fusion submodule restores the output of each deep feature extraction layer to the spatial resolution of the previous layer through an upsampling operation, and splices it with the output of the previous layer along the channel dimension to complete the fusion of features of different scales and levels; the output generation submodule outputs the fused feature map as an enhanced illumination component.

[0014] According to a low-illumination small target image recognition method based on deep learning in some embodiments of the present application, in step S2, the reflection component and the enhanced illumination component are reconstructed. The specific method is: the reflection component and the enhanced illumination component are multiplied and fused element by element to generate an enhanced target image.

[0015] According to a low-light small target image recognition method based on deep learning in some embodiments of the present application, in step S2, during the training of the multi-scale enhancement model, low-light and normal-light image pairs are randomly extracted in each training batch; the loss is calculated through forward propagation of the model, and the optimizer is used to backpropagate the loss and update the model parameters; at the end of every 5 epochs, the model performance is evaluated and the best model is recorded.

[0016] According to a low-illumination small target image recognition method based on deep learning in some embodiments of the present application, in step S3, the small target semantic segmentation model includes a backbone network DCSR and a feature fusion layer;

[0017] The backbone network DCSR includes an input layer, a convolutional layer and a layer composed of multiple basic building blocks (CSResidual block); the input of the input layer is the enhanced low-light small target image and the label map in the low-light small target data set; the convolutional layer performs preliminary feature extraction on the input image; the layer composed of multiple basic building blocks includes four layers, of which layer1 contains 3 basic building blocks, layer2 contains 4 basic building blocks, layer3 contains 6 basic building blocks, layer4 contains 3 basic building blocks and a void space pyramid pooling module; each basic building block is composed of two DC modules connected in sequence, and the DC module includes a convolutional layer, a batch normalization layer and a Leaky-ReLu activation function connected in sequence, and a channel attention module and a spatial attention module are inserted in sequence between the batch normalization layer and the Leaky-ReLu activation function in the second DC module, thereby improving the network's ability to capture important information in the image; the DC module In the block, the convolution layer is used to extract image features, and the convolution kernel is applied to each channel of the input to generate the same number of channels. These channels are then stacked according to the channel dimension, and a new feature map is generated by combining the generated multi-channel feature maps for point-by-point convolution. The information of different channels is weighted and combined to output the feature map; the batch normalization layer normalizes each channel of the output feature map so that the output data has a stable distribution; the Leaky-ReLu activation function maps the normalized feature map to the new space to generate the final feature map; the void spatial pyramid pooling module processes the output feature information of layer3 through four void convolution layers to obtain feature maps of four different scales; and then the global average pooling branch is used to obtain the global context feature map; the feature maps of four different scales are restored to the size of the original input feature map by upsampling, and the feature maps of four different scales and the global context feature map are spliced ​​in the channel dimension to obtain the spliced ​​feature map;

[0018] The feature fusion layer adopts a dense connection mechanism. In each module of the overall network framework, the outputs of all previous modules are spliced ​​and input into the basic building block of the backbone network DCSR contained in the current layer, and then transmitted to the next layer.

[0019] According to a method for low-light small target image recognition based on deep learning in some embodiments of the present application, in step S3, the specific implementation method of the dense connection mechanism adopted by the feature fusion layer is as follows: when building the overall network framework of the small target semantic segmentation model, the input image is first sent to the backbone network DCSR, and the initial feature extraction module X is used to extract the image. 0_0 Convolution operation is performed to extract basic feature information from the image; then, the extracted preliminary features are fed into a series of deep feature extraction modules X 1_0、X 2_0 、X 3_0 、X 4_0 The above five modules constitute the backbone network DCSR, which gradually mines the deep feature information in the image by downsampling the features and further extracting the features. As the number of layers increases, the depth of feature extraction gradually deepens, thus being able to capture more complex and subtle image features. 1_0 The output is upsampled with X 0 _0 The output is concatenated in the channel dimension and fed into module X 0_1 , the module X 0_1 It contains a basic building block of the backbone network DCSR, which completes the fusion and output of feature information of different scales, enhancing the model's ability to capture detailed features; other layers also complete the above operations in sequence, and after splicing with the output of each previous layer, they are jointly input into the basic building block of the backbone network DCSR contained in each layer, and continue to be transmitted to the next layer; finally, the dense connection of the entire network is completed, so that each layer in the network is connected to all previous layers to obtain richer feature information.

[0020] According to a low-light small target image recognition method based on deep learning in some embodiments of the present application, in step S3, when training the small target semantic segmentation model, a loss function combining Dice loss and Focal loss is used to better address the problems of category imbalance and pixel-level imbalance; the loss function is:

[0021] Loss=βDice·DiceLoss+(1-βDice)·FocalLoss (1)

[0022] Among them, βDice is a hyperparameter used to adjust the weight between Dice loss and Focal loss loss functions;

[0023] The Dice loss function is calculated based on the Dice coefficient. The calculation formulas for the Dice coefficient and DiceLoss are as follows:

[0024]

[0025] Where X represents the predicted output image after inputting the low-light small target image into the small target semantic segmentation model, Y represents the true label segmentation result, |X∩|Y| represents the number of intersection elements between X and Y, and |X| and |Y| represent the number of elements in X and Y respectively;

[0026] The FocalLoss loss function reduces the loss weight of easy-to-classify samples and increases the loss weight of difficult-to-classify samples, so that the model can pay more attention to difficult-to-classify samples (such as small objects). The formula of the Focal Loss loss function is as follows:

[0027] FocalLoss = -a(1-p) λ log(p) (4)

[0028] Among them, p is the probability of the small object semantic segmentation model predicting the correct category; α is a sample weight used to adjust the weights of easy-to-classify samples and difficult-to-classify samples, which is usually the inverse of the category frequency; λ represents the difficult-to-classify sample weight, which is used to weigh the difficult-to-classify samples and easy-to-classify samples.

[0029] According to a low-illumination small target image recognition method based on deep learning in some embodiments of the present application, in step S4, the Canny operator is selected to perform contour detection on the segmented image, bilateral filtering is used to remove noise, the Sobel operator is used to calculate the gradient amplitude and gradient, and the image is screened by non-maximum suppression interpolation method. The threshold is selected using an adaptive threshold method, thereby effectively suppressing false edges and noise edges, improving the performance of edge detection, and ensuring the accuracy of positioning.

[0030] According to a low-illumination small target image recognition method based on deep learning in some embodiments of the present application, in step S5, the contour center of gravity position is located according to the target edge features by calculating the spatial moment, and the Kalman filtering method is used for further smoothing and denoising and dynamic adjustment of weights to ensure the accuracy of the center positioning coordinates.

[0031] Beneficial results of the present invention: The present invention can achieve accurate and stable detection and recognition for images with characteristics such as low contrast, color distortion, and small targets. The present invention enhances low-light images through the constructed MSEM model, which helps to better retain and transmit multi-scale feature information, complete the fusion of deep-level features, and capture more detailed features; optimize the network architecture of the MSEM model to reduce redundancy and improve computational efficiency, making the network more lightweight and easy to train, and can achieve recognition in low-light environments. The present invention embeds a channel space attention mechanism in the backbone network DCSR through the constructed DR-UNet model to enhance the model's perception of small targets in low light; the structure of the network model is more in-depth, improving its feature extraction and expression capabilities, enabling the model to better adapt to image segmentation tasks of different scales and complexities, improving the generalization ability of the model, and optimizing the network structure and performance. The present invention significantly improves the performance of edge detection and provides more accurate target edge feature information for subsequent image positioning and other processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is an overall flow chart of the method of the present invention.

[0033] Figure 2 Schematic diagram of the decomposition module in the MSEM model of the present invention.

[0034] Figure 3 Schematic diagram of the enhancement module in the MSEM model of the present invention.

[0035] Figure 4 This is an example of the output results of the MSEM model of the present invention.

[0036] Figure 5 This is the network structure diagram of the DR-UNet model of the present invention.

[0037] Figure 6 This is the residual block network structure diagram.

[0038] Figure 7 This is the structural diagram of the attention mechanism.

[0039] Figure 8 This is an example of the output results of the DR-UNet model. DETAILED DESCRIPTION

[0040] The embodiments of the present invention are implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0041] This embodiment provides a low-light small target image recognition method based on deep learning, which includes three modules: MSEM module, DR-UNet module and contour detection module. The overall framework is shown in the figure. Figure 1 As shown in the figure, the MSEM module enhances the low-light image and sends the enhanced image to the DR-UNet module to segment the target. The contour detection module performs contour recognition and positioning on the segmented image. The specific steps of this method are as follows:

[0042] S1: Construct a low-light small target dataset;

[0043] Collect image data that meets the conditions of low illumination and small targets and the corresponding normal illumination images, and perform labelme annotation and preprocessing on the collected images, including resizing and normalization, to ensure that the images input to the network have a consistent format and range; divide the low illumination small target dataset into training set, validation set and test set to ensure the generalization ability of the model on different data.

[0044] S2: Construct a multi-scale enhancement model, and use the low-illumination images and corresponding normal-illumination images in the low-illumination small target dataset to train the multi-scale enhancement model to obtain an MSEM model. The specific process is as follows:

[0045] The constructed dataset containing low-light and normal-light image pairs is fed into the MSEM model to enable it to learn the mapping from low-light images to normal-light images. First, the low-light and normal-light image pairs are fed into the decomposition module of the MSEM model. The decomposition module consists of two sub-networks with shared parameters. The network structure is as follows: Figure 2 As shown in the figure, the two sets of data are each fed into a convolutional layer with 64 channels and a kernel size of 3 to extract shallow features. They are then fed into an activation sequence layer consisting of five convolutional layers and Leaky-ReLu activation blocks connected in sequence for deeper feature extraction. The convolutional layers have 64 channels and a kernel size of 3. The two sub-networks exchange and share information by sharing parameters. This design not only reduces the number of model parameters and computational complexity, but also improves the model's generalization ability. By sharing parameters, the two sub-networks can learn more robust and universal feature representations, better adapting to image enhancement tasks under different lighting conditions and making the mapping from low-light images to normal-light images more accurate and stable. The depth-extracted feature map output by the activation sequence layer is fed into the final reconstruction layer, which contains a convolutional layer and a sigmoid function. The convolutional layer converts the feature map output by the activation sequence layer from 64 channels back to 4 channels, two of which represent reflection components, which capture the surface reflection characteristics of objects in the image, and the other two channels correspond to illumination components, representing the lighting information in the image. The sigmoid function is then used to map the output to the 0-1 interval, and the feature map is divided into reflection and illumination components.

[0046] The decomposed illumination component and reflection component are sent to the enhancement module, and its network structure is as follows: Figure 3As shown in the figure, the illuminance component is enhanced and the reflectance component is denoised, and a more sophisticated multi-scale feature fusion strategy and an optimized network architecture are introduced to improve the quality and efficiency of image re-illumination. The enhancement module includes three submodules: feature extraction, multi-scale feature fusion and output generation; the feature extraction submodule includes a convolution layer and three deep feature extraction layers; first, the reflection component and the illumination component are spliced ​​in the channel dimension to form a 4-channel input image, the number of channels of the convolution layer is 64, and the size of the convolution kernel is 3. The feature information of the reflection component and the illumination component is preliminarily extracted to capture the basic texture and edge information in the input image; then three layers of deep extraction feature layers are constructed, and the output of the convolution layer is used as the input of the first deep feature extraction layer, and downsampled through a convolution operation with a stride of 2 to reduce the spatial resolution of the feature map and increase its receptive field; the second deep feature extraction layer receives the output of the first deep feature extraction layer as input, and downsamples again through a convolution operation with a stride of 2. Compared with the first layer, this layer may capture more abstract and higher-level feature information; the third deep feature extraction layer receives the output of the second deep feature extraction layer as input, and downsamples through a third convolution operation with a stride of 2. Through this layer, the network can further extract and refine feature information, providing strong support for the final enhancement effect. Each deep feature extraction layer incorporates a multi-scale feature fusion module. The output of each deep feature extraction layer is upsampled to match the output of the previous layer. The output is then concatenated to form the input to the upsampling layer, completing the fusion of features at different scales and levels. In each fusion step, the Leaky-ReLU activation function is used to increase the nonlinearity of the network. The fused feature maps are concatenated along the channel dimension to form the final fused feature map. The fused feature map is further processed through convolutional layers to fuse features at different scales and generate the final enhanced illumination component.

[0047] Reconstruct the reflection component and the enhanced illumination component: multiply the reflection component and the enhanced illumination component element by element to generate an enhanced target image.

[0048] When training the multi-scale enhancement model, pairs of low-light and normal-light images are randomly sampled for each training batch. The model forward propagates the loss, and the optimizer backpropagates the loss and updates the model parameters. At the end of every five epochs, model performance is evaluated, and the best model is recorded. The model state is saved at each training stage for subsequent loading and evaluation.

[0049] After feeding the low-light pictures of falling balls taken on the lake surface at night into the trained MSEM model, the contrast enhancement effect is as follows: Figure 4 shown.

[0050] S3: Construct a small object semantic segmentation model, and use the low-light small object image enhanced by the MSEM model and the label map in the low-light small object dataset to train the small object semantic segmentation model to obtain a DR-UNet model. The specific process is as follows:

[0051] Construct a small target semantic segmentation model, whose network framework is as follows Figure 5 As shown in Figure 2, it is divided into a backbone network DCS and a feature fusion layer. The structure of the backbone network DCSR is shown in Figure 2. Figure 6 As shown in , it includes an input layer, a convolutional layer, and a layer composed of multiple residual blocks (CSResidual block); wherein, the enhanced low-light small target image and the label map in the low-light small target data set are used as inputs of the input layer; the convolutional layer performs preliminary feature extraction on the image input by the input layer; the layer composed of multiple residual blocks is the core part of DCSR, which includes four layers, layer1 contains 3 residual blocks, layer2 contains 4 residual blocks, layer3 contains 6 residual blocks, and layer4 contains 3 residual blocks; each residual block contains two DC modules, and the DC module includes a convolutional layer, a batch normalization layer, and a Leaky-ReLu activation function connected in sequence. The channel attention module and the spatial attention module are inserted in sequence between the batch normalization layer and the Leaky-ReLu activation function in the second DC module. The structure diagram is as shown in Figure 7As shown in the figure, this improves the network's ability to capture important information in the image. In the DC module, the convolution layer is used to extract image features. A convolution kernel is applied to each input channel to generate the same number of channels. These channels are then stacked according to the channel dimension. A new feature map is generated by combining the generated multi-channel feature maps and performing point-by-point convolution. The information from different channels is weighted and combined to output the feature map. The output feature map is normalized by the batch normalization layer to reduce internal covariate shift. Finally, nonlinearity is introduced through the Leaky-ReLu activation function to generate a new feature representation for use in subsequent layers. The output of the second convolutional layer serves as the input to the channel attention module, which performs global average pooling and global max pooling on the feature map. The two pooled feature maps are passed through two shared fully connected layers, mapped to per-channel weights, and then summed. A sigmoid activation function is then applied to constrain the value of each element to a value between 0 and 1. The output of the previous layer is multiplied by the weighted output from the channel attention module to assign a weight to each channel. Together, these weights serve as the input to the spatial attention module, which performs global average pooling and global max pooling on the input feature map along the channel dimension. The results of average and max pooling are concatenated along the channel dimension to obtain the spatial weights of the feature map. This concatenated feature map is then convolved to represent the importance of each point. The spatial weights are then multiplied by the feature map output from the previous layer to assign a weight to each point in the feature map. Combining the two attention modules enhances the network's feature extraction capabilities. This design of the DCSR backbone network helps reduce computational overhead, accelerate training, and improve model generalization.

[0052] like Figure 5 As shown, the input image is fed into module X 0_0 , perform convolution operation on the input image to achieve preliminary extraction of feature information. 0_0 The output of module X 1_0 The module contains three basic building blocks of DCSR backbone network. The initial extracted features are downsampled and further feature extracted; the module X 1_0 The output of module X 2_0 The module contains four basic building blocks of the backbone network DCSR, which downsamples and extracts deeper features; 2_0 The output of module X 3_0 The module contains six basic building blocks of the backbone network DCSR, which continues to downsample and extract features to prepare for deeper feature fusion; 3_0 The output of module X 4 _0 Input, X 3_0The output feature map of is processed through four hole convolution layers respectively to obtain feature maps of four different scales; then processed through the global average pooling branch to obtain the global context feature map; restored to the size of the original input feature map by upsampling, the feature maps of four different scales and the global context feature map are spliced ​​in the channel dimension to obtain the spliced ​​feature map. Define a batch normalization layer and a ReLu activation function layer to normalize the feature map and perform nonlinear transformation to obtain the final feature map. Optimize the pooling method to retain more useful information. After the hole spatial pyramid pooling module, it is sent to the three backbone network basic building blocks of DCSR to complete the extraction of feature information. Module X is used to extract the feature information. 1_0 The output is upsampled with X 0_0 The output is concatenated in the channel dimension and fed into module X 0_1 , the module X 0_1 It contains a basic building block of the backbone network DCSR, which completes the fusion and output of feature information of different scales, enhancing the model's ability to capture detailed features; other layers also complete similar operations in sequence, and after splicing with the output of each previous layer, they are jointly input into the basic building block of the backbone network DCSR contained in each layer and continue to be transmitted to the next layer; finally, the dense connection of the entire network is completed, so that each layer in the network is connected to all previous layers to obtain richer feature information.

[0053] The enhanced low-light small target image obtained in step S2 and the label map in the dataset are input into the small target semantic segmentation model. The feature information is extracted through forward propagation through multiple convolutional layers and pooling layers, and the loss between the output layer and the true label is calculated. The loss function combining Dice loss and Focal loss is used:

[0054] Loss=βDice·DiceLoss+(1-βDice)·FocalLoss (1)

[0055] Among them, βDice is a hyperparameter used to adjust the weight between Dice loss and Focal loss; by adjusting the value of βDice, better performance can be obtained on different tasks and datasets.

[0056] The optimizer updates the network weights based on the forward propagation gradient and dynamically adjusts the learning rate based on the change in loss. Backpropagation promotes the network to produce more accurate predictions in the next forward propagation, completing the training of the entire small object semantic segmentation model and obtaining the DR-UNet model. The low-light enhanced image is used as the input of the DR-UNet model for semantic segmentation. The results are as follows: Figure 8 shown.

[0057] S4: Perform contour detection on the segmented image to extract the target edge features of the segmented image;

[0058] Edge detection mainly includes multiple steps such as noise reduction filtering, calculation of gradient amplitude and direction, non-maximum suppression and hysteresis threshold processing. First, in the Gaussian filtering stage, the traditional Canny algorithm uses a Gaussian function to smooth the image to reduce the impact of noise on edge detection. However, while removing noise, Gaussian filtering may also cause some important edge information in the image to be blurred or lost. Considering that for image recognition of small targets, the blurring of edge information may greatly affect the final positioning and tracking, this embodiment uses bilateral filtering instead of Gaussian filtering to pre-process the image to eliminate noise interference and better preserve the edge information of the target. The Canny operator is selected to perform contour detection on the segmented image, the Sobel operator is used to calculate the gradient amplitude and gradient, the image edge is finely extracted by the first-order differential operator, and then the non-maximum suppression operation is performed using the gradient information to ensure that only the local maximum at the edge is retained. In the application of the first-order differential operator, the horizontal and vertical directions are specifically selected to effectively extract edge features. Finally, the threshold is selected using an adaptive threshold method according to actual needs, and the processed image is edge-connected to obtain complete and accurate edge detection results, effectively suppressing false edges and noise edges, improving the performance of edge detection, and ensuring the accuracy of positioning.

[0059] S5: Target center positioning is performed based on the target edge features to obtain the position of the small target.

[0060] When obtaining the coordinates of the target's center point, the target is treated as a two-dimensional object with uniform density, and its center of gravity is determined by finding its center of gravity. In a two-dimensional image, the center of gravity of the target body is the average position of all pixels contained within it, also known as the center of mass or centroid. The contour's center of gravity is located by calculating spatial moments. To ensure the stability of the positioning results and the accuracy of the center coordinates, a Kalman filter is used for further smoothing and denoising, as well as dynamic weight adjustment to ensure the accuracy of the center coordinates.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A low-light small target image recognition method based on deep learning, characterized in that: The method comprises the following steps: S1: Collect low-illumination small target images and corresponding normal-illumination images, and annotate and preprocess the collected images to obtain a low-illumination small target dataset; S2: Constructing a multi-scale enhancement model, using the low-illumination images and corresponding normal-illumination images in the low-illumination small target dataset to train the multi-scale enhancement model to obtain an MSEM model, thereby achieving illumination enhancement of the low-illumination small target images; the multi-scale enhancement model includes a decomposition module and an enhancement module; The decomposition module includes three submodules: shallow feature extraction, activation layer sequence, and final reconstruction layer; the shallow feature extraction submodule includes a convolution layer for performing preliminary feature extraction on the input low-light image and the corresponding normal-light image; the activation layer sequence submodule performs deep feature extraction on the preliminary extracted feature information, which is connected in sequence by five convolution blocks consisting of convolution layers and Leaky-ReLu activation functions; the final reconstruction layer submodule includes a convolution layer and a sigmoid function, wherein the convolution layer converts the feature map output by the activation layer sequence submodule from 64 channels back to 4 channels, and the sigmoid function maps the output to the 0-1 interval, and the feature map is divided into a reflection component and an illumination component; The enhancement module includes three submodules: feature extraction, multi-scale feature fusion and output generation; the feature extraction submodule includes a convolution layer and three deep feature extraction layers, wherein the convolution layer receives the reflection component and the illumination component as input and splices them along the channel dimension. The network can capture the basic texture and edge information in the input image and complete the preliminary extraction of feature information; the output of the convolution layer is used as the input of the first deep feature extraction layer and is downsampled through a convolution operation with a step size of 2; the second deep feature extraction layer receives the output of the first deep feature extraction layer as input and is downsampled again through a convolution operation with a step size of 2; the third deep feature extraction layer receives the output of the second deep feature extraction layer as input and is downsampled through a third convolution operation with a step size of 2; the multi-scale feature fusion submodule restores the output of each deep feature extraction layer to the spatial resolution of the previous layer through an upsampling operation, and splices it with the output of the previous layer along the channel dimension to complete the fusion of features of different scales and levels; the output generation submodule outputs the fused feature map as an enhanced illumination component; S3: Construct a small object semantic segmentation model, and use the low-light small object image enhanced by the MSEM model and the label map in the low-light small object dataset to train the small object semantic segmentation model to obtain a DR-UNet model to achieve segmentation and recognition of the enhanced image; the small object semantic segmentation model includes a backbone network DCSR and a feature fusion layer; The backbone network DCSR includes an input layer, a convolution layer and a layer composed of multiple basic building blocks; the input of the input layer is the enhanced low-light small target image and the label map in the low-light small target data set; the convolution layer performs preliminary feature extraction on the input image; the layer composed of multiple basic building blocks includes four layers, wherein layer1 contains 3 basic building blocks, layer2 contains 4 basic building blocks, layer3 contains 6 basic building blocks, layer4 contains 3 basic building blocks and a void space pyramid pooling module; each basic building block is composed of two DC modules connected in sequence, and the DC module The invention comprises a convolutional layer, a batch normalization layer and a Leaky-ReLu activation function connected in sequence. A channel attention module and a spatial attention module are sequentially inserted between the batch normalization layer and the Leaky-ReLu activation function in the second DC module, thereby improving the network's ability to capture important information in the image. In the DC module, the convolutional layer is used to extract image features, perform weighted combination of information from different channels, and output a feature map. The batch normalization layer normalizes each channel of the output feature map so that the output data has a stable distribution. The Leaky-ReLu activation function maps the normalized feature map to a new space to generate a final feature map. The dilated spatial pyramid pooling module processes the output feature information of layer 3 through four dilated convolutional layers to obtain feature maps of four different scales. This is then processed by the global average pooling branch to obtain a global context feature map. The feature maps of the four different scales are restored to the size of the original input feature map by upsampling, and the global context feature map is concatenated in the channel dimension to obtain a concatenated feature map. The feature fusion layer adopts a dense connection mechanism. In each module of the overall network framework, the outputs of all previous modules are spliced ​​together and input into the basic building block of the backbone network DCSR contained in the current layer, and then continue to be transmitted to the next layer. S4: Perform contour detection on the segmented image to extract the target edge features of the segmented image; S5: Target center positioning is performed based on the target edge features to obtain the position of the small target.

2. The method for low-light small target image recognition based on deep learning according to claim 1, characterized in that: In the step S2, the reflection component and the enhanced illumination component are reconstructed by performing element-by-element multiplication and fusion on the reflection component and the enhanced illumination component to generate an enhanced target image.

3. A method for low-light small target image recognition based on deep learning according to claim 1 or 2, characterized in that: In step S2, during the training of the multi-scale enhancement model, low-light and normal-light image pairs are randomly selected in each training batch; the loss is calculated by forward propagation of the model, and the optimizer is used to backpropagate the loss and update the model parameters; at the end of every 5 epochs, the model performance is evaluated and the best model is recorded.

4. The method for low-light small target image recognition based on deep learning according to claim 1, characterized in that: In step S3, the specific implementation of the dense connection mechanism adopted by the feature fusion layer is as follows: when building the overall network framework of the small object semantic segmentation model, the input image is first sent to the backbone network DCSR, and the initial feature extraction module X is used to extract the image. 0_0 Convolution operation is performed to extract basic feature information from the image; then, the extracted preliminary features are fed into a series of deep feature extraction modules X 1_0 、X 2_0 、X 3_0 、X 4_0 The above five modules form the backbone network DCSR, which gradually mines the deep feature information in the image by downsampling the features and further extracting the features; 1_0 The output is upsampled with X 0_0 The output is concatenated in the channel dimension and fed into module X 0_1 , the module X 0_1 It contains a basic building block of the backbone network DCSR, which completes the fusion and output of feature information of different scales; other layers also complete the above operations in sequence, and after splicing with the output of each previous layer, they are input into the basic building block of the backbone network DCSR contained in each layer, and then continue to be transmitted to the next layer; finally, the dense connection of the entire network is completed, so that each layer in the network is connected to all previous layers to obtain richer feature information.

5. The method for low-light small target image recognition based on deep learning according to claim 4, characterized in that: In step S3, when training the small object semantic segmentation model, the loss function adopts a loss function combining Dice loss and Focalloss: Loss=βDice·DiceLoss+(1-βDice)·FocalLoss (1) Among them, βDice is a hyperparameter used to adjust the weight between Dice loss and Focal loss loss functions; The Dice loss function is calculated based on the Dice coefficient. The calculation formulas for the Dice coefficient and DiceLoss are as follows: Where X represents the predicted output image after inputting the low-light small target image into the small target semantic segmentation model, Y represents the true label segmentation result, |X∩Y| represents the number of intersection elements between X and Y, and |X| and |Y| represent the number of elements in X and Y respectively; The formula of the FocalLoss loss function is as follows: FocalLoss=-a(1-p) λ log(p) (4) Among them, p is the probability that the small object semantic segmentation model predicts the correct category; α is a sample weight used to adjust the weights of easy-to-classify samples and difficult-to-classify samples, which is the inverse of the category frequency; λ represents the weight of difficult-to-classify samples.

6. The method for low-light small target image recognition based on deep learning according to claim 1, characterized in that: In step S4, the Canny operator is selected to perform contour detection on the segmented image, bilateral filtering is used to remove noise, the Sobel operator is used to calculate the gradient amplitude and gradient, and the image is screened using the non-maximum suppression interpolation method, and the threshold is selected using an adaptive threshold method.

7. The method for low-light small target image recognition based on deep learning according to claim 1, characterized in that: In step S5, according to the target edge features, the contour center of gravity position is located by calculating the spatial moment, and the Kalman filter method is used for further smoothing and denoising and dynamic weight adjustment to ensure the accuracy of the center positioning coordinates.

Images