Low-illumination small target image recognition method based on deep learning

Abstract

The invention provides a low-illumination small target image recognition method based on deep learning, and belongs to the field of image recognition. The method comprises the following steps: establishing a low-illumination small target data set, training a multi-scale enhancement model and a small target semantic segmentation model, enhancing a low-illumination image, sending the enhanced image into the semantic segmentation model, realizing segmentation identification of a small target, and positioning a target track according to a segmentation image of the small target. According to the invention, for a low-illumination small target image, the network identification precision is improved, and the probability of false detection and missing detection is effectively reduced; while the brightness and the contrast ratio of the low-light image are improved, the amplification of noise and the loss of details are inhibited, and the detail information is reserved, so that the accurate and stable detection and identification of the enhanced image are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a low-illumination small target image recognition method based on deep learning. Background Art

[0002] Image enhancement and recognition tasks are important directions in the field of image recognition. With the development of society, image enhancement and recognition tasks for different environments, such as low-light environments, are becoming increasingly important. Low-light small target recognition is a branch of target recognition tasks. Due to the complexity and diversity of the real environment in which images are acquired, images often have low contrast, color distortion, small targets, etc., resulting in loss of image details, blurred contours, and difficulty in detection and recognition. Therefore, low-light image enhancement and recognition tasks are still very challenging. It is necessary to improve the brightness and contrast of low-light images while suppressing the amplification of noise and the loss of details during the enhancement process, retaining detail information, and then achieving accurate and stable detection and recognition of the enhanced images.

[0003] So far, researchers have proposed a variety of networks that combine neural networks with target recognition. For example, in 2018, Zhou et al. proposed a new encoder-decoder architecture U-Net++, in which the encoder and decoder subnetworks are connected by a series of nested dense jump paths, reducing the semantic gap between the feature maps of the encoder and decoder subnetworks. However, in low-light small target scenes, U-Net++ may lose certain feature information in multiple downsampling and upsampling operations. This deficiency is particularly obvious when processing targets with a low proportion of the entire image, resulting in a decrease in segmentation effect. Wang et al. proposed the HCF-Net network for the problems of small target detection loss and low background discrimination. The network improves the feature information of different scales by introducing parallelized patch-aware attention modules, thereby improving the recognition accuracy of small target images. However, the overall network generalization ability is insufficient. For small targets in low-light scenes, there are still problems such as loss of image details, increased noise, and reduced contrast between the target and the background, which make it impossible to effectively recognize the image. Therefore, it is particularly important to study a recognition algorithm suitable for low-light small target images, so as to improve the segmentation accuracy of low-light small targets. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention proposes a low-illumination small target recognition method based on deep learning, which enhances the perception ability of low-illumination small targets, improves the recognition ability and segmentation accuracy of small targets, and realizes the positioning of small targets.

[0005] The technical solution of the present invention is as follows:

[0006] A low-light small target image recognition method based on deep learning includes the following steps:

[0007] S1: Collect low-illumination small target images and corresponding normal-illumination images, and annotate and preprocess the collected images to obtain a low-illumination small target dataset;

[0008] S2: construct a multi-scale enhancement model, and use the low-illumination images and corresponding normal-illumination images in the low-illumination small target data set to train the multi-scale enhancement model to obtain a MSEM model, so as to achieve illumination enhancement of the low-illumination small target image;

[0009] S3: constructing a small target semantic segmentation model, using the low-light small target image enhanced by the MSEM model and the label map in the low-light small target data set to train the small target semantic segmentation model, obtaining a DR-UNet model, and realizing segmentation and recognition of the enhanced image;

[0010] S4: Perform contour detection on the segmented image to extract target edge features of the segmented image;

[0011] S5: locating the center of the target based on the target edge features to obtain the position of the small target.

[0012] According to a method for low-light small target image recognition based on deep learning in some embodiments of the present application, in step S2, the multi-scale enhancement model includes a decomposition module and an enhancement module; the decomposition module includes three sub-modules: shallow feature extraction, activation layer sequence and final reconstruction layer; the shallow feature extraction sub-module includes a convolution layer, the number of channels of the convolution layer is 64, the size of the convolution kernel is 3, and it is used to perform preliminary feature extraction on the input low-light image and the corresponding normal illumination image; the activation sequence layer sub-module performs deep feature extraction on the initially extracted feature information, which is performed through five convolution layers. The convolution blocks composed of a convolution layer and a Leaky-ReLu activation function are connected in sequence, wherein the number of channels of the convolution layer is 64 and the size of the convolution kernel is 3; the final reconstruction layer submodule remaps the feature map extracted by the activation sequence layer submodule back to the original image space to generate a reconstructed image or feature map, and the final reconstruction layer submodule includes a convolution layer and a sigmoid function, wherein the convolution layer converts the feature map output by the activation sequence layer submodule from 64 channels back to 4 channels, and the sigmoid function maps the output to the interval of 0-1, and the feature map is divided into a reflection component and an illumination component;

[0013] The enhancement module includes three submodules: feature extraction, multi-scale feature fusion and output generation; the feature extraction submodule includes a convolution layer and three deep feature extraction layers; the convolution layer has 64 channels and a convolution kernel size of 3, receives reflection component and illumination component as input, and concatenates them along the channel dimension. Through this layer, the network can capture the basic texture and edge information in the input image and complete the preliminary extraction of feature information; the output of the convolution layer is used as the input of the first deep feature extraction layer, and is downsampled through a convolution operation with a step size of 2 to reduce the spatial resolution of the feature map and increase its receptive field; the second deep feature extraction layer receives the output of the first deep feature extraction layer as input, and again passes through a Downsampling is performed with a convolution operation with a step size of 2. Compared with the first layer, this layer can capture more abstract and higher-level feature information; the third deep feature extraction layer receives the output of the second deep feature extraction layer as input, and downsamples through the third convolution operation with a step size of 2. Through this layer, the network can further extract and refine feature information, providing strong support for the final enhancement effect; the multi-scale feature fusion submodule restores the output of each deep feature extraction layer to the spatial resolution of the previous layer through an upsampling operation, and splices it with the output of the previous layer along the channel dimension to complete the fusion of features of different scales and levels; the output generation submodule outputs the fused feature map as an enhanced illumination component.

[0014] According to a low-illumination small target image recognition method based on deep learning in some embodiments of the present application, in step S2, the reflection component and the enhanced illumination component are reconstructed, and the specific method is: the reflection component and the enhanced illumination component are multiplied and fused element by element to generate an enhanced target image.

[0015] According to a low-illumination small target image recognition method based on deep learning in some embodiments of the present application, in step S2, during the training of the multi-scale enhancement model, low-illumination and normal-illumination image pairs are randomly selected for each training batch; the loss is calculated through forward propagation of the model, and the optimizer is used to backpropagate the loss and update the model parameters; at the end of every 5 epochs, the model performance is evaluated and the best model is recorded.

[0016] According to a low-illumination small target image recognition method based on deep learning in some embodiments of the present application, in step S3, the small target semantic segmentation model includes a backbone network DCSR and a feature fusion layer;

[0017] The backbone network DCSR includes an input layer, a convolution layer and a layer composed of multiple basic building blocks (CSResidual block); the input of the input layer is the enhanced low-light small target image and the label map in the low-light small target data set; the convolution layer performs preliminary feature extraction on the input image; the layer composed of multiple basic building blocks includes four layers, among which layer1 includes 3 basic building blocks, layer2 includes 4 basic building blocks, layer3 includes 6 basic building blocks, layer4 includes 3 basic building blocks and a void space pyramid pooling module; each basic building block is composed of two DC modules connected in sequence, and the DC module includes a convolution layer, a batch normalization layer and a Leaky-ReLu activation function connected in sequence, and a channel attention module and a spatial attention module are inserted in sequence between the batch normalization layer and the Leaky-ReLu activation function in the second DC module, so as to enhance the network's ability to capture important information in the image; the DC module In the block, the convolution layer is used to extract image features. The convolution kernel is applied to each channel of the input to generate the same number of channels. These channels are then stacked according to the channel dimension, and a new feature map is generated by combining the generated multi-channel feature maps for point-by-point convolution. The information of different channels is weighted and combined to output the feature map. The batch normalization layer normalizes each channel of the output feature map so that the output data has a stable distribution. The Leaky-ReLu activation function maps the normalized feature map to a new space to generate the final feature map. The atrous spatial pyramid pooling module processes the output feature information of layer3 through four atrous convolution layers to obtain feature maps of four different scales. The global average pooling branch is then used to obtain the global context feature map. The feature maps of four different scales and the global context feature map are restored to the size of the original input feature map by upsampling, and the feature maps of four different scales and the global context feature map are spliced ​​in the channel dimension to obtain the spliced ​​feature map.

[0018] The feature fusion layer adopts a dense connection mechanism. In each module of the overall network framework, the outputs of all previous modules are spliced ​​and input into the basic building blocks of the backbone network DCSR contained in the current layer, and then transmitted to the next layer.

[0019] According to a method for low-light small target image recognition based on deep learning in some embodiments of the present application, in step S3, the specific implementation method of the dense connection mechanism adopted by the feature fusion layer is as follows: when building the overall network framework of the small target semantic segmentation model, firstly, the input image is sent to the backbone network DCSR, and then the initial feature extraction module X is used to extract the image. 0_0 Convolution operation is performed to extract basic feature information from the image; then, the extracted preliminary features are sent to a series of deep feature extraction modules X 1_0, X 2_0 , X 3_0 , X 4_0 The above five modules constitute the backbone network DCSR, which gradually mines the deep feature information in the image by downsampling the features and further extracting the features; with the increase of the number of layers, the depth of feature extraction gradually deepens, so that more complex and subtle image features can be captured; module X 1_0 The output is upsampled with X 0 _0 The outputs are concatenated in the channel dimension and fed into module X 0_1 , the module X 0_1 It contains a basic building block of the backbone network DCSR, which completes the fusion and output of feature information of different scales, and enhances the model's ability to capture detailed features; other layers also complete the above operations in sequence, and after splicing with the output of each previous layer, they are jointly input into the basic building block of the backbone network DCSR contained in each layer, and continue to be transmitted to the next layer; finally, the dense connection of the entire network is completed, so that each layer in the network is connected to all previous layers to obtain richer feature information.

[0020] According to a low-light small target image recognition method based on deep learning in some embodiments of the present application, in step S3, when training the small target semantic segmentation model, the loss function adopts a loss function combining Dice loss and Focal loss to better deal with the problems of category imbalance and pixel level imbalance; the loss function is:

[0021] Loss=βDice·DiceLoss+(1-βDice)·FocalLoss (1)

[0022] Among them, βDice is a hyperparameter used to adjust the weight between Dice loss and Focal loss loss functions;

[0023] The Dice loss function is calculated based on the Dice coefficient. The calculation formulas of the Dice coefficient and DiceLoss are as follows:

[0024]

[0025] Where X represents the predicted output image obtained by inputting the low-light small target image into the small target semantic segmentation model, Y represents the real label segmentation result, |X∩|Y| represents the number of intersection elements between X and Y, and |X| and |Y| represent the number of elements in X and Y respectively;

[0026] The FocalLoss loss function reduces the loss weight of easy-to-classify samples and increases the loss weight of difficult-to-classify samples, so that the model can pay more attention to difficult-to-classify samples (such as small targets); the formula of the Focal Loss loss function is as follows:

[0027] FocalLoss = -a(1-p) λ log(p) (4)

[0028] Among them, p is the probability that the small object semantic segmentation model predicts the correct category; α is a sample weight used to adjust the weights of easy-to-classify samples and difficult-to-classify samples, which is usually the inverse of the category frequency; λ represents the difficult-to-classify sample weight, which is used to weigh the difficult-to-classify samples and easy-to-classify samples.

[0029] According to a low-illumination small target image recognition method based on deep learning in some embodiments of the present application, in step S4, the Canny operator is selected to perform contour detection on the segmented image, bilateral filtering is used to remove noise, the Sobel operator is used to calculate the gradient amplitude and gradient, and the image is screened by non-maximum suppression interpolation method. The threshold is selected using an adaptive threshold method, thereby effectively suppressing false edges and noise edges, improving the performance of edge detection, and ensuring the accuracy of positioning.

[0030] According to a low-illumination small target image recognition method based on deep learning in some embodiments of the present application, in step S5, the contour center of gravity position is located by calculating the spatial moment according to the target edge features, and the Kalman filtering method is used for further smoothing and denoising and dynamic adjustment of weights to ensure the accuracy of the center positioning coordinates.

[0031] Beneficial results of the present invention: The present invention can achieve accurate and stable detection and recognition for images with low contrast, color distortion, small targets and other characteristics. The present invention enhances low-light images through the constructed MSEM model, which helps to better retain and transmit multi-scale feature information, complete the fusion of deep-level features, and capture more detailed features; optimize the network architecture of the MSEM model to reduce redundancy and improve computational efficiency, making the network more lightweight and easy to train, and can achieve recognition in low-light environments. The present invention embeds a channel space attention mechanism in the backbone network DCSR through the constructed DR-UNet model to enhance the model's perception of small targets in low light; the structure of the network model is more in-depth, improving its feature extraction and expression capabilities, enabling the model to better adapt to image segmentation tasks of different scales and complexities, improving the generalization ability of the model, and optimizing the network structure and performance. The present invention significantly improves the performance of edge detection and provides more accurate target edge feature information for subsequent image positioning and other processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is an overall flow chart of the method of the present invention.

[0033] Figure 2 Schematic diagram of the decomposition module in the MSEM model of the present invention.

[0034] Figure 3 Schematic diagram of the enhancement module in the MSEM model of the present invention.

[0035] Figure 4 This is an example of the output results of the MSEM model of the present invention.

[0036] Figure 5 This is the network structure diagram of the DR-UNet model of the present invention.

[0037] Figure 6 This is the residual block network structure diagram.

[0038] Figure 7 This is the structure diagram of the attention mechanism.

[0039] Figure 8 This is an example of the output results of the DR-UNet model. DETAILED DESCRIPTION

[0040] The embodiments of the present invention are implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0041] This embodiment provides a low-light small target image recognition method based on deep learning, which includes three modules: MSEM module, DR-UNet module and contour detection module. Figure 1 As shown, the MSEM module enhances the low-light image and sends the enhanced image to the DR-UNet module to segment the target. The contour detection module recognizes and locates the contour of the segmented image. The specific steps of this method are as follows:

[0042] S1: Construct a low-light small target dataset;

[0043] Collect image data that meets the conditions of low illumination and small targets and the corresponding normal illumination images, and perform labelme annotation and preprocessing on the collected images, including resizing and normalization, to ensure that the images input to the network have a consistent format and range; divide the low illumination small target dataset into training set, validation set and test set to ensure the generalization ability of the model on different data.

[0044] S2: construct a multi-scale enhancement model, and use the low-illumination images and corresponding normal-illumination images in the low-illumination small target dataset to train the multi-scale enhancement model to obtain a MSEM model; the specific process is as follows:

[0045] The constructed dataset containing low-light and normal-light image pairs is fed into the MSEM model to enable it to learn the mapping from low-light images to normal-light images. First, the low-light and normal-light image pairs are fed into the decomposition module of the MSEM model. The decomposition module consists of two sub-networks with shared parameters. The network structure is as follows: Figure 2 As shown in the figure, the two sets of data are respectively sent to a convolution layer with 64 channels and a convolution kernel size of 3 to complete shallow feature extraction; then they are sent to the activation sequence layer composed of five convolution blocks composed of convolution layers and Leaky-ReLu activation functions connected in sequence for deep feature extraction, where the convolution layer has 64 channels and a convolution kernel size of 3; the two sub-networks exchange and share information by sharing parameters. This design not only reduces the number of parameters of the model and reduces the computational complexity, but also improves the generalization ability of the model. By sharing parameters, the two sub-networks can learn more robust and universal feature representations, thereby better adapting to image enhancement tasks under different lighting conditions, making the mapping from low-light images to normal-light images more accurate and stable. The deeply extracted feature map output by the activation sequence layer is sent to the final reconstruction layer, which contains a convolution layer and a sigmoid function. The convolution layer converts the feature map output by the activation sequence layer from 64 channels back to 4 channels, two of which represent reflection components, which capture the surface reflection characteristics of objects in the image, and the other two channels correspond to illumination components, which represent the lighting information in the image. The sigmoid function is then used to map the output to the 0-1 interval, and the feature map is divided into reflection component and illumination component.

[0046] The decomposed illumination component and reflection component are sent to the enhancement module, and its network structure is as follows: Figure 3As shown in the figure, the illuminance component is enhanced and the reflectance component is denoised, and a more sophisticated multi-scale feature fusion strategy and an optimized network architecture are introduced to improve the quality and efficiency of image re-illumination. The enhancement module includes three submodules: feature extraction, multi-scale feature fusion and output generation; the feature extraction submodule includes a convolution layer and three deep feature extraction layers; first, the reflection component and the illumination component are spliced ​​in the channel dimension to form a 4-channel input image, the number of channels of the convolution layer is 64, the size of the convolution kernel is 3, the feature information of the reflection component and the illumination component is preliminarily extracted, and the basic texture and edge information in the input image is captured; then three layers of deep extraction feature layers are constructed, the output of the convolution layer is used as the input of the first deep feature extraction layer, and down-sampled through a convolution operation with a step size of 2 to reduce the spatial resolution of the feature map and increase its receptive field; the second deep feature extraction layer receives the output of the first deep feature extraction layer as input, and down-samples again through a convolution operation with a step size of 2. Compared with the first layer, this layer may capture more abstract and higher-level feature information; the third deep feature extraction layer receives the output of the second deep feature extraction layer as input, and down-samples through a third convolution operation with a step size of 2. Through this layer, the network can further extract and refine feature information, providing strong support for the final enhancement effect. Each deep feature extraction layer introduces a multi-scale feature fusion module, which upsamples the output of each deep feature extraction layer to match its size with the output of the previous layer, and then splices them together to form the input of the upsampling layer to complete the fusion of features of different scales and levels. In each fusion step, the Leaky-ReLU activation function is used to increase the nonlinearity of the network. The fused feature maps are spliced ​​together in the channel dimension to form the final fused feature map. The fused feature map is further processed by the convolution layer to fuse features of different scales to generate the final enhanced illumination component.

[0047] Reconstruct the reflection component and the enhanced illumination component: multiply and fuse the reflection component and the enhanced illumination component element by element to generate an enhanced target image.

[0048] When training the multi-scale enhancement model, randomly select low-light and normal-light image pairs for each training batch, calculate the loss through the model forward propagation, use the optimizer to backpropagate the loss and update the model parameters. At the end of every 5 epochs, evaluate the model performance and record the best model. Save the model state at each training stage for subsequent loading and evaluation.

[0049] After sending the low-light pictures of falling balls taken on the lake surface at night into the trained MSEM model, the contrast enhancement effect is as follows: Figure 4 shown.

[0050] S3: construct a small target semantic segmentation model, and use the low-light small target image enhanced by the MSEM model and the label map in the low-light small target dataset to train the small target semantic segmentation model to obtain a DR-UNet model; the specific process is as follows:

[0051] Construct a small target semantic segmentation model, whose network framework is as follows Figure 5 As shown in FIG. 1 , it is divided into a backbone network DCS and a feature fusion layer. The structure diagram of the backbone network DCSR is shown in FIG. Figure 6 As shown in FIG, the invention comprises an input layer, a convolutional layer and a layer composed of multiple residual blocks (CSResidual block); wherein the enhanced low-light small target image and the label map in the low-light small target data set are used as the input of the input layer; the convolutional layer performs preliminary feature extraction on the image input by the input layer; the layer composed of multiple residual blocks is the core part of DCSR, and comprises four layers, layer1 comprises 3 residual blocks, layer2 comprises 4 residual blocks, layer3 comprises 6 residual blocks, and layer4 comprises 3 residual blocks; each residual block comprises two DC modules, and the DC module comprises a convolutional layer, a batch normalization layer and a Leaky-ReLu activation function connected in sequence, and a channel attention module and a spatial attention module are inserted in sequence between the batch normalization layer and the Leaky-ReLu activation function in the second DC module, and the structure diagram is shown in FIG. Figure 7As shown in Figure 1, the network's ability to capture important information in the image is improved. In the DC module, the convolution layer is used to extract image features. The convolution kernel is applied to each input channel to generate the same number of channels. These channels are then stacked according to the channel dimension, and a new feature map is generated by combining the generated multi-channel feature maps for point-by-point convolution. The information of different channels is weighted and combined to output the feature map. The output feature map is normalized by the batch normalization layer to reduce internal covariate shift. Finally, nonlinearity is introduced through the Leaky-ReLu activation function to generate a new feature representation for use in subsequent layers. The output of the second convolution layer is used as the input of the channel attention module, and the feature map is globally averaged and globally max pooled. The two pooled feature maps are respectively passed through two shared fully connected layers and mapped to the weight of each channel. The mapped weights are added, followed by a sigmoid activation function to limit the value of each element between 0 and 1. The output of the previous level is multiplied by the weight output obtained by the channel attention module to assign weights to each channel. Together, they are used as the input of the spatial attention module, and the input feature map is globally averaged and globally max pooled in the channel dimension. The results of average pooling and max pooling are spliced ​​along the channel dimension to obtain the spatial weight of the feature map. The spliced ​​feature map is convoluted to characterize the importance of each point, and the feature map output of the previous level is multiplied by the obtained spatial weight to assign weights to each point on the feature map. The network's ability to extract features is enhanced by combining the two attention modules. The design of the backbone network DCSR helps reduce the amount of calculation, accelerate the training process, and improve the generalization ability of the model.

[0052] like Figure 5 As shown, the input image is fed into module X 0_0 , perform convolution operation on the input image to achieve preliminary extraction of feature information. 0_0 The output of module X 1_0 The module contains three basic building blocks of the backbone network DCSR. Down-sampling and further feature extraction of the initially extracted features; module X 1_0 The output of module X 2_0 The module contains four basic building blocks of the backbone network DCSR, which downsamples and extracts deeper features; 2_0 The output of module X 3_0 The module contains six basic building blocks of the backbone network DCSR, which continues to downsample and extract features to prepare for deeper feature fusion; module X 3_0 The output of module X 4 _0 Input, X 3_0The output feature map of is processed through four hole convolution layers respectively to obtain feature maps of four different scales; then processed through the global average pooling branch to obtain the global context feature map; restored to the size of the original input feature map by upsampling, the feature maps of four different scales and the global context feature map are spliced ​​in the channel dimension to obtain the spliced ​​feature map. Define a batch normalization layer and a ReLu activation function layer to normalize the feature map and perform nonlinear transformation to obtain the final feature map. Optimize the pooling method to retain more useful information. After the hole space pyramid pooling module, it is sent to the three backbone network DCSR basic building blocks to complete the extraction of feature information. Module X 1_0 The output is upsampled with X 0_0 The outputs are concatenated in the channel dimension and fed into module X 0_1 , the module X 0_1 It contains a basic building block of the backbone network DCSR, which completes the fusion and output of feature information of different scales, and enhances the model's ability to capture detailed features; other layers also complete similar operations in turn, and after splicing with the output of each previous layer, they are jointly input into the basic building block of the backbone network DCSR contained in each layer, and continue to be transmitted to the next layer; finally, the dense connection of the entire network is completed, so that each layer in the network is connected to all previous layers to obtain richer feature information.

[0053] The enhanced low-light small target image obtained in step S2 and the label map in the data set are input into the small target semantic segmentation model. The feature information is extracted through forward propagation through multiple convolutional layers and pooling layers, and the loss between the output layer and the true label is calculated. The loss function combining Dice loss and Focal loss is used:

[0054] Loss=βDice·DiceLoss+(1-βDice)·FocalLoss (1)

[0055] Among them, βDice is a hyperparameter used to adjust the weight between Dice loss and Focal loss; by adjusting the value of βDice, better performance can be obtained on different tasks and datasets.

[0056] The optimizer updates the network weights according to the forward propagation gradient and dynamically adjusts the learning rate according to the change of loss. Back propagation promotes the network to produce more accurate predictions in the next forward propagation, completing the training of the entire small target semantic segmentation model and obtaining the DR-UNet model. The low-light enhanced image is used as the input of the DR-UNet model for semantic segmentation. The results are as follows: Figure 8 shown.

[0057] S4: Perform contour detection on the segmented image to extract target edge features of the segmented image;

[0058] Edge detection mainly includes multiple links such as noise reduction filtering, calculation of gradient amplitude and direction, non-maximum suppression and hysteresis threshold processing. First, in the Gaussian filtering stage, the traditional Canny algorithm uses a Gaussian function to smooth the image to reduce the impact of noise on edge detection. However, while removing noise, Gaussian filtering may also cause some important edge information in the image to be blurred or lost. Considering the image recognition for small targets, the blurring of edge information may greatly affect the final positioning and tracking problems. Therefore, this embodiment uses bilateral filtering instead of Gaussian filtering to pre-process the image to eliminate noise interference and better preserve the edge information of the target. The Canny operator is selected to perform contour detection on the segmented image, the Sobel operator is used to calculate the gradient amplitude and gradient, the image edge is finely extracted by the first-order differential operator, and then the non-maximum suppression operation is performed using the gradient information to ensure that only the local maximum value at the edge is retained. In the application of the first-order differential operator, the horizontal and vertical directions are particularly selected to effectively extract edge features. Finally, the threshold is selected according to actual needs using an adaptive threshold method, and the processed image is edge connected to obtain a complete and accurate edge detection result, effectively suppressing false edges and noise edges, and improving the performance of edge detection to ensure positioning accuracy.

[0059] S5: locating the center of the target based on the target edge features to obtain the position of the small target.

[0060] When obtaining the coordinates of the target center point, the target is regarded as a two-dimensional object with uniform density, and then its center point position is determined by finding its center of gravity. In a two-dimensional image, the center of gravity of the target body is the average position of all the pixels contained, which can also be called the center of mass or centroid. The method of calculating the spatial moment is used to locate the center of gravity of the contour; in order to ensure the stability of the positioning result and make the center positioning coordinates more accurate, the Kalman filter method is used to further smooth and denoise and dynamically adjust the weights to ensure the accuracy of the center positioning coordinates.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A low-light small target image recognition method based on deep learning, characterized in that: The method comprises the following steps: S1: Collect low-illumination small target images and corresponding normal-illumination images, and annotate and preprocess the collected images to obtain a low-illumination small target dataset; S2: construct a multi-scale enhancement model, and use the low-illumination images and corresponding normal-illumination images in the low-illumination small target data set to train the multi-scale enhancement model to obtain a MSEM model, so as to achieve illumination enhancement of the low-illumination small target image; S3: constructing a small target semantic segmentation model, using the low-light small target image enhanced by the MSEM model and the label map in the low-light small target data set to train the small target semantic segmentation model, obtaining a DR-UNet model, and realizing segmentation and recognition of the enhanced image; S4: Perform contour detection on the segmented image to extract target edge features of the segmented image; S5: locating the center of the target based on the target edge features to obtain the position of the small target.

2. The method for low-light small target image recognition based on deep learning according to claim 1, characterized in that: In step S2, the multi-scale enhancement model includes a decomposition module and an enhancement module; The decomposition module includes three submodules: shallow feature extraction, activation layer sequence and final reconstruction layer; the shallow feature extraction submodule includes a convolution layer, which is used to perform preliminary feature extraction on the input low-illuminance image and the corresponding normal illumination image; the activation sequence layer submodule performs deep feature extraction on the preliminary extracted feature information, which is connected in sequence through five convolution blocks consisting of convolution layers and Leaky-ReLu activation functions; the final reconstruction layer submodule includes a convolution layer and a sigmoid function, wherein the convolution layer converts the feature map output by the activation sequence layer submodule from 64 channels back to 4 channels, and the sigmoid function maps the output to the interval of 0-1, and the feature map is divided into a reflection component and an illumination component; The enhancement module includes three submodules: feature extraction, multi-scale feature fusion and output generation; the feature extraction submodule includes a convolution layer and three deep feature extraction layers, wherein the convolution layer receives the reflection component and the illumination component as input and splices them along the channel dimension, so that the network can capture the basic texture and edge information in the input image and complete the preliminary extraction of feature information; the output of the convolution layer is used as the input of the first deep feature extraction layer, and is downsampled through a convolution operation with a step size of 2; the second deep feature extraction layer receives the output of the first deep feature extraction layer as input, and is downsampled again through a convolution operation with a step size of 2; the third deep feature extraction layer receives the output of the second deep feature extraction layer as input, and is downsampled through a third convolution operation with a step size of 2; the multi-scale feature fusion submodule restores the output of each deep feature extraction layer to the spatial resolution of the previous layer through an upsampling operation, and splices it with the output of the previous layer along the channel dimension to complete the fusion of features of different scales and levels; the output generation submodule outputs the fused feature map as an enhanced illumination component.

3. The method for low-light small target image recognition based on deep learning according to claim 2, characterized in that: In the step S2, the reflection component and the enhanced illumination component are reconstructed, and the specific method is: the reflection component and the enhanced illumination component are multiplied and fused element by element to generate an enhanced target image.

4. A low-light small target image recognition method based on deep learning according to any one of claims 1-3, characterized in that: In step S2, during the training of the multi-scale enhancement model, low-illumination and normal-illumination image pairs are randomly selected in each training batch; the loss is calculated by forward propagation of the model, and the optimizer is used to backpropagate the loss and update the model parameters; at the end of every 5 epochs, the model performance is evaluated and the best model is recorded.

5. A low-light small target image recognition method based on deep learning according to any one of claims 1-3, characterized in that: In step S3, the small object semantic segmentation model includes a backbone network DCSR and a feature fusion layer; The backbone network DCSR includes an input layer, a convolution layer and a layer composed of multiple basic building blocks; the input of the input layer is an enhanced low-light small target image and a label map in a low-light small target data set; the convolution layer performs preliminary feature extraction on the input image; the layer composed of multiple basic building blocks includes four layers, wherein layer1 includes 3 basic building blocks, layer2 includes 4 basic building blocks, layer3 includes 6 basic building blocks, and layer4 includes 3 basic building blocks and a void space pyramid pooling module; each basic building block is composed of two DC modules connected in sequence, and the DC module includes a convolution layer, a batch normalization layer and a Leaky-ReLu activation function connected in sequence, and a channel attention module and a spatial attention module are inserted in sequence between the batch normalization layer and the Leaky-ReLu activation function in the second DC module, thereby improving the network's ability to capture important information in the image; In the DC module, the convolution layer is used to extract image features, perform weighted combination of information from different channels, and output a feature map; The batch normalization layer normalizes each channel of the output feature map so that the output data has a stable distribution; the Leaky-ReLu activation function maps the normalized feature map to a new space to generate the final feature map; The atrous spatial pyramid pooling module processes the output feature information of layer3 through four atrous convolutional layers to obtain feature maps of four different scales. The global context feature map is then obtained through global average pooling. The feature maps of four different scales and the global context feature map are restored to the size of the original input feature map through upsampling, and the feature maps of four different scales and the global context feature map are spliced ​​in the channel dimension to obtain the spliced ​​feature map. The feature fusion layer adopts a dense connection mechanism. In each module of the overall network framework, the outputs of all previous modules are spliced ​​and input into the basic building blocks of the backbone network DCSR contained in the current layer, and then transmitted to the next layer.

6. The method for low-light small target image recognition based on deep learning according to claim 5, characterized in that: In step S3, the specific implementation method of the dense connection mechanism adopted by the feature fusion layer is as follows: when building the overall network framework of the small target semantic segmentation model, the input image is first sent to the backbone network DCSR, and the initial feature extraction module X is used to extract the image. 0_0 Convolution operation is performed to extract basic feature information from the image; then, the extracted preliminary features are sent to a series of deep feature extraction modules X 1_0 , X 2_0 , X 3_0 , X 4_0 The above five modules constitute the backbone network DCSR, which gradually mines the deep feature information in the image by downsampling the features and further extracting the features; 1_0 The output is upsampled with X 0_0 The outputs are concatenated in the channel dimension and fed into module X 0_1 , the module X 0_1 It contains a basic building block of the backbone network DCSR, which completes the fusion and output of feature information of different scales; other layers also complete the above operations in sequence, and after splicing with the output of each previous layer, they are jointly input into the basic building block of the backbone network DCSR contained in each layer, and continue to be transmitted to the next layer; finally, the dense connection of the entire network is completed, so that each layer in the network is connected to all previous layers to obtain richer feature information.

7. The method for low-light small target image recognition based on deep learning according to claim 6, characterized in that: In step S3, when training the small object semantic segmentation model, the loss function adopts a loss function combining Dice loss and Focalloss: Loss=βDice·DiceLoss+(1-βDice)·FocalLoss (1) Among them, βDice is a hyperparameter used to adjust the weight between Dice loss and Focal loss loss functions; The Dice loss function is calculated based on the Dice coefficient. The calculation formulas of the Dice coefficient and DiceLoss are as follows: Where X represents the predicted output image obtained by inputting the low-light small target image into the small target semantic segmentation model, Y represents the real label segmentation result, |X∩Y| represents the number of intersection elements between X and Y, and |X| and |Y| represent the number of elements in X and Y respectively; The formula of the FocalLoss loss function is as follows: FocalLoss=-a(1-p) λ log(p) (4) Among them, p is the probability that the small object semantic segmentation model predicts the correct category; α is a sample weight used to adjust the weights of easy-to-classify samples and difficult-to-classify samples, which is the inverse of the category frequency; λ represents the weight of difficult-to-classify samples.

8. The method for low-light small target image recognition based on deep learning according to claim 1, characterized in that: In step S4, the Canny operator is selected to perform contour detection on the segmented image, bilateral filtering is used to remove noise, the Sobel operator is used to calculate the gradient amplitude and gradient, the image is screened by non-maximum suppression interpolation method, and the threshold is selected by adaptive thresholding.

9. The method for low-light small target image recognition based on deep learning according to claim 1, characterized in that: In step S5, according to the target edge features, the contour center of gravity position is located by calculating the spatial moment, and the Kalman filter method is used for further smoothing and denoising and dynamic weight adjustment to ensure the accuracy of the center positioning coordinates.

Images