Dark weak target detection method based on deep learning

Through the U-Net cyclic residual convolutional neural network based on deep learning, the detection problem of low signal-to-noise ratio dark targets in complex backgrounds is solved, efficient and accurate target detection is achieved, and false alarm rate is reduced and detection is improved. speed.

CN118941781AInactive Publication Date: 2024-11-12INST OF OPTICS & ELECTRONICS CHINESE ACAD OF SCI

Patent Information

Application Number
CN202411420369.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-12
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The prior art is difficult to effectively detect dark targets with low signal-to-noise ratios, especially under complex backgrounds and noise interference, resulting in high false alarm rates and low detection rates.

Method used

Using a deep learning-based method, U-Net cyclic residual convolutional neural network is used to accumulate and remove background noise and random noise through multi-frame images, a training data set is constructed and a deep learning model is trained to achieve detection of dark weak targets.

Benefits of technology

Accurate detection is achieved in the number of unknown targets and complex backgrounds, with small calculation amounts and strong adaptability, and can quickly detect and draw the location of dark and weak targets, reducing false alarm rates and improving detection rates.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941781A_ABST
    Figure CN118941781A_ABST
Patent Text Reader

Abstract

The invention provides a dark target detection method based on deep learning, which belongs to the technical field of image processing, and comprises the following steps: obtaining a real dark target image; constructing a preprocessing algorithm, and carrying out preliminary preprocessing on the collected images, including accumulation of multiple frames of images, removal of background noise, removal of random noise and the like; constructing a deep learning model, and establishing a training data set and a verification data set by using the obtained images; training the deep learning model to obtain a trained deep learning model; and on the basis of the trained deep learning model, carrying out faint target detection, and representing the faint target in an original image by using a square frame according to the obtained position of the faint target. According to the method, automatic selection of target features is realized by using deep learning; the method can adapt to an original image, an accumulated image or a pre-processed image; compared with a traditional image processing method, the operation time is short, the influence of a false target is more easily avoided, and real-time target detection is facilitated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a dark target detection method based on deep learning. Background Art

[0002] For the detection system, when the distance between the detected target and the detector reaches more than ten or even dozens of kilometers, under the influence of external factors such as atmospheric disturbance, optical scattering and diffraction, the imaged target presents the characteristics of "weak" and "small". The "weak" mainly refers to the low signal-to-noise ratio of the target and the low contrast between the target and the background; the "small" mainly refers to the small number of pixels occupied by the target in the entire image. Weak and small targets in low signal-to-noise ratio images often appear in a variety of scenes including the sky, ground and sea surface, and are mixed in natural clutter and are difficult to distinguish, resulting in target detection facing the problem of high false alarm rate and low detection rate.

[0003] The detection of dim targets has the following difficulties: The target contrast is low, and there are few features that can be used for target detection. At long distances, the target signal strength received by the detector is limited, and the signal-to-noise ratio is very low. The target imaging area is small, and some have only a few pixels, so there is a lack of contour and shape features, and there is relatively little information available.

[0004] The background is complex and the target is often submerged in the background or noise. Long-distance targets are usually interfered by complex backgrounds, such as sky cloud background, ground background and sea background. The statistical characteristics of these backgrounds are different and often change continuously.

[0005] The mobility of the target is not easy to grasp, the motion trajectory of the detected target is uncertain, and sometimes it exhibits strong mobility. In addition, the tracking process may encounter interference from noise and similar targets.

[0006] In the prior art, there are many methods for extracting dim targets. For example, the accumulation method is used to superimpose multiple frames of images to enhance target information. This method has an upper limit in improving the signal-to-noise ratio and is not suitable for moving targets. There are also literatures that use the differences and singularities between the target and the background, and the noise in the structural features such as local intensity, gradient and divergence of the image to detect dim targets, but this method has a large amount of calculation and is not conducive to real-time detection and tracking. In short, the existing methods have not yet completely solved the impact of background clutter and strong noise, which has created obstacles to the practical application of dim target detection in low signal-to-noise ratio images. Summary of the invention

[0007] In view of the above technical problems, the present invention provides a dark target detection method based on deep learning, a dark target detection method based on deep learning, the method comprising the following steps: Step 1: Obtain real dim target images and build a training data set; Step 2: construct a preprocessing algorithm to preliminarily preprocess the dim target image collected in step 1. The preprocessing algorithm includes: multi-frame image accumulation, background noise removal, and random noise removal; construct a verification data set; Step 3, build a deep learning model, and train the deep learning model using the training data set and the verification data set obtained in step 1 and step 2 to obtain a trained deep learning model; the deep learning model is a cyclic residual convolutional neural network based on U-Net, and the cyclic residual convolutional neural network includes a multi-layer cyclic residual convolutional neural network block, and the cyclic residual convolutional neural network block consists of a standard convolutional layer, a cyclic convolutional layer and a residual block; the standard convolutional layer convolves the input data and obtains the initial feature map and state through the activation function; in the first time step, the cyclic convolutional layer first uses a set of convolution kernels to perform a convolution operation on the input data to extract spatial features; in subsequent time steps, the state of the previous time step is sent to the convolution kernel together with the input data of the current time step, and the convolution operation is performed, the state is updated, and the feature map of the current time step is obtained through the activation function; the above process is repeated until the data of all time steps are processed; the residual block uses a skip connection to allow the input to skip certain layers and directly add to the output of the current layer; Step 4: Based on the above trained deep learning model, the position of the dim target is obtained by performing dim target detection; Step 5: Use the obtained position of the dim target to represent the dim target with a box in the real dim target image.

[0008] The beneficial effects of the present invention compared with the prior art are: (1) The present invention can detect multi-target images under the premise of unknown target number; (2) Even if the intensity of the background and clutter is greater than the target intensity, it will not affect the correct detection of the target; (3) The amount of calculation is small, and the target detection speed is not affected by the number of targets; (4) Strong adaptability: changes in the brightness of the external environment have little effect on the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is the original image of the dim target collected in the embodiment; Figure 2 is a three-dimensional image of an original image of a dim target in the embodiment; Figure 3 is a target image after traditional image processing in the embodiment; Figure 4 is the label of the target in the training set in the embodiment; Figure 5is a structural diagram of a cyclic residual convolutional neural network model based on U-Net in an embodiment; Figure 6 is a schematic diagram of the result of target detection by a deep learning model in an embodiment; Figure 7 It is a schematic diagram of the target position drawn in the original image based on the deep learning target detection result in the embodiment. DETAILED DESCRIPTION

[0010] The technical solution provided by the present invention is described in detail below in conjunction with specific embodiments. The following embodiments are limited to explaining the present invention, but are not intended to be limiting of the present invention. The protection scope of the present invention should include the entire contents of the claims. All similar implementations that achieve the same effects by substantially the same methods fall within the protection scope of the present invention. Through the following embodiments, those skilled in the art can realize the entire contents of the claims of the present invention.

[0011] This embodiment takes an image collected in a dim target detection experiment as an example. Figure 1 As shown in Figure 1, it can be seen that due to the strong background light, the target is barely visible. The three-dimensional image of the original image is as follows: Figure 2 As shown, the target is almost submerged in the noise, so the target detection is hindered. The specific implementation steps of the dim target detection method based on deep learning of the present invention are as follows.

[0012] Step 1: Obtain real dim target images and build a training data set; This step takes the real faint target image obtained in a certain experiment as an example, including star images of different magnitudes, such as Figure 1 shown.

[0013] Step 2: construct a preprocessing algorithm to preliminarily preprocess the dim target image collected in step 1. The preprocessing algorithm includes: multi-frame image accumulation, background noise removal, and random noise removal; construct a verification data set; Since the camera captures images at a high frequency and the target moves slowly, the direct stacking and averaging method is used to accumulate images within n seconds, n∈[0,4].

[0014] The steps of the background noise removal algorithm are: Step 2.1: Use median filtering on the original image to remove background noise. , where x is the original image, y is the filtered image, m is the current pixel position, and w is the filter window size; is the median value; Step 2.2: Threshold segmentation and connected domain analysis to eliminate large-area background noise; Step 2.3: Use morphological opening operation to eliminate small particle noise. The morphological opening operation is: ; Where A is the image set, B is the structure element set, represents the corrosion of A to B, It represents the expansion of A to B; Step 2.4: Threshold segmentation to obtain candidate target seeds; Step 2.5: Use the watershed algorithm to repair the target.

[0015] The input of the watershed algorithm is a grayscale image I; the output is a labeled image L containing the watershed line. The specific steps are as follows: Step 2.5.1, record the candidate target seeds as sequence P; Step 2.5.2, initialize the labeled image L, and mark each pixel as -1; Step 2.5.3, for each pixel p in the sequence P, if the pixel p is a local minimum point, then: assign a new label to the pixel p; add the pixel p to the flood filling queue Q; Step 2.5.4: For each pixel q in queue Q, if there are marked pixels in the neighbors of pixel q, mark pixel q with the same mark as the neighbors; if there are multiple neighbors with different marks around q, mark pixel q as a watershed line and remove pixel q from queue Q; otherwise, continue with step 2.5.3.

[0016] Take the original dim target image in step 1 as an example, Figure 1 As shown in Figure 2, the target image obtained by the image enhancement algorithm is as follows: Figure 3 As shown, mark the target position as a label, such as Figure 4 shown, and Figure 4 Also as Figure 1 Similarly, 21 accumulated images and 105 pre-processed images are obtained.

[0017] Step 3: Build a deep learning model, and use the training data set and verification data set obtained in steps 1 and 2 to train the deep learning model to obtain a trained deep learning model.

[0018] Using the 105 original images obtained in step 1, the 21 accumulated images obtained in step 2, and the 105 pre-processed images, and dividing them into a training data set and a test data set, a cyclic residual convolutional neural network model based on U-Net is constructed in this embodiment, and 210 images and their labels are used as a training data set, and 21 images and their labels are used as a verification data set. Then, according to the settings in Table 1 below, training is started, and finally a cyclic residual convolutional neural network model is obtained. The model structure diagram is shown in FIG. Figure 5As shown in the figure. Among them, conv means using a 3×3 convolution kernel to perform a convolution operation on the image; Relu represents an activation function; BatchNorm is normalization; add superimposes the semantic information extracted multiple times, that is, the feature maps are added, and the number of channels remains unchanged; MaxPool is the maximum pooling, such as using a 2×2 kernel to perform a pooling operation on the image, and selecting the maximum value to output to the next layer; convTranspose is a deconvolution, which first expands the size of the input image by padding 0 according to a certain ratio, and then rotates the convolution kernel to perform forward convolution; Concat is the full name of the concatenate layer, which is used to fuse the features extracted by multiple convolutional feature extraction frameworks. The features describing the image itself are increased, but the information under each feature is not increased. The cyclic residual convolutional neural network includes a multi-layer cyclic residual convolutional neural network block, which is composed of a standard convolutional layer, a cyclic convolutional layer and a residual block; the standard convolutional layer convolves the input data and obtains the initial feature map and state through the activation function; in the first time step, the cyclic convolutional layer first uses a set of convolution kernels to perform a convolution operation on the input data to extract spatial features; in subsequent time steps, the state of the previous time step is sent to the convolution kernel together with the input data of the current time step, and the convolution operation is performed to update the state, and the feature map of the current time step is obtained through the activation function; the above process is repeated until the data of all time steps are processed; the residual block uses a skip connection to allow the input to skip some layers and directly add to the output of the current layer.

[0019] Table 1 Settings of the U-Net-based cyclic residual convolutional neural network

[0020] In this embodiment, a cyclic residual convolutional neural network model based on U-Net is used. is the input of the l-th layer of the recurrent residual convolutional neural network block, is the output of the lth recurrent convolutional layer, and its expression is shown in formula (1): (1) in, represents time t, and Represent the input of the standard convolution layer and the recurrent convolution layer respectively, and denote the weights of the standard convolutional layer and the recurrent convolutional layer, respectively. is the bias and T is the transpose. The output of the recurrent convolutional layer is then fed to the ReLU activation function To increase the nonlinear ability of the model and help the network learn more complex features, see formula (2): (2) Max means taking the maximum value. It represents the output of the l-th layer of circular convolutional unit, and then the final output is obtained through the residual unit, which is also the input of the next layer, denoted as : (3) In addition, the implementation of the residual path in this embodiment uses a separate convolutional layer to adjust the number of feature maps. Finally, after the original image passes through the cyclic residual convolutional network, the predicted image Y is obtained. The Dice Loss function, see formula (4), is used to compare its error with the label X, and the weights in the update network are adjusted with this.

[0021] (4) Indicates taking the absolute value, In this embodiment, the Lookahead optimizer is used, which includes a slow weight and fast weight ,in Updated by Adam, see formulas (5)-(7): , (5) , (6) (7) in, is the gradient at time t, which is approximately the loss function The gradient of , They are the first-order and second-order moment estimates of the gradient, , Respectively , The correction, are the exponential decay rates of the first-order moment and the second-order moment, respectively. is the learning rate, is a constant. Each pair of fast weights After k training iterations, in the weight space Use linear interpolation in the direction to update the slow weights (See formula (8)), and then the updated slow weight Assign value to fast weight Continue training, and after training, slow weight As the final model training output weight.

[0022] (8) Step 4: Based on the above trained deep learning model, perform dim target detection.

[0023] This embodiment takes another dim target image as an example, and introduces the cyclic residual convolutional neural network model trained in step 2 to obtain the prediction result. The white area in the prediction result is the detected dim target, such as Figure 6 shown.

[0024] Step 5: Use the target position obtained above to represent the dim target with a box in the real dim target image.

[0025] According to the weak target detected in step 3, mark the target with a box in the original image, such as Figure 7 shown.

[0026] The present invention does not elaborate on some of the well-known technologies that belong to those skilled in the art.

Claims

1. A dark target detection method based on deep learning, characterized in that: The method comprises the following steps: Step 1: Obtain real dim target images and build a training data set; Step 2: construct a preprocessing algorithm to preliminarily preprocess the dim target image collected in step 1. The preprocessing algorithm includes: multi-frame image accumulation, background noise removal, and random noise removal; construct a verification data set; Step 3, build a deep learning model, use the training data set obtained in step 1 and the verification data set in step 2 to train the deep learning model, and obtain the trained deep learning model; the deep learning model is a cyclic residual convolutional neural network based on U-Net, the cyclic residual convolutional neural network includes a multi-layer cyclic residual convolutional neural network block, and the cyclic residual convolutional neural network block consists of a standard convolutional layer, a cyclic convolutional layer and a residual block; the standard convolutional layer convolves the input data and obtains the initial feature map and state through the activation function; in the first time step, the cyclic convolutional layer first uses a set of convolution kernels to perform a convolution operation on the input data to extract spatial features; in subsequent time steps, the state of the previous time step is sent to the convolution kernel together with the input data of the current time step, and the convolution operation is performed, the state is updated, and the feature map of the current time step is obtained through the activation function; the data of all time steps are processed; the residual block uses a skip connection to allow the input to skip certain layers and directly add to the output of the current layer; Step 4: Based on the above trained deep learning model, the position of the dim target is obtained by performing dim target detection; Step 5: Use the obtained position of the dim target to represent the dim target with a box in the real dim target image.

2. The method for detecting dim targets based on deep learning according to claim 1, characterized in that: In step 3, make is the input of the l-th layer of the recurrent residual convolutional neural network block, is the output of the recurrent convolutional layer in the lth layer, and its expression is shown in formula (1): (1) in, represents time t, and Represent the input of the standard convolution layer and the recurrent convolution layer respectively, and denote the weights of the standard convolutional layer and the recurrent convolutional layer, respectively. is the bias, T represents the transpose, and then the output of the recurrent convolutional layer is fed into the ReLU activation function , see formula (2): (2) It represents the output of the circular convolutional layer in the lth layer after passing through the activation function, and then the final output is obtained through the residual block, which is also the input of the next layer of circular residual convolutional neural network block, denoted as , (3) The implementation of the residual path uses a separate convolutional layer to adjust the number of feature maps. Finally, the original image passes through the cyclic residual convolutional neural network model to obtain the predicted image Y. The Dice Loss function, see formula (4), is used to compare its error with the label X, and the weights in the update network are adjusted with this. (4) Indicates taking the absolute value, Indicates taking the intersection, using the Lookahead optimizer, which includes slow weights and fast weight , where the fast weight Updated by the Adam optimization algorithm, see formula (5)-formula (7): , (5) , (6) (7) in, is the fast weight at time t, is the gradient at time t, and is the loss function for the fast weights The gradient of , They are the first-order moment and second-order moment estimates of the gradient, , Respectively , The correction, are the exponential decay rates of the first-order moment and the second-order moment, respectively. is the learning rate, is a constant, each pair of fast weights After k training iterations, in the weight space Use linear interpolation in the direction to update the slow weights , see formula (8), and then update the slow weight Assign value to fast weight Continue training, and after training, slow weight As the final model training output weight; (8)。 3. The method for detecting dim targets based on deep learning according to claim 1, characterized in that: The specific algorithm for accumulating multiple frames of images is as follows: if the camera acquisition frequency reaches above 100 Hz, the n-frame image is accumulated by directly stacking and taking the average value, where n∈[0,400].

4. The method for detecting dim targets based on deep learning according to claim 1, characterized in that: The steps to remove background noise are: Step 2.1: Use median filtering on the original image to remove background noise. The specific formula is: , where x is the original image, y is the filtered image, m is the current pixel position, and w is the filter window size; is the median value; Step 2.2: Threshold segmentation and connected domain analysis to eliminate large-area background noise; Step 2.3: Use morphological opening operation to eliminate small particle noise. The morphological opening operation is: ; Where A is the image set, B is the structure element set, represents the corrosion of A to B, It represents the expansion of A to B; Step 2.4: Threshold segmentation to obtain candidate target seeds; Step 2.5: Use the watershed algorithm to repair the target.

5. The method for detecting dim targets based on deep learning according to claim 4, characterized in that: The input of the watershed algorithm is a grayscale image I; the output is a labeled image L containing the watershed line. The specific steps are as follows: Step 2.5.1, record the candidate target seeds as sequence P; Step 2.5.2, initialize the labeled image L, and mark each pixel as -1; Step 2.5.3, for each pixel p in the sequence P, if the pixel p is a local minimum point, then: assign a new label to the pixel p; and add the pixel p to the flood filling queue Q; Step 2.5.4: For each pixel q in queue Q, if there are marked pixels in the neighbors of pixel q, mark pixel q with the same mark as the neighbors; if there are multiple neighbors with different marks around q, mark pixel q as a watershed line and remove pixel q from queue Q; otherwise, continue with step 2.5.3.

Citation Information

Patent Citations

  • Non-vision-field target signal identification method based on deep learning

    CN113919398A

  • Dark and weak target detection method based on multistage noise suppression

    CN115187516A

  • Three-dimensional image blurring method and device, equipment and storage medium

    CN117152398A

Cited By

  • Deep learning-based celestial body faint target feature extraction method

    CN120355933A

  • Telescope detection dark space target image data set generation method

    CN121415188A