Target detection system and detection method thereof
The spatial targets are subjected to double threshold verification and cross-blocking processing through convolutional neural networks, which solves the problem of spatial target detection with low signal-to-noise ratio, and effectively recognizes and segments of weak targets, improving detection effect and image clarity.
Patent Information
- Application Number
- CN202210849926.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-20
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-07-20
AI Technical Summary
The prior art is difficult to effectively detect spatial targets with low signal-to-noise ratio, especially small-sized spatial fragments, and their imaging has no detailed profile characteristics, resulting in the traditional method's low signal-to-noise ratio object detection performance, which is difficult to distinguish from image background and noise.
Convolutional neural network is used to perform double threshold tests on the target rough contour features and grayscale values. Through cross-blocking processing and semantic segmentation network structure, the original spatial image is segmented and integrated, and combined with upsampling and pooling operations are achieved. Effective detection of low signal-to-noise ratio dark targets.
The recognition and detection effect of dark and weak targets is improved, the image is clear after segmentation, which is convenient for subsequent tracking, can overcome interference such as clouds and vignetting, adapt to different environments, and achieve target segmentation with a signal-to-noise ratio below 5.
Smart Images

Figure CN115223154B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image detection, and in particular to a target detection system and a detection method thereof. Background Art
[0002] For space targets, using ground-based telescopes to observe them is a very mature technical means. With the increasing demand for satellite data in various industries, more and more satellites are launched into Earth orbit, which also brings a large number of satellite debris, leading to the deterioration of the space environment. In order to ensure the safety of the space environment, orbit tracking of satellites and detection and avoidance of space debris have become particularly important.
[0003] For small-sized space debris, there is no detailed contour feature in the imaging; and due to the small amount of reflected sunlight flux, the detection signal-to-noise ratio of the target is very low, which brings great difficulty to image detection. For space targets without detailed contour features, traditional target detection methods generally use grayscale value as the detection threshold. By setting the detection threshold, the targets with grayscale values higher than the set threshold in the original image are extracted, which will reduce the detection performance of dim targets with low signal-to-noise ratio in the image and not significantly different from the image background and noise. If the target grayscale value is higher than the detection threshold, it is judged as a target. However, the grayscale value of low signal-to-noise ratio targets is not significantly different from the skylight background, making it difficult to achieve high-efficiency detection.
[0004] In response to the above needs, there are currently many relevant solutions at home and abroad.
[0005] Patent CN107316004A proposes a method for spatial target recognition based on deep learning. First, a 9-layer deep convolutional network model is constructed. Then, based on the network, the optimal data augmentation method is found, and the data obtained by several better augmentation methods are combined. The optimal combined data is used in the model training and testing process to complete the spatial target recognition. This method is simple and feasible, but its recognition ability is limited. When recognizing spatial targets, the input data is an image with obvious features and large size, and the recognition effect on small targets is not good.
[0006] Patent CN111191583A proposes a spatial target recognition system and method based on convolutional neural network. This patent includes model training and target recognition. Model training includes inputting positive and negative samples, extracting features using sparse convolutional neural network, training classifiers, and forming a learning dictionary. Images can be classified by training the network, and images containing spatial targets and images without targets can be classified. Although this method can preliminarily determine whether the image contains spatial targets, it cannot further determine the number of targets and the specific location of the targets at the pixel level.
[0007] Patent CN109188462A proposes a method and device for detecting space targets under a complex starry sky background. This patent obtains measurement requirements, cooperative target parameters, star sensor working data, and designs a relative angle measurement algorithm to obtain space target detection results. This method can effectively bring the advantages of high precision and high update rate of star sensors into play in the field of space detection by applying the performance advantages of star sensors to space target detection, but it cannot accurately detect unknown targets for which there is no prior knowledge. Summary of the invention
[0008] In view of the above problems, the purpose of the present invention is to propose a target detection system and a detection method thereof, which use a convolutional neural network to perform double threshold tests on the target's coarse contour features and grayscale value features, thereby achieving effective detection of low signal-to-noise ratio dim spatial targets whose grayscale values are not obviously different from the image background and noise, and achieving segmentation of targets with a signal-to-noise ratio lower than 5.
[0009] To achieve the above object, the present invention adopts the following specific technical solutions:
[0010] The present invention provides a target detection method, comprising the following steps:
[0011] S1, cross-blocking the original spatial image, adjusting the resolution of the original spatial image to a first set resolution and a second set resolution respectively to obtain a first group of spatial images and a second group of spatial images containing different numbers of image blocks;
[0012] S2, inputting the image blocks in the first group of spatial images and the second group of spatial images into the semantic segmentation network structure for segmentation processing one by one, so as to separate the target and the background;
[0013] S3, integrating the segmented image blocks in the first group of spatial images and the second group of spatial images respectively to obtain a first restored image and a second restored image;
[0014] S4, performing edge cutting on the first restored image and the second restored image to obtain two images with the same resolution;
[0015] S5. Perform mask addition on two images of the same resolution to obtain a final image.
[0016] Preferably, step S2 includes the following sub-steps:
[0017] S21, performing upsampling operations on image blocks in the first group of spatial images and the second group of spatial images one by one;
[0018] S22, after the upsampling operation is completed, performing a pooling operation on the segmented image blocks;
[0019] S23. After the pooling operation is completed, the pooled image block is input into the activation function of the output layer to obtain an image block in which the target and the background are separated.
[0020] Preferably, the first set resolution is 2050*2050, and the process of adjusting the resolution of the original spatial image to the first set resolution is:
[0021] The original spatial image is zero-filled in both horizontal and vertical directions and then divided into 100 small-size image blocks with a resolution of 205*205, thereby obtaining the first set of spatial images.
[0022] Preferably, the second set resolution is 2255*2255, and the process of adjusting the resolution of the original spatial image to the second set resolution is:
[0023] The boundaries of the original spatial image are padded with zeros in both horizontal and vertical directions, and then divided into 121 small-size image blocks with a resolution of 205*205, thereby obtaining a second set of spatial images.
[0024] Preferably, the semantic segmentation network structure consists of 12 convolutional layers and 3 pooling layers, and each convolutional layer is followed by an activation function and a BN layer;
[0025] Except for the sigmoid function set after the last convolution layer, the remaining activation functions are linear rectification functions.
[0026] Preferably, the first group of spatial images is integrated to obtain a first restored image, and the resolution of the first restored image is 2050*2050;
[0027] The second group of spatial images are integrated to obtain a second restored image, and the resolution of the second restored image is 2255*2255.
[0028] Preferably, the simulated image training method of the semantic segmentation network in step S2 comprises the following steps:
[0029] Input the image block with a resolution of 205*205 into the semantic segmentation network;
[0030] The image block f(x,y) is preprocessed by image standardization. The image standardization formula is as follows:
[0031]
[0032] Among them, std(f(x, y)) is the standard deviation calculation function, min(f(x, y)) is the minimum value calculation function;
[0033] During the training process of the semantic segmentation network structure, the loss function is used to calculate the loss of each pixel in the image block to complete the pixel-level classification;
[0034] The loss function formula is as follows:
[0035]
[0036] Where N is the number of pixels, y and p are the label and network output respectively;
[0037] After 100 rounds of training, the simulation training of the semantic segmentation network is completed.
[0038] The present invention also provides a target detection system, comprising: an image segmentation module, an image division module, an image integration module, an image cutting module and an image addition module;
[0039] The image blocking module is used to perform cross-blocking processing on the original spatial image, and adjust the resolution of the original spatial image to a first set resolution and a second set resolution respectively to obtain a first group of spatial images and a second group of spatial images containing different numbers of image blocks;
[0040] The image segmentation module is used to input the image blocks in the first group of spatial images and the second group of spatial images into the semantic segmentation network structure for segmentation processing one by one, so as to separate the target from the background;
[0041] The image integration module is used to integrate the segmented image blocks in the first group of spatial images and the second group of spatial images respectively to obtain a first restored image and a second restored image;
[0042] The image cutting module is used to perform edge cutting on the first restored image and the second restored image to obtain two images with the same resolution;
[0043] The image addition module is used to perform mask addition on two images of the same resolution to obtain the final image.
[0044] Preferably, the image segmentation module comprises: an upsampling unit, a pooling unit and an image block separation unit;
[0045] The up-sampling unit is used for performing up-sampling operation on the image blocks in the first group of spatial images and the second group of spatial images one by one;
[0046] The pooling unit performs a pooling operation on the segmented image blocks;
[0047] The image block separation unit is used to input the pooled image blocks into the activation function of the output layer to obtain image blocks with separated targets and backgrounds.
[0048] Preferably, the image segmentation module includes an image zero-padding processing unit;
[0049] The zero-filling processing unit is used to divide the original spatial image into 100 small-size image blocks with a resolution of 205*205, so as to obtain a first set of spatial images;
[0050] The zero-filling processing unit is also used to divide the original spatial image into 121 small-sized image blocks with a resolution of 205*205, so as to obtain a second set of spatial images.
[0051] Compared with the traditional space target detection method, the present invention has the following advantages:
[0052] (1) Better recognition and detection of dim targets. Different from the traditional energy-based detection method, the deep learning method of the present invention can better combine the target shape contour and grayscale features, thereby achieving good segmentation and detection effects for dim targets.
[0053] (2) The segmented image is clear and easy to track. The segmented image after forward propagation using the deep learning network is free of various types of clutter, with only the spatial target and the background. The processed image target is clear and distinct, which is conducive to the subsequent target tracker to further track the spatial target.
[0054] (3) Little interference from clouds, vignetting and other detector equipment. The present invention trains a suitable neural network to learn features that can overcome interference. By adding data with interference such as clouds and vignetting to the training set, the network can be more adaptable to different environments without the need to write additional denoising programs. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 It is a flowchart of a target detection method provided according to an embodiment of the present invention.
[0056] Figure 2 It is a schematic diagram of the overall steps of spatial target detection according to the target detection method provided by an embodiment of the present invention.
[0057] Figure 3 Schematic diagram of image cross-block processing of a target detection method provided according to an embodiment of the present invention.
[0058] Figure 4 Schematic diagram of the semantic segmentation network structure of the target detection method provided according to an embodiment of the present invention. DETAILED DESCRIPTION
[0059] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same modules are represented by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, the detailed description thereof will not be repeated.
[0060] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.
[0061] Figure 1 The target detection method provided according to an embodiment of the present invention is shown.
[0062] Figure 2 It is a schematic diagram of the overall steps of spatial target detection according to the target detection method provided by an embodiment of the present invention.
[0063] like Figure 1 and Figure 2 As shown, the target detection method provided by the embodiment of the present invention includes the following steps:
[0064] S1. Cross-blocking the original spatial images respectively, and adjusting the resolution of the original spatial images to a first set resolution and a second set resolution respectively to obtain a first group of spatial images and a second group of spatial images containing different numbers of image blocks.
[0065] Step S1 is to construct a suitable size for subsequent input into the segmentation network and eliminate the detection influence of the target at the block boundary.
[0066] The preset resolution of the original space target image is 2048*2048.
[0067] Figure 3 Schematic diagram of image cross-block processing of a target detection method provided according to an embodiment of the present invention.
[0068] like Figure 3 As shown, the process of adjusting the resolution of the original spatial image to the first set resolution is:
[0069] The scale of the original spatial image is modified to 2050*2050, that is, the original spatial image is padded with zeros in both the horizontal and vertical directions, and then the image is divided into 100 small-size image blocks with a resolution of 205*205, that is, the first group of spatial images, and the first group of spatial images is input into the semantic segmentation network in step S2 for segmentation operation.
[0070] The process of adjusting the resolution of the original spatial image to the second set resolution is:
[0071] The scale of the original spatial image is modified to 2255*2255, and zero padding is performed on the boundaries of the original spatial image. The image is then divided into 121 small-size image blocks with a resolution of 205*205, i.e., the second group of spatial images. The second group of spatial images is input into the semantic segmentation network in step S2 for segmentation operation.
[0072] The boundary of the first set of images is exactly the midline of the second set of images, thereby eliminating the influence of the target falling on the boundary of the image block.
[0073] S2. Input the image blocks in the first group of spatial images and the second group of spatial images into the semantic segmentation network for segmentation processing one by one to separate the target and the background.
[0074] Figure 4 Schematic diagram of the semantic segmentation network structure of the target detection method provided according to an embodiment of the present invention.
[0075] like Figure 4 As shown in the figure, the semantic segmentation network structure mainly consists of 12 convolutional layers and 3 pooling layers, and each convolutional layer is followed by an activation function and a BN (batch normalization) layer. Except for the sigmoid (S-shaped growth curve) function set after the last convolutional layer, the remaining activation functions are all ReLU (linear rectification function), and the image is upsampled by bilinear interpolation at the beginning of the network to increase the size of the image block and the characteristics of the dim target.
[0076] S21 . Perform upsampling operations on image blocks in the first group of spatial images and the second group of spatial images one by one.
[0077] The upsampling operation provided by the present invention is specifically processed by bilinear interpolation, and the specific formula of bilinear interpolation is as follows:
[0078]
[0079] Where Q 11 , Q 12 , Q 21 , Q 22 They are four adjacent pixel points, x1, x2, y1, y2 are the corresponding horizontal and vertical coordinate values, and x and y are the coordinates of the point to be interpolated.
[0080] Bilinear interpolation can reduce the number of parameters while ensuring the accuracy of the segmentation results.
[0081] The interpolation is to linearly use the pixel values of the nearest neighbors to increase the image resolution. Convolution is an important structure for processing feature maps, and its specific content is as follows:
[0082]
[0083] Among them, f(x,y) is the feature map to be convolved, w is the size (2a+1,2b+1) is the convolution kernel, and s and t are the convolution kernels at different positions.
[0084] By iteratively training the parameters of the convolution kernel, the convolution parameters that can extract target features are continuously optimized.
[0085] S22. After the upsampling operation is completed, a pooling operation is performed on the segmented image blocks.
[0086] The pooling operations used in the present invention are all maximum pooling, that is, the maximum value of the reserved area is used as the lower layer feature input, and the pooling step size is 2, which ensures that the resolution of the feature map is reduced by half after each pooling. Pooling, as an important operation in convolutional neural networks, can cooperate with convolution to extract image features and ensure feature invariance. And pooling can simplify network complexity and reduce the amount of calculation. Further, pooling can prevent overfitting of the network to a certain extent.
[0087] S23. After the pooling operation is completed, the pooled image block is input into the activation function of the output layer to obtain an image block in which the target and the background are separated.
[0088] The present invention uses both sigmoid (S-shaped growth curve) and ReLU (linear rectification function) activation functions, and their formulas are respectively
[0089]
[0090] The activation function in the network can increase the nonlinearity of the network and optimize the fitting effect of the network. The activation function of the output layer is used for pixel classification, which can better separate the target from the background.
[0091] The present invention also provides a simulation image training method for a semantic segmentation network, which specifically comprises the following steps:
[0092] The dataset used for training is the constructed simulation image training set data, with an image block resolution of 205*205 and an image channel number of 1.
[0093] Image standardization is used to preprocess the image blocks. The image standardization formula is as follows:
[0094]
[0095] Among them, std(f(x, y)) is the standard deviation calculation function, min(f(x, y)) is the minimum value calculation function;
[0096] The optimizer selected during the training of the semantic segmentation network structure is SGD (stochastic gradient descent), and the initial learning rate is 1×10 -4 , the learning rate decays to 5×10 -7 , a more accurate network weight update is achieved through a decaying learning rate. The loss function is cross entropy, and the cross entropy loss function is shown in formula (5):
[0097]
[0098] Among them, N is the number of pixels, y and p are the label and network output respectively.
[0099] This loss function can calculate the loss for each pixel in the image block and complete pixel-level classification.
[0100] Due to the large image resolution of the input semantic segmentation network and the limitation of GPU (graphic processing unit) video memory, the batch size of image blocks fed into the semantic segmentation network is 1, and the training is carried out for 100 rounds.
[0101] S3. Integrate the segmented image blocks in the first group of spatial images and the second group of spatial images respectively to obtain a first restored image and a second restored image.
[0102] The first group of spatial images are integrated to obtain a first restored image, and the resolution of the first restored image is 2050*2050.
[0103] The second group of spatial images are integrated to obtain a second restored image, and the resolution of the second restored image is 2255*2255.
[0104] S4. After performing edge cutting on the first restored image and the second restored image, two images with the same resolution are obtained.
[0105] The resolution of two images with the same resolution is 2048*2048.
[0106] S5. Perform mask addition on two images of the same resolution to obtain a final image.
[0107] The present invention also provides a target detection system, comprising: an image segmentation module, an image division module, an image integration module, an image cutting module and an image addition module;
[0108] The image blocking module is used to perform cross-blocking processing on the original spatial image, and adjust the resolution of the original spatial image to a first set resolution and a second set resolution respectively to obtain a first group of spatial images and a second group of spatial images containing different numbers of image blocks;
[0109] The image segmentation module is used to input the image blocks in the first group of spatial images and the second group of spatial images into the semantic segmentation network structure for segmentation processing one by one, so as to separate the target from the background;
[0110] The image segmentation module includes: an upsampling unit, a pooling unit and an image block separation unit;
[0111] The up-sampling unit is used for performing up-sampling operation on the image blocks in the first group of spatial images and the second group of spatial images one by one;
[0112] The pooling unit performs a pooling operation on the segmented image blocks;
[0113] The image block separation unit is used to input the pooled image blocks into the activation function of the output layer to obtain image blocks with separated targets and backgrounds.
[0114] The image integration module is used to integrate the segmented image blocks in the first group of spatial images and the second group of spatial images respectively to obtain a first restored image and a second restored image;
[0115] The image cutting module is used to perform edge cutting on the first restored image and the second restored image to obtain two images with the same resolution;
[0116] The image addition module is used to perform mask addition on two images of the same resolution to obtain the final image.
[0117] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
[0118] The above specific implementations of the present invention do not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A target detection method, characterized in that: The following steps are involved: S1. Performing cross-block processing on the original spatial image, adjusting the resolution of the original spatial image to a first set resolution and a second set resolution respectively, to obtain a first group of spatial images and a second group of spatial images containing different numbers of image blocks, and the boundary of the first group of spatial images is the midline of the second group of spatial images; The first set resolution is 2050*2050, and the process of adjusting the resolution of the original spatial image to the first set resolution is: The original spatial image is zero-filled in both horizontal and vertical directions, and then divided into 100 small-size image blocks with a resolution of 205*205, so as to obtain the first set of spatial images; The second set resolution is 2255*2255, and the process of adjusting the resolution of the original space image to the second set resolution is: Performing zero padding processing on the boundaries of the original spatial image in both horizontal and vertical directions, and then dividing the image into 121 small-size image blocks with a resolution of 205*205, so as to obtain the second set of spatial images; S2, performing bilinear interpolation upsampling operations on the image blocks in the first group of spatial images and the second group of spatial images one by one, and then inputting them into the semantic segmentation network structure for segmentation processing one by one, and then performing pooling operations on the segmented image blocks; after the pooling operation is completed, inputting the pooled image blocks into the activation function of the output layer to obtain image blocks with separated targets and backgrounds; wherein the semantic segmentation network structure is composed of 12 convolutional layers and 3 pooling layers, and each convolutional layer is provided with an activation function and a BN layer; Except for the sigmoid function set after the last convolution layer, the other activation functions are linear rectification functions; S3, integrating the segmented image blocks in the first group of spatial images and the second group of spatial images respectively to obtain a first restored image and a second restored image; S4, performing edge cutting on the first restored image and the second restored image to obtain two images with the same resolution; S5. Perform mask addition on the two images with the same resolution to obtain a final image.
2. The target detection method according to claim 1, characterized in that: The first group of spatial images is integrated to obtain a first restored image, and the resolution of the first restored image is 2050*2050; The second group of spatial images are integrated to obtain a second restored image, and the resolution of the second restored image is 2255*2255.
3. The target detection method according to claim 1, characterized in that: The simulation image training method of the semantic segmentation network in step S2 comprises the following steps: Input the image block with a resolution of 205*205 into the semantic segmentation network; The image block f(x, y) is preprocessed by image standardization, and the image standardization formula is as follows: Among them, std(f(x, y)) is the standard deviation calculation function, min(f(x, y)) is the minimum value calculation function; During the training process of the semantic segmentation network structure, a loss function is used to calculate the loss of each pixel in the image block to complete pixel-level classification; The loss function formula is as follows: in, N is the number of pixels, y and p are labels and network outputs respectively; After 100 rounds of training, the simulation training of the semantic segmentation network is completed.
4. A target detection system, characterized in that: include: Image block module, image segmentation module, image integration module, image cutting module and image addition module; The image blocking module is used to perform cross-blocking processing on the original spatial image, and obtain a first group of spatial images and a second group of spatial images containing different numbers of image blocks after adjusting the resolution of the original spatial image to a first set resolution and a second set resolution, respectively, and the boundary of the first group of spatial images is the midline of the second group of spatial images; The first set resolution is 2050*2050, and the process of adjusting the resolution of the original spatial image to the first set resolution is: The original spatial image is zero-filled in both horizontal and vertical directions, and then divided into 100 small-size image blocks with a resolution of 205*205, so as to obtain the first set of spatial images; The second set resolution is 2255*2255, and the process of adjusting the resolution of the original space image to the second set resolution is: Performing zero padding processing on the boundaries of the original spatial image in both horizontal and vertical directions, and then dividing the image into 121 small-size image blocks with a resolution of 205*205, so as to obtain the second set of spatial images; The image segmentation module is used to perform bilinear interpolation upsampling operations on the image blocks in the first group of spatial images and the second group of spatial images one by one, and then input them into the semantic segmentation network structure for segmentation processing one by one, and then perform pooling operations on the segmented image blocks; after the pooling operation is completed, the pooled image blocks are input into the activation function of the output layer to obtain image blocks with separated targets and backgrounds; wherein the semantic segmentation network structure is composed of 12 convolutional layers and 3 pooling layers, and each convolutional layer is provided with an activation function and a BN layer; Except for the sigmoid function set after the last convolution layer, the other activation functions are linear rectification functions; The image integration module is used to integrate the segmented image blocks in the first group of spatial images and the second group of spatial images respectively to obtain a first restored image and a second restored image; The image cutting module is used to obtain two images with the same resolution after performing edge cutting on the first restored image and the second restored image; The image addition module is used to perform mask addition on the two images with the same resolution to obtain a final image.
5. The target detection system according to claim 4, characterized in that: The image segmentation module includes: an upsampling unit, a pooling unit and an image block separation unit; The up-sampling unit is used to perform up-sampling operations on the image blocks in the first group of spatial images and the second group of spatial images one by one; The pooling unit performs a pooling operation on the segmented image blocks; The image block separation unit is used to input the pooled image blocks into the activation function of the output layer to obtain image blocks separated from the target and the background.
6. The target detection system according to claim 5, characterized in that: The image block module includes an image zero-padding processing unit; The zero-padding processing unit is used to divide the original spatial image into 100 small-size image blocks with a resolution of 205*205, so as to obtain the first group of spatial images; The zero-padding processing unit is also used to divide the original spatial image into 121 small-sized image blocks with a resolution of 205*205, so as to obtain the second group of spatial images.
