Infrared weak and small target multi-feature recognition method based on residual learning
By combining multiple infrared features and deep learning technologies, the infrared weak target recognition is adopted based on residual learning, which solves the problem of insufficient feature extraction and adaptability in the existing technology, and achieves efficient identification and accuracy in different scenarios Identify.
Patent Information
- Application Number
- CN202410881948.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-03
- Publication Date
- 2024-11-15
AI Technical Summary
The existing infrared target recognition algorithms have shortcomings in feature extraction and adaptability. The traditional algorithm features are not learnable and have poor scalability. Deep learning algorithms require a large amount of data and computing resources, and are sensitive to initial parameters, making it difficult to be effective in different scenarios. Identify weak targets.
Using the multi-feature recognition method of infrared weak targets based on residual learning, combined with local binary features, directional gradient histogram features, corner features, spatial infrared target profile features, geometric features and invariant moment features, features are performed through convolutional neural networks. Select and deep learning object detection, and use deep learning image super-resolution algorithm for object recognition.
It improves the accuracy and robustness of infrared target recognition, can effectively identify weak targets in different scenarios, enhances recognition effect and adaptability, reduces dependence on initial parameters, and reduces the demand for computing resources.
Smart Images

Figure CN118968014A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image recognition, and in particular relates to a multi-feature recognition method for infrared dim small targets based on residual learning. Background Art
[0002] Infrared target recognition algorithms can be divided into two categories: traditional algorithms and deep learning algorithms. Traditional algorithms mainly rely on manual feature extraction for target recognition, including gray-level co-occurrence matrix, gradient histogram, etc. Deep learning algorithms use a series of convolutional neural network models to extract features and classify.
[0003] Traditional algorithms have the following advantages: (1) Simple implementation: Traditional algorithms only need to use some simple mathematical methods to extract features. (2) Stable and controllable: Traditional algorithms are stable and controllable, and their results are relatively reliable. Their shortcomings are: (1) Features are not learnable, which limits the applicability of the algorithm and makes it difficult to adapt to new scenarios and problems. (2) Poor scalability: Each target requires a different feature extraction method, which increases the amount of calculation and makes it difficult to achieve scalable design.
[0004] Deep learning algorithms have the following advantages: (1) Features are learnable: Features can be learned automatically and optimized through the back-propagation algorithm. (2) Strong adaptability: Feature extraction methods are highly adaptive and can adapt to different scenarios and problems. (3) Good scalability: Feature extraction methods can achieve scalability design by increasing the number of network layers or increasing training data. The shortcomings of deep features are: (1) Training requires a large amount of data: Lack of data may lead to problems such as overfitting or underfitting of the algorithm. (2) Long training time: This consumes a lot of time and computing resources. (3) Sensitive to initial parameters: The choice of initial parameters will affect the final results of the algorithm.
[0005] Among them, various methods such as the fusion of traditional algorithms and deep learning algorithms, multi-classifier fusion, and infrared and visible light image fusion are also constantly developing and improving. Summary of the invention
[0006] In order to solve the above technical problems, the present invention proposes a multi-feature recognition scheme for infrared dim small targets based on residual learning.
[0007] The first aspect of the present invention proposes a method for multi-feature recognition of infrared dim small targets based on residual learning. The method comprises:
[0008] Step S1, extracting typical target infrared features from the infrared image to be identified;
[0009] The typical target infrared features include local binary features, directional gradient histogram features, corner features, one-dimensional entropy features, spatial infrared target contour features, geometric features, and invariant moment features;
[0010] Step S2, using a convolutional neural network feature selection network to perform feature selection on the extracted typical target infrared features;
[0011] Step S3: Use a deep learning target detection algorithm to perform full-image recognition of the large field of view image on the selected features to obtain the target position, target size and target outline;
[0012] Step S4: using a deep learning image super-resolution algorithm, super-resolution reconstruction is performed on the local image of the small field of view tracking target based on the target position, target size and target contour, the resolution of the local target image is expanded, and the target classification algorithm is used to complete the target recognition.
[0013] According to the method described in the first aspect of the present invention, in step S1:
[0014] Extracting local binary features LBP specifically includes:
[0015] Divide the infrared image to be identified into K*K sub-region cells;
[0016] For each pixel of the sub-block, determine the circular neighborhood radius R and the number of sampling points P, and calculate the consistent LBP features of the central pixel;
[0017] For the LBP features of the sub-blocks, a statistical histogram is established and normalized;
[0018] The histogram features of K*K sub-blocks are cascaded to describe the texture features of the entire image;
[0019] Extracting the directional gradient histogram feature HOG specifically includes:
[0020] Use the Gamma correction method to normalize the infrared image to be identified, F(x,y)=F(x,y) gamma , where F(x,y) represents the pixel value of the infrared image to be identified, and gamma represents the compression parameter;
[0021] A one-dimensional discrete differential template is used to calculate pixel gradients in both horizontal and vertical directions;
[0022] Determine the size of the sub-region cell and the number of bins for the statistical window of the gradient direction; for each pixel in the cell, perform a weighted vote based on its gradient direction value as the histogram channel, and the weight is calculated by Gaussian weighting of the gradient amplitude;
[0023] If the gradient with a sign is calculated, the gradient direction from -180 to 180 degrees is counted; if the gradient without a sign is calculated, the gradient direction from 0 to 180 degrees is counted;
[0024] Determine the size of the block, concatenate the features of all cells in a block as the features of the block, and normalize them;
[0025] The overlap rate of the area blocks is determined, and the features of all area blocks in the cascade image constitute the final HOG feature vector.
[0026] Extracting corner features specifically includes:
[0027] Calculate the matrix M, Among them I x and I y Indicates the gradient size in two directions in the infrared image to be identified, G is the Gaussian template;
[0028] Calculate the autocorrelation function R(x,y)=det[M(x,y)]-k·(tr[M(x,y)]) 2 , where k is a constant, det represents the determinant of the matrix, and tr represents the trace of the matrix;
[0029] If R(x, y) at a certain point exceeds a preset threshold, the point is extracted as a corner point.
[0030] According to the method described in the first aspect of the present invention, in step S1:
[0031] Extracting one-dimensional entropy features specifically includes:
[0032] Given a segmentation threshold, calculate the entropy corresponding to the background and foreground respectively;
[0033] Calculate the total entropy of the image under all segmentation thresholds, determine the maximum entropy, and use the segmentation threshold corresponding to the maximum entropy as the final threshold;
[0034] Pixels in the image with grayscale greater than the threshold are regarded as foreground, otherwise as background;
[0035] Extracting the contour features of space infrared targets specifically includes:
[0036] M b (i, j) is the binary image of the infrared image to be identified, where j=1,2,…,n, where m and n are the width and height of the image respectively;
[0037] For the i-th row of the binary image, determine whether each pixel is 1. b When (i, j) = 1, continue to judge the next pixel, and so on until pixel M b(i,j+N)=0;
[0038] The area with the first pixel value of 1 in this row is numbered as t n , the number of pixels in this area is N, and the number is set to t n +1, repeat the above steps until the last pixel M b (i,n);
[0039] Get all the regions with a value of 1 in a row, the number of each region, the pixels that make up each region, and the total number of regions M i ;
[0040] For the i+1th row of the image, the number is set to t n +M i Repeat the above process to obtain the regions with all values of 1 in the i+1th row, the numbers of each region, the pixels that make up each region, and the total number of regions M i+1 ;
[0041] Merge the connected areas of the i-th row and the i+1-th row, and reset the number of the i+1-th row to the number of the area connected to the i-th row;
[0042] If a region in the (i+1)th row can be merged with multiple regions, all of them are merged into the same region, the number is set to the smallest number value, and the data is updated.
[0043] According to the method described in the first aspect of the present invention, in step S1:
[0044] The extracted geometric features include:
[0045] M n (i, j) is the pixel composition of the target in the infrared image M to be identified, and the aspect ratio is p1 = [max(i)-min(i)] / [max(j)-min(j)];
[0046] Use the chain code table to count the number of pixel points on the target boundary to obtain the perimeter P, and then calculate the area A of the target area according to the number of pixel points in the area;
[0047] Calculate the complexity p2 = P / A, use horizontal and vertical projections to get the width W and length H of the rectangle, and calculate the compactness p3 = A / (H×W);
[0048] Extracting invariant moment features specifically includes:
[0049] M b (x,y) is a grayscale image, calculate the p+q moment and target center of gravity (x0, y0) = (m 10 ,m 01 ) / m00 ;
[0050] Calculate the p+q order central moment Normalized central moment Where r = (p+q+2) / 2, p+q = 2, 3, 4, ...
[0051] According to the method described in the first aspect of the present invention, in the step S2, a feature selection network based on a convolutional neural network is used at the decision level to select features with optimized anti-interference performance, wherein: the feature selection network based on a convolutional neural network adopts a convolutional architecture, using three convolutional layers Conv1-Conv3 and two fully connected layers FC4-FC5; in order to train a highly discriminative target feature selection network, a multi-branch target-specific fully connected layer is configured after FC5, each branch corresponds to a single feature, and feature confidence is generated by another fully connected layer.
[0052] According to the method of the first aspect of the present invention, in step S3, the deep learning target detection algorithm is implemented by a neural network model, a deep convolutional network structure based on a pyramid model is adopted, fine-tuning is performed on the basis of a residual network, and an integrated RPN network is used to replace the selective search module in the R-CNN network in the generation of the detection frame; wherein:
[0053] A deep convolutional network based on the pyramid model is constructed. The convolutional network is selected from the residual network Resnet50. The feature pyramid structure is used to output three different feature maps. The training image is normalized in size by a three-layer feature fusion method and then the feature map is output.
[0054] The detection frame is generated using the integrated RPN network, which is a fully convolutional network. The first layer is a 3×3 convolution kernel and activation function ReLU, which is used to generate 256-dimensional features. The second layer is a regression layer for classifying and deviating the detection frame, which uses 1×1 convolution and outputs k groups of classification values and coordinate deviations, where k is the type of the reference frame.
[0055] The feature pyramid structure adopts a Laplacian pyramid; the original image is mapped to the target image, up-sampled, and then convolved with a Gaussian kernel, and then the original image and the image after the Gaussian convolution are subtracted to finally obtain a Laplacian pyramid image;
[0056] The definition of the i-th layer of its implementation process is Among them, G i represents the original image of the i-th layer, UP() means mapping the pixel at position (k, j) in the original image to the position (2k+1, 2j+1) of the target image to achieve up sampling, g 5×5Represents a 5×5 Gaussian convolution kernel; the training process of the deep convolutional network based on the feature pyramid structure is achieved by fine-tuning the residual network Resnet50.
[0057] According to the method described in the first aspect of the present invention, in step S4: image super-resolution reconstruction refers to improving the resolution of the target local area image, and the detection sensor does not make any changes; the target local area image output by the target detection part is reconstructed through super-resolution, so that the target resolution is enhanced.
[0058] According to the method of the first aspect of the present invention, in step S4, for target recognition and classification, a deep network model based on residual learning is used to convert the implicit mapping relationship between learning levels into a residual between the input data and the output data after mapping by optimizing the residual value. The closer the residual value is to 0, the closer the features extracted by the network are to the original input; wherein:
[0059] After receiving the super-resolution reconstructed target image, the deep residual network ResNet34 is used to complete the final recognition of the target type. The convolution layer consists of a 3×3 convolution kernel, each convolution layer is followed by a ReLU activation function, and a convolution operation with a step size of 2 is used instead of the pooling layer.
[0060] A second aspect of the present invention provides an infrared dim small target multi-feature recognition system based on residual learning, the system comprising a processor, the processor being configured to execute:
[0061] Extract typical target infrared features from the infrared image to be identified;
[0062] The typical target infrared features include local binary features, directional gradient histogram features, corner features, one-dimensional entropy features, spatial infrared target contour features, geometric features, and invariant moment features.
[0063] The feature selection network of convolutional neural network is used to select the extracted typical target infrared features;
[0064] Use deep learning target detection algorithm to perform full-image recognition of large field of view images on the selected features to obtain the target position, target size and target outline;
[0065] Using deep learning image super-resolution algorithm, the local image of the target tracked in a small field of view is super-reconstructed based on the target position, target size and target contour, the resolution of the local image of the target is expanded, and the target classification algorithm is used to complete target recognition.
[0066] The third aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for multi-feature recognition of infrared dim small targets based on residual learning described in the first aspect is implemented.
[0067] In summary, the infrared target recognition algorithm based on multiple feature fusion is one of the research hotspots in the field of computer vision in recent years. Aiming at the problem of typical infrared weak and small target recognition, the present invention develops a typical infrared target multi-feature judgment and fusion model based on the typical target infrared feature extraction method, establishes weights between different features, and ensures the accuracy of modeling the typical infrared target and its background appearance. At the same time, the infrared target recognition mechanism based on multi-feature fusion can better cope with complex target recognition problems in different scenes and environments, enhance the infrared target recognition effect, and further improve the recognition accuracy and robustness, thereby realizing the accurate detection and intelligent recognition of typical weak and small targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0069] Figure 1 It is a flow chart of the infrared dim target multi-feature recognition method based on residual learning;
[0070] Figure 2 A structural diagram of a convolutional neural network according to an embodiment of the present invention;
[0071] Figure 3 is a structural diagram of a super-resolution imaging network according to an embodiment of the present invention;
[0072] Figure 4 Detection and recognition effect diagram of some test pictures according to an embodiment of the present invention. DETAILED DESCRIPTION
[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0074] First embodiment
[0075] In order to realize the recognition of typical infrared dim targets, the present invention proposes a multi-feature recognition method for infrared dim targets based on residual learning, and the process is as follows ( Figure 1 shown):
[0076] Typical target infrared feature extraction. The present invention uses 7 feature extraction methods, including: local binary feature extraction, directional gradient histogram feature, corner point feature, one-dimensional entropy feature, spatial infrared typical target contour extraction method based on active contour model, geometric feature extraction, and invariant moment feature extraction.
[0077] Feature selection. In order to provide accurate information for subsequent recognition algorithms to the greatest extent possible, the present invention uses a feature determination method based on a convolutional neural network to select a feature extraction model with optimized anti-interference performance at the decision level.
[0078] like Figure 2 As shown in the figure, the feature selection network based on convolutional neural network adopts a convolutional architecture. To ensure real-time performance, three convolutional layers (Conv1-3) and two fully connected layers (FC4-5) are used. In order to train a highly discriminative target feature selection network, FC5 is followed by a multi-branch target-specific fully connected layer, each branch corresponds to a single feature, and a fully connected layer generates feature confidence. Through this mechanism, the feature selection network of this framework can ensure strong discriminability and select the feature extraction method with the highest confidence.
[0079] Based on the multi-category features of the target and its background extracted in the first two steps, infrared target recognition technology mainly includes two aspects: 1) In the target detection stage, a deep learning target detection algorithm is used to perform full-image recognition on the large field of view image to obtain the target position, target size and target outline; 2) A deep learning image super-resolution algorithm and a target classification algorithm are used to perform super-resolution reconstruction on the local image of the target tracked in the small field of view, expand the resolution of the local target image, and use the target classification network to complete target recognition.
[0080] Construct a neural network model. The present invention adopts a deep convolutional network structure based on the pyramid model and performs fine-tuning on the basis of the residual network. In the generation of the detection frame, the integrated Region Proposal Network (RPN) network is used to replace the Selective Search module in the traditional R-CNN network to generate the detection frame.
[0081] Image super-resolution reconstruction. Figure 3As shown in the figure, image super-resolution reconstruction is a technology that improves the resolution of the local area image of the target. The detection sensor does not make any changes, and the original image resolution is improved. The local area image of the target output by the target detection part is reconstructed through super-resolution to enhance the target resolution.
[0082] Target recognition and classification. A deep network model based on residual learning is used to convert the implicit mapping relationship between learning layers into the residual between the input data and the output data after mapping by optimizing the residual value. The closer the residual value is to 0, the closer the features extracted by the network are to the original input, and the better the network's ability to learn abstract features is. Receive the target image reconstructed by super-resolution in the previous step and complete the recognition of the target type.
[0083] Second embodiment
[0084] The basic process of the infrared dim small target recognition method based on multi-feature fusion of the present invention is as follows: Figure 4 As shown, the specific implementation of the present invention is described by using an example, but the technical content of the present invention is not limited to the scope described. The specific implementation includes the following steps:
[0085] 1. Typical target infrared feature extraction and feature selection, the specific process is as follows:
[0086] (1) Extract local binary features (LBP): Divide the image into K*K small areas (cells); for each pixel of the sub-block, determine the circular neighborhood radius R and the number of sampling points P, and calculate the consistent LBP features of the central pixel; for the LBP features of each sub-block, establish a statistical histogram and perform normalization; cascade the histogram features of the K*K sub-blocks, and the feature vector describes the texture features of the entire image.
[0087] (2) Extract the Histogram of Oriented Gradients (HOG) feature: (i) Normalize the input image using the Gamma correction method, F(x,y) = F(x,y) gamma, where F(x,y) represents the pixel value of the input image and gamma represents the compression parameter. (ii) Use a one-dimensional discrete differential template to calculate the pixel gradient in both the horizontal and vertical directions. (iii) Determine the cell block size and the number of statistical windows (bins) for the gradient direction. For each pixel in the cell, a weighted vote is performed for the histogram channel based on its gradient direction value, and its weight is calculated by Gaussian weighting of the gradient amplitude. If a signed gradient is calculated, the gradient direction of -180 to 180 degrees is counted; if an unsigned gradient direction is calculated, the gradient direction of 0 to 180 degrees is counted. (iv) Determine the size of the region block. The features of all cells in a block are concatenated as the features of the block and normalized. (v) Determine the overlap rate of the region blocks. The features of all region blocks in the cascaded image constitute the final HOG feature vector.
[0088] (3) Extract corner features: (i) Calculate the matrix M as follows Among them I x and I y represents the gradient size in two directions in the image, G is the Gaussian template; (ii) calculate the autocorrelation function R(x,y) = det[M(x,y)]-k·(tr[M(x,y)]) 2 , where k is a constant, det represents the determinant of the matrix, and tr represents the trace of the matrix; (iii) If R(x, y) at a certain point exceeds the preset threshold, it is extracted as a corner point.
[0089] (4) Extract one-dimensional entropy features: (i) Given a segmentation threshold, calculate the entropy corresponding to the background and foreground respectively; (ii) Calculate the total entropy of the image under all segmentation thresholds, find the maximum entropy, and use the segmentation threshold corresponding to the maximum entropy as the final threshold. Pixels in the image with grayscale greater than this threshold are regarded as foreground, otherwise they are regarded as background.
[0090] (5) Extracting typical spatial infrared target contours based on active contour model: (i) Let the binary image be M b (i, j), where i = 1, 2, ..., m, j = 1, 2, ..., n, m and n are the width and height of the image respectively; (ii) for the i-th row of the binary image, determine whether each pixel is 1. When M b When (i, j) = 1, continue to judge the next pixel, and so on until pixel M b (i, j+N) = 0. The area with the first pixel value of 1 in this row is numbered as t n , we know that the number of pixels in this area is N. Set the number to t n +1, repeat the above steps until the last pixel M b(i,n). At this point, we can obtain all the regions with a value of 1 in a row, the number of each region, the pixels that make up each region, and the total number of regions M. i (iii) For the i+1th row of the image, the number is set to t n +M i Repeat the above process to obtain the regions with all values of 1 in the i+1th row, the numbers of each region, the pixels that make up each region, and the total number of regions M i+1 Merge the connected regions in row i and row i+1 and reset the number of row i+1 to the number of the region connected to row i. If a region in row i+1 can be merged with multiple regions, merge them all into the same region, set the number to the smallest value, and update the data.
[0091] (6) Extracting geometric features: (i) Let M n (i, j) are the pixels that make up the target in the original infrared image M, and the aspect ratio p1 = [max(i)-min(i)] / [max(j)-min(j)] can be obtained; (ii) The number of pixel points on the target boundary is counted using the chain code table to obtain the perimeter P, and the area A of the target area is obtained based on the number of pixels in the area. The complexity can be obtained by the following formula p2 = P / A; (iii) The width W and length H of the rectangle can be obtained by horizontal and vertical projection. The compactness can be obtained by the following formula p3 = A / (H×W).
[0092] (7) Extracting invariant moment features: (i) Using M b (x,y) represents a grayscale image, and calculates the p+q moment and target center of gravity (x0, y0) = (m 10 ,m 01 ) / m 00 , calculate the p+q order central moment Calculate the normalized central moment Where r = (p+q+2) / 2, p+q = 2, 3, 4, ...
[0093] 2. According to Figure 1 The network structure is used for feature selection, where the convolutional layers (Conv1-3) and fully connected layers (FC4-5) are taken from the convolutional architecture.
[0094] 3. Build a deep convolutional network based on the pyramid model. The specific process is as follows:
[0095] A deep convolutional network based on the pyramid model is constructed. The convolutional network in the figure is selected from the residual network Resnet50. On this basis, the feature pyramid structure can output three different feature maps. The three-layer feature fusion method is used to normalize the size of the input training image, and the final output feature map is the feature map.
[0096] The detection box is generated by integrating the Region Proposal Network (RPN). The RPN network is a fully convolutional network. The first layer generates 256-dimensional features under the action of a 3×3 convolution kernel and ReLU. The second layer is used to implement the classification of the detection box and the regression of the coordinate deviation, using 1×1 convolution and output k groups of classification values and coordinate deviations. The k is the type of the reference box.
[0097] The pyramid in the present invention adopts a Laplacian pyramid. First, the original image is mapped to the target image, up-sampled, and then convolved with a Gaussian kernel. Then, the original image and the image after Gaussian convolution are subtracted to finally obtain a Laplacian pyramid image. The i-th layer of the implementation process is defined as Among them, G i represents the original image of the i-th layer. The UP() operation maps the pixel at position (k, j) in the original image to the position (2k+1, 2j+1) of the target image to achieve upsampling. 5×5 Represents a 5×5 Gaussian convolution kernel.
[0098] The training of the deep convolutional network based on the pyramid model is achieved by fine-tuning the residual network Resnet50.
[0099] 4. Local image target recognition and classification, the specific process is as follows:
[0100] in accordance with Figure 3 The target local area image output by the target detection part is reconstructed through super-resolution to enhance the target resolution.
[0101] After receiving the super-resolution reconstructed target image, the deep residual network ResNet34 is used to complete the final recognition of the target type. The convolution layer consists of a 3×3 convolution kernel, each convolution layer is followed by a ReLU activation function, and a convolution operation with a stride of 2 is used instead of the pooling layer.
[0102] The detection and recognition effects of some test images are as follows: Figure 4 As shown, the left picture is the test picture, and the right picture is the recognition result.
[0103] The beneficial effects brought by the present invention mainly include: feature selection based on a multi-feature fusion infrared dim target recognition method, combined with a feature extraction model with optimized anti-interference performance, converting the implicit mapping relationship between learning levels into the residual between the optimized input data and the output data after mapping, performing full-image recognition on large-field-of-view images, obtaining the target position, target size and target contour, and improving recognition accuracy and robustness.
[0104] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above-mentioned embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several variations and improvements can be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.
Claims
1. A multi-feature recognition method for infrared dim small targets based on residual learning, characterized in that: The method comprises: Step S1, extracting typical target infrared features from the infrared image to be identified; The typical target infrared features include local binary features, directional gradient histogram features, corner features, one-dimensional entropy features, spatial infrared target contour features, geometric features, and invariant moment features; Step S2, using a convolutional neural network feature selection network to perform feature selection on the extracted typical target infrared features; Step S3: Use a deep learning target detection algorithm to perform full-image recognition of the large field of view image on the selected features to obtain the target position, target size and target outline; Step S4: using a deep learning image super-resolution algorithm, super-resolution reconstruction is performed on the local image of the small field of view tracking target based on the target position, target size and target contour, the resolution of the local target image is expanded, and the target classification algorithm is used to complete the target recognition.
2. According to the residual learning-based infrared dim small target multi-feature recognition method of claim 1, it is characterized in that: In step S1: Extracting local binary features LBP specifically includes: Divide the infrared image to be identified into K*K sub-region cells; For each pixel of the sub-block, determine the circular neighborhood radius R and the number of sampling points P, and calculate the consistent LBP features of the central pixel; For the LBP features of the sub-blocks, a statistical histogram is established and normalized; The histogram features of K*K sub-blocks are cascaded to describe the texture features of the entire image; Extracting the directional gradient histogram feature HOG specifically includes: Use the Gamma correction method to normalize the infrared image to be identified, F(x,y)=F(x,y) gamma , where F(x,y) represents the pixel value of the infrared image to be identified, and gamma represents the compression parameter; A one-dimensional discrete differential template is used to calculate pixel gradients in both horizontal and vertical directions; Determine the size of the sub-region cell and the number of bins for the statistical window of the gradient direction; for each pixel in the cell, perform a weighted vote based on its gradient direction value as the histogram channel, and the weight is calculated by Gaussian weighting of the gradient amplitude; If the gradient with a sign is calculated, the gradient direction from -180 to 180 degrees is counted; if the gradient without a sign is calculated, the gradient direction from 0 to 180 degrees is counted; Determine the size of the block, concatenate the features of all cells in a block as the features of the block, and normalize them; The overlap rate of the area blocks is determined, and the features of all area blocks in the cascade image constitute the final HOG feature vector. Extracting corner features specifically includes: Calculate the matrix M, Among them I x and I y Indicates the gradient size in two directions in the infrared image to be identified, G is the Gaussian template; Calculate the autocorrelation function R(x,y)=det[M(x,y)]-k·(tr[M(x,y)]) 2 , where k is a constant, det represents the determinant of the matrix, and tr represents the trace of the matrix; If R(x, y) at a point exceeds a preset threshold, the point is extracted as a corner point.
3. The method for multi-feature recognition of infrared small targets based on residual learning according to claim 2 is characterized in that: In step S1: Extracting one-dimensional entropy features specifically includes: Given a segmentation threshold, calculate the entropy corresponding to the background and foreground respectively; Calculate the total entropy of the image under all segmentation thresholds, determine the maximum entropy, and use the segmentation threshold corresponding to the maximum entropy as the final threshold; Pixels in the image with grayscale greater than the threshold are regarded as foreground, otherwise as background; Extracting the contour features of space infrared targets specifically includes: M b (i, j) is the binary image of the infrared image to be identified, where j=1,2,...,n, where m and n are the width and height of the image respectively; For the i-th row of the binary image, determine whether each pixel is 1. b When (i, j) = 1, continue to judge the next pixel, and so on until pixel M b (i,j+N)=0; The area with the first pixel value of 1 in this row is numbered as t n , the number of pixels in this area is N, and the number is set to t n +1, repeat the above steps until the last pixel M b (i,n); Get all the regions with a value of 1 in a row, the number of each region, the pixels that make up each region, and the total number of regions M i ; For the i+1th row of the image, the number is set to t n +M i Repeat the above process to obtain the regions with all values of 1 in the i+1th row, the numbers of each region, the pixels that make up each region, and the total number of regions M i+1 ; Merge the connected areas of the i-th row and the i+1-th row, and reset the number of the i+1-th row to the number of the area connected to the i-th row; If a region in the (i+1)th row can be merged with multiple regions, all of them are merged into the same region, the number is set to the smallest number value, and the data is updated.
4. The method for multi-feature recognition of infrared small targets based on residual learning according to claim 3 is characterized in that: In step S1: The extracted geometric features include: M n (i, j) is the pixel composition of the target in the infrared image M to be identified, and the aspect ratio is p1 = [max(i)-min(i)] / [max(j)-min(j)]; Use the chain code table to count the number of pixel points on the target boundary to obtain the perimeter P, and then calculate the area A of the target area according to the number of pixel points in the area; Calculate the complexity p2 = P / A, use horizontal and vertical projections to get the width W and length H of the rectangle, and calculate the compactness p3 = A / (H×W); Extracting invariant moment features specifically includes: M b (x,y) is a grayscale image, calculate the p+q moment and target center of gravity (x0, y0) = (m 10 ,m 01 ) / m 00 ; Calculate the p+q order central moment Normalized central moment Where r = (p+q+2) / 2, p+q = 2, 3, 4, ...
5. The method for multi-feature recognition of infrared small targets based on residual learning according to claim 4 is characterized in that: In the step S2, a feature selection network based on a convolutional neural network is used at the decision level to select features with optimized anti-interference performance, wherein: the feature selection network based on a convolutional neural network adopts a convolutional architecture, using three convolutional layers Conv1-Conv3 and two fully connected layers FC4-FC5; in order to train a highly discriminative target feature selection network, a multi-branch target-specific fully connected layer is configured after FC5, each branch corresponds to a single feature, and feature confidence is generated by another fully connected layer.
6. The method for multi-feature recognition of infrared small targets based on residual learning according to claim 5 is characterized in that: In step S3, the deep learning target detection algorithm is implemented through a neural network model, using a deep convolutional network structure based on a pyramid model, fine-tuning is performed on the basis of a residual network, and an integrated RPN network is used to replace the selective search module in the R-CNN network in the generation of the detection frame; wherein: A deep convolutional network based on the pyramid model is constructed. The convolutional network is selected from the residual network Resnet50. The feature pyramid structure is used to output three different feature maps. The training image is normalized in size by a three-layer feature fusion method and then the feature map is output. The detection frame is generated using the integrated RPN network, which is a fully convolutional network. The first layer is a 3×3 convolution kernel and activation function ReLU, which is used to generate 256-dimensional features. The second layer is a regression layer for classifying and deviating the detection frame, which uses 1×1 convolution and outputs k groups of classification values and coordinate deviations, where k is the type of the reference frame. The feature pyramid structure adopts a Laplacian pyramid; the original image is mapped to the target image, up-sampled, and then convolved with a Gaussian kernel, and then the original image and the image after the Gaussian convolution are subtracted to finally obtain a Laplacian pyramid image; The definition of the i-th layer of its implementation process is Among them, G i represents the original image of the i-th layer, UP() means mapping the pixel at position (k, j) in the original image to the position (2k+1, 2j+1) of the target image to achieve up sampling, g 5× 5 represents a 5×5 Gaussian convolution kernel; the training process of the deep convolutional network based on the feature pyramid structure is achieved by fine-tuning the residual network Resnet50.
7. The method for multi-feature recognition of infrared small targets based on residual learning according to claim 6 is characterized in that: In step S4: image super-resolution reconstruction refers to improving the resolution of the target local area image, and the detection sensor does not make any changes; the target local area image output by the target detection part is reconstructed through super-resolution, so that the target resolution is enhanced.
8. The method for multi-feature recognition of infrared small targets based on residual learning according to claim 7 is characterized in that: In step S4, for target recognition and classification, a deep network model based on residual learning is used to convert the implicit mapping relationship between learning levels into the residual between the input data and the output data after mapping by optimizing the residual value. The closer the residual value is to 0, the closer the features extracted by the network are to the original input; wherein: After receiving the super-resolution reconstructed target image, the deep residual network ResNet34 is used to complete the final recognition of the target type. The convolution layer consists of a 3×3 convolution kernel, each convolution layer is followed by a ReLU activation function, and a convolution operation with a step size of 2 is used instead of the pooling layer.
9. An infrared dim target multi-feature recognition system based on residual learning, characterized in that: The system includes a processor configured to execute: Extract typical target infrared features from the infrared image to be identified; The typical target infrared features include local binary features, directional gradient histogram features, corner features, one-dimensional entropy features, spatial infrared target contour features, geometric features, and invariant moment features. The feature selection network of convolutional neural network is used to select the extracted typical target infrared features; Use deep learning target detection algorithm to perform full-image recognition of large field of view images on the selected features to obtain the target position, target size and target outline; Using deep learning image super-resolution algorithm, the local image of the target tracked in a small field of view is super-reconstructed based on the target position, target size and target contour, the resolution of the local image of the target is expanded, and the target classification algorithm is used to complete target recognition.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method for multi-feature recognition of infrared dim targets based on residual learning as described in any one of claims 1 to 8 is implemented.
Citation Information
Cited By
Polypeptide drug membrane filtration pollution identification method and system based on convolutional neural network
CN120259830A
A convolutional neural network-based method and system for identifying peptide drug membrane filtration contamination
CN120259830B
Infrared small target detection method for graph structure analysis
CN120451512A
An infrared small target detection method based on graph structure analysis
CN120451512B
Large-scale rotary equipment blade vibration measurement method combined with multiple feature mark points
CN120869331A
