Infrared target detection method based on generated data set
By generating a data set and video continuous frame segmentation model, the problems of scarcity and labeling difficulties in infrared object detection are solved, and efficient detection of small infrared objects is achieved to adapt to the detection needs of different scenarios.
Patent Information
- Application Number
- CN202411143557.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-20
AI Technical Summary
The existing infrared object detection methods require a large number of data sets during model training, and manual labeling of small moving targets is difficult and time-consuming, resulting in scarce infrared monitoring small target labeling data sets and cannot meet the training data needs of specific task scenarios.
Using infrared object detection method based on the generated data set, by collecting background video and adding artificial object information, a video continuous frame segmentation training model is constructed, and the generated data set is used for model training, and the infrared detector video is analyzed in real time through the trained model to Detect tiny moving targets.
It improves the diversity and authenticity of training data, simplifies the data annotation process, provides flexibility to adapt to different types of targets and backgrounds, and improves the performance of infrared small object detection.
Smart Images

Figure CN119048861A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of target detection and relates to an infrared target detection method based on generating a data set. Background Art
[0002] In some monitoring scenarios equipped with infrared detectors, the pixels of the monitored target in the field of view are very small, and the target may move quickly. Therefore, in order to better monitor the maneuvering target in the field of view, it is necessary to combine its appearance and operation information fusion to achieve it. The detection of small moving targets based on infrared detectors is an important research field, and its applications include detecting long-range maneuvering targets, studying the behavior of monitored targets, and judging their intentions based on the behavior of monitored targets. However, due to the low resolution of infrared images themselves, blurred boundaries, and the lack of unique shape and texture details of small targets, traditional image processing methods and the direct application of modern target detection methods based on CNN show quite low performance.
[0003] The prior art with publication number CN117409192A discloses a method and device for infrared small target detection based on data enhancement. It first adopts a background filling method to obtain a clean background image, and uses a diffusion model to generate a more complex and diverse background image; then a target adaptive fusion method based on a generative adversarial network is designed to better fuse the target mask and the generated background image, thereby generating a more realistic infrared small target enhanced image.
[0004] However, in the process of model training, a large number of data sets are usually required, and manual annotation of tiny moving objects in videos is very difficult and time-consuming. Since it is difficult for human observers to detect tiny moving objects in recorded videos, manual annotation of a large number of videos is a difficult and error-prone task. Currently, there are few public small target annotation data sets that meet the requirements of infrared monitoring. Therefore, for specific mission scenarios, there is no available training data set that can be used directly. Summary of the invention
[0005] In view of this, the purpose of the present invention is to provide an infrared target detection method based on a generated data set. The present invention proposes a new model establishment, training and evaluation method. In the training stage, background video is collected and artificial target information is added to the background video to obtain a generated data set. The model is trained by the generated data set. After the training is completed, the model is used to analyze the video information obtained by the infrared detector in real time to detect small moving targets.
[0006] In order to achieve the above object, the present invention provides the following technical solutions:
[0007] An infrared target detection method based on generating a data set comprises the following steps:
[0008] S1, collecting background videos in typical scenes and stabilizing the background videos;
[0009] S2, using a target generation algorithm to initialize artificial target information in the first frame of the original background video sequence, and simulating the artificial target in each subsequent frame of the original background video sequence to obtain a synthetic video sequence;
[0010] S3, constructing a video continuous frame segmentation training model, which at least includes an encoder and a decoder, wherein the encoder is used to extract features from input data, and the decoder is used to process features output by the encoder;
[0011] S4, using the synthetic video sequence as a data set sample space to train the constructed video continuous frame segmentation training model;
[0012] S5. Use the trained video continuous frame segmentation model to analyze the real-time infrared small target detection video and output the target's location information.
[0013] Further, in step S1, the following steps are included:
[0014] First, background videos are collected based on real scenes. The selection of scenes takes into account typical backgrounds in various situations around the clock. During the collection process, ensure that there are no surveillance targets in the video.
[0015] An affine transformation is performed between each frame and the first frame in the sequence of the collected original background video, that is, based on the same common reference frame, the video stabilization processing is achieved; the key points in the two frames are detected, and the descriptors of the picture frames are calculated using the ORB detector to estimate the affine transformation between the two frames; then, the descriptors are compared using the Hamming distance to find matching key point pairs, and the RANSAC algorithm is used to optimize the robustness of key point matching.
[0016] Further, in step S1, the ORB descriptor is used to quickly create a feature vector descriptor for the key points in the image, and identify the object in the image through the feature vector descriptor, which includes two steps:
[0017] The first step is to use the FAST algorithm to find key points in the image;
[0018] The second step is to use the BRIEF descriptor to convert the key points found in the first step into feature vectors to distinguish different objects;
[0019] The Hamming distance is used to calculate the number of different characters in corresponding positions of two strings of equal length. For binary strings A and B, the calculation formula is:
[0020]
[0021] Among them, 1(a i ≠b i ) is the indicator function, if a i Not equal to b i , its value is 1; otherwise it is 0;
[0022] In the optimization of key point matching robustness, the RANSAC algorithm removes the wrong matches when it is known that there are wrong matches, thereby estimating the homography matrix of the model transformation. The basic steps of the RANSAC algorithm are:
[0023] 1) Random sampling: randomly select a minimum subset from the data set that is sufficient to fit the model parameters;
[0024] 2) Model fitting: Fit the model using the randomly selected subset;
[0025] 3) Calculate inliers: Calculate the distance between each point in the entire data set and the fitted model. If the distance is within a predefined threshold, it is considered an inlier.
[0026] 4) Evaluate the model: record the number of inliers of the current model. If the number of inliers exceeds that of the previous best model, update the best model.
[0027] 5) Repeat: Repeat the above steps several times until the number of iterations is met;
[0028] 6) Model re-estimation: Re-estimate the model parameters using the best set of inliers.
[0029] Further, in step S2, the following steps are specifically included:
[0030] S21, initializing the size, quantity, initial position, initial flight direction and initial speed of the artificial target in the first frame of the original background video;
[0031] According to different target features, spots with different distributions are used to represent small targets, and Gaussian two-dimensional distribution is used for simulation, that is, the size of the artificial target is initialized by controlling the standard deviation of the two dimensions of the two-dimensional Gaussian distribution; the number, initial position, initial flight direction and initial speed of the artificial targets are randomly selected from the uniform distribution function that meets the specified parameter interval; the texture of the artificial target is modeled using Gaussian noise; the distribution type and value range used to generate the artificial target can be combined with the subsequent CNN training network for comprehensive optimization and value selection;
[0032] When adding an artificial target to the original image, the pixel value of the artificial target is used as the alpha channel to blend the background and the target;
[0033] S22. After completing the initialization of the first frame, simulate the movement of the artificial target in each subsequent frame, and the movement direction and movement speed of the artificial target are calculated using the following formula:
[0034] υ t =υ t-1 +Δ υ ,
[0035] θ t =θ t-1 +Δ θ ,
[0036] According to the position, speed and direction of the target in the previous frame, the speed and direction of the next frame are simulated, and then the new position of the target is calculated by the new speed and direction, and the artificial targets beyond the image boundary are eliminated;
[0037] Based on the above method, artificial targets are synthesized from the original background video to form a new video sequence. At the same time, the generated artificial target images are saved separately as labels for subsequent model training.
[0038] S23, segmenting the generated video, taking n consecutive frames of images as a video sequence, the label corresponding to each video sequence is the label of the last frame, and establishing an initial data set with n-frame video sequences and the last frame label corresponding to each other;
[0039] A probabilistic method based on the number of objects in the field of view is used to establish the training set. For each video sequence, the probability of it being selected into the final data set is dynamically adjusted according to the number of artificial objects in its field of view. That is, the more objects a video sequence has in its field of view, the greater its probability of being selected.
[0040] The video background estimation and difference method are used to process the video dataset to emphasize the moving pairs in each frame. The mean μ(x, y) and standard deviation σ(x, y) are calculated frame by frame for each pixel of each video, and then the z-score normalized data processing is performed:
[0041]
[0042] where I(x, y, t), I b s(x, y, t) are the original pixel value at the coordinate x, y position at time t and the pixel value after background difference.
[0043] Further, in step S3, the constructed video continuous frame segmentation training model adopts a CNN training network based on an improved U-Net, which includes an encoder and a decoder, wherein the encoder is responsible for feature extraction of input data, and the decoder processes according to the features output by the encoder, wherein the encoder includes at least a number of blocks consisting of a convolutional layer, a batch normalization layer and a ReLU activation function, and a maximum pooling layer following each block;
[0044] The decoder includes at least several blocks consisting of an upsampling layer, a convolutional layer, a batch normalization layer and a ReLU activation function, and a convolutional layer and a sigmoid activation function in the last layer, wherein the upsampling layer upsamples the obtained feature map using nearest neighbor interpolation;
[0045] Among them, the convolutional layer outputs of the encoder and decoder use symmetric jump connections with feature maps of the same size to fuse the low-level features with the high-level features;
[0046] The size of the segmented image finally output by the decoder is half of the original frame image, and the obtained segmentation map is then upsampled using bilinear interpolation to ensure that the output frame is consistent with the original frame image size.
[0047] The encoder consists of three blocks consisting of a convolutional layer, a batch normalization layer and a ReLU activation function, and a maximum pooling layer following each block, where the convolution kernel size in the first block is 3*3, the convolution kernel size in the second block is 2*2, and the size in the third block is 1*1.
[0048] The decoder consists of two blocks consisting of an upsampling layer, a convolutional layer, a batch normalization layer, and a ReLU activation function.
[0049] A convolution layer and a batch normalization layer are set between the encoder and the decoder, which are used to pass the output of the encoder to the decoder.
[0050] Furthermore, in step S4, the generated dataset sample space is divided into a training set, a validation set, and a test set in proportion. During the training process, the L2 loss function of the model output and the label is used as the optimization target to complete the training and determine the final network weights.
[0051] Further, in step S5, the following operations are specifically included:
[0052] Get real-time infrared small target detection video;
[0053] Perform video stabilization on n consecutive frames;
[0054] Perform background estimation and subtraction on n frames of video;
[0055] Then the video frame is input into the trained continuous frame segmentation based CNN target prediction network for target prediction;
[0056] The target estimation result is thresholded, and when the target estimation result meets the threshold condition, the coordinate information is output.
[0057] The beneficial effects of the present invention are:
[0058] The video synthesis algorithm provided by the present invention has the function of adjustable parameters and can easily adapt to different types of target objects and real-world backgrounds. By adding artificially synthesized spot-like objects to simulate the monitoring target and initializing it using methods such as Gaussian two-dimensional distribution, accurate simulation of the target's motion mode and flight state is achieved. This method not only improves the diversity and authenticity of training data, but also provides strong support for subsequent model training.
[0059] When simulating artificial targets, the present invention not only considers key factors such as the size, quantity, initial position, initial flight direction and initial speed of the targets, but also uses Gaussian noise for texture modeling, which further improves the accuracy and authenticity of the simulation. In addition, by calculating the movement direction and speed of the artificial targets, the accurate simulation of the target movement is achieved, ensuring the continuity and consistency of the synthesized video sequence.
[0060] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:
[0062] Figure 1 The overall process of the infrared target detection method based on generating a data set of the present invention;
[0063] Figure 2 It is a structural schematic diagram of the CNN video continuous frame segmentation training model of the present invention;
[0064] Figure 3 It is a real-time detection flow chart of the CNN video continuous frame segmentation training model of the present invention;
[0065] Figure 4 This is the video stabilization processing flow of the present invention. DETAILED DESCRIPTION
[0066] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.
[0067] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.
[0068] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0069] See also Figure 1 to Figure 4 , which is an infrared target detection method based on generated datasets.
[0070] Example
[0071] This embodiment describes in detail the specific implementation of the infrared target detection method based on the generated data set proposed by the present invention. Figure 1 As shown, it specifically includes the following steps:
[0072] S1, collecting background videos in typical scenes and stabilizing the background videos;
[0073] S2, using a target generation algorithm to initialize artificial target information in the first frame of the original background video sequence, and simulating the artificial target in each subsequent frame of the original background video sequence to obtain a synthetic video sequence;
[0074] S3, constructing a video continuous frame segmentation training model, which at least includes an encoder and a decoder, wherein the encoder is used to extract features from input data, and the decoder is used to process features output by the encoder;
[0075] S4, using the synthetic video sequence as a data set sample space to train the constructed video continuous frame segmentation training model;
[0076] S5. Use the trained video continuous frame segmentation model to analyze the real-time infrared small target detection video and output the target's location information.
[0077] In step S1, the present embodiment first collects background video according to the real scene. The selection of the scene should take into account the typical background in various situations around the clock, and ensure that there is no surveillance target in the video during the collection process. In order to eliminate the possible jitter of the detector, the present embodiment performs an affine transformation between each frame in the sequence and the first frame, that is, based on the same common reference frame, the video is stabilized. In order to estimate the affine transformation between the two frames, the present embodiment first detects the key points in the two frames and uses the ORB detector to calculate the descriptor of the picture frame. Then, the descriptors are compared using the Hamming distance to find the matching key point pairs, and the RANSAC algorithm is used to optimize the robustness of the key point matching.
[0078] The ORB descriptor is used to quickly create feature vector descriptors for key points in an image and identify objects in the image through feature vector descriptors. It includes two steps:
[0079] The first step is to use the FAST algorithm to find key points in the image;
[0080] The second step is to use the BRIEF descriptor to convert the key points found in the first step into feature vectors to distinguish different objects;
[0081] The Hamming distance is used to calculate the number of different characters in corresponding positions of two strings of equal length. For binary strings A and B, the calculation formula is:
[0082]
[0083] Among them, 1(a i ≠b i ) is the indicator function, if a i Not equal to b i , its value is 1; otherwise it is 0;
[0084] In the optimization of key point matching robustness, the RANSAC algorithm removes the wrong matches when it is known that there are wrong matches, thereby estimating the homography matrix of the model transformation. The basic steps of the RANSAC algorithm are:
[0085] 1) Random sampling: randomly select a minimum subset from the data set that is sufficient to fit the model parameters;
[0086] 2) Model fitting: Fit the model using the randomly selected subset;
[0087] 3) Calculate inliers: Calculate the distance between each point in the entire data set and the fitted model. If the distance is within a predefined threshold, it is considered an inlier.
[0088] 4) Evaluate the model: record the number of inliers of the current model. If the number of inliers exceeds that of the previous best model, update the best model.
[0089] 5) Repeat: Repeat the above steps several times until the number of iterations is met;
[0090] 6) Model re-estimation: Re-estimate the model parameters using the best set of inliers.
[0091] In step S2, the method for generating and synthesizing training data by the video synthesis algorithm provided in this embodiment can easily adapt to different types of target objects and different real-world backgrounds, and has the function of adjustable parameters. It specifically includes the following steps:
[0092] S21, initializing the size, quantity, initial position, initial flight direction and initial speed of the artificial target in the first frame of the original background video.
[0093] Specifically, artificial targets refer to artificially synthesized spot-like objects that are used to simulate surveillance targets and are intended to imitate the target's foreseeable movement patterns and flight states.
[0094] Depending on the characteristics of the target, spots with different distributions are used to represent tiny targets. Usually, Gaussian two-dimensional distribution can be used for simulation, that is, the size of the artificial target is initialized by controlling the standard deviation of the two dimensions of the two-dimensional Gaussian distribution.
[0095] In order to better simulate artificial targets, the number, initial position, initial flight direction and initial speed of artificial targets can usually be randomly selected from a uniform distribution function that satisfies a specified parameter interval.
[0096] For the texture of artificial objects, Gaussian noise is usually used for modeling.
[0097] The distribution type and value range used in the above-mentioned generation of artificial targets need to be comprehensively optimized and valued in combination with the subsequent CNN training network.
[0098] When adding an artificial target to the original image, this embodiment uses the pixel value of the artificial target as the alpha channel to mix the background and the target. In this way, the artificial target with a lower texture mean value will appear lighter, that is, the contrast with the background is higher, and the artificial target with a higher texture mean value will appear heavier, that is, the contrast with the background is lower.
[0099] S22. After completing the initialization of the first frame, it is necessary to simulate the movement of the artificial target in each subsequent frame. The movement direction and movement speed of the artificial target can be calculated using the following formula:
[0100] υ t =υ t-1 +Δ v ,
[0101] θ t =θ t-1 +Δ θ ,
[0102] According to the position, speed and direction of the target in the previous frame, the speed and direction of the next frame are simulated. Then the new position of the target is calculated by the new speed and direction. For artificial targets that exceed the image boundary during the generation process, they are removed once they exceed the image boundary.
[0103] Based on the above method, artificial targets are synthesized from the collected original background video to form a new video sequence. At the same time, the generated artificial target images are saved separately as labels for subsequent model training.
[0104] S23, segment the generated video, take the image of n consecutive frames as a video sequence, and the label corresponding to each video sequence is the label of the last frame, and establish an initial data set with n-frame video sequences and the label of the last frame corresponding to each other. In order to effectively alleviate the problem of sample imbalance in the data set, this embodiment adopts a probabilistic method based on the number of targets in the field of view (Field of View, FoV) to construct a training set. The core idea of this strategy is that for each video sequence, the probability of being selected into the final data set is dynamically adjusted according to the number of artificial targets (i.e., the specific objects of concern in this embodiment) in its field of view. Specifically, the more video sequences with more targets in the field of view, the greater the probability of being selected, and vice versa. This approach is intended to ensure that the distribution of various types of samples in the training set is more balanced, and to avoid the model being overly biased towards a certain type of sample.
[0105] It is worth noting that in the implementation process, this embodiment also needs to specially handle those video sequences that have no target in the field of view. Although these sequences may not contain the target information that this embodiment currently focuses on, in some cases, they may also contribute to the generalization ability of the model or participate in the training as negative samples. Therefore, when constructing the data set, this embodiment needs to flexibly handle these target-free sequences, perhaps by reducing their probability of being selected, or treating them separately as specific types of samples, to ensure the comprehensiveness and effectiveness of the training process.
[0106] In order to emphasize moving objects in each frame, the video background estimation and difference method are used to process the video dataset. For each pixel in each video, the mean μ(x, y) and standard deviation σ(x, y) are calculated frame by frame, and then z-score normalization data processing is performed.
[0107]
[0108] Among them I(x,y,t),I bs (x, y, t) are the original pixel value at the coordinate x, y position at time t and the pixel value after background difference.
[0109] In step S3, the video continuous frame segmentation training model constructed in this embodiment adopts a CNN training network based on an improved U-Net, such as Figure 2 As shown, it includes an encoder and a decoder, wherein the encoder is responsible for extracting features from the input data, and the decoder processes the features output by the encoder to obtain an output segmentation map of a desired size.
[0110] The encoder includes at least several blocks consisting of a convolution layer, a batch normalization layer and a ReLU activation function, and a maximum pooling layer following each block. The convolution layer in the encoder is used to gradually extract low-level to high-level features of the input image; applying a batch normalization layer after the convolution layer can accelerate the training process and help prevent the gradient from disappearing or exploding, while also improving the generalization ability of the model; the ReLU activation function introduces nonlinearity to the network, allowing the network to learn more complex feature representations. Applying the ReLU activation function after the convolution layer can enhance the nonlinear modeling ability of the network; using the maximum pooling layer in the encoder can gradually reduce the size of the feature map and extract more abstract feature representations. At the same time, by reducing the number of parameters, overfitting problems can be prevented.
[0111] In this embodiment, the encoder includes 3 blocks consisting of convolutional layers, batch normalization layers, and ReLU activation functions, each of which is followed by a maximum pooling layer. The convolution kernel size in the first block is 3*3, the convolution kernel size in the second block is 2*2, and the size in the third block is 1*1.
[0112] In the decoder, the feature map obtained is upsampled using nearest neighbor interpolation and then input into a block containing a 3*3 kernel convolution layer, a batch normalization layer, and a ReLU activation function. The convolution layer, batch normalization layer, and ReLU activation function processing block are repeated twice. Considering that the target object is small, in order to ensure that the detailed information in the input video is used in the encoding process, a symmetric jump connection of the same size feature map is used between the output of the convolution layer of the encoder and decoder to fuse the underlying features with the high-level features and provide segmentation accuracy. The last layer of the decoder is a convolution layer, which contains a 1*1 kernel and a sigmoid activation function. In order to reduce the number of weight parameters and improve the calculation speed, the segmented image output by the last layer is half the size of the original frame image. Finally, the segmented map is directly upsampled using bilinear interpolation to achieve the same size of the output frame as the original frame image.
[0113] In this embodiment, a 1*1 convolution layer and a batch normalization layer are arranged between the encoder and the decoder of the CNN video continuous frame segmentation training model, which are used to pass the output of the encoder to the decoder.
[0114] In step S4, the generated data set sample space is divided into 50% as a training set, 25% as a validation set, and 25% as a test set. During the training process, the L2 loss function of the model output and the label is used as the optimization target to complete the training and determine the final network weight.
[0115] In step S5, Figure 3 As shown in the figure, in the process of real-time infrared small target detection, the video stabilization process is first performed on the continuous n frames. The specific method is as follows Figure 4 As shown, consistent with the aforementioned processing method of the original background video, an affine transformation is performed between each frame in the sequence and the first frame, that is, based on the same common reference frame, the video stabilization processing is realized. In order to estimate the affine transformation between two frames, this embodiment first detects the key points in the two frames and uses the ORB detector to calculate the descriptor of the picture frame. Then, the descriptors are compared using the Hamming distance to find the matching key point pairs, and the RANSAC algorithm is used to optimize the robustness of the key point matching.
[0116] Then, the background estimation and difference of n frames of video are performed. The processing method is the same as the above processing method. The input is based on the CNN target prediction network of continuous frame segmentation. In order to output the location of possible targets more accurately, the output image is thresholded and the location information of the target is output.
[0117] To further improve the accuracy of target estimation, the system may threshold the target estimation results. This step usually involves setting certain threshold conditions, and only when the target estimation results meet these conditions will they be considered valid. In this way, the system can eliminate some false detections or noise interference and improve the reliability of target estimation.
[0118] Finally, the system will output the processed target coordinates. These coordinates represent the exact position of the target object in the video frame. In video stabilization applications, these coordinate information can be used to adjust the transformation parameters of the video frame (such as translation, rotation, etc.) so that the position of the target object in the video remains relatively stable, thereby achieving video stabilization.
[0119] By adopting the method of generating videos, the problem of manual labeling of small moving targets in video datasets can be effectively solved. By adopting the method of simulating the generation of small targets, different small target detection application scenarios can be flexibly responded to by adjusting the small target simulation distribution function and parameter range, target texture, distribution number, motion mode and other simulation parameters. The video segmentation CNN training model is used to complete the parameter adjustment and training and complete the deployment.
[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.
Claims
1. An infrared target detection method based on generated dataset, Features: It includes the following steps: S1, collecting background videos in typical scenes and stabilizing the background videos; S2, using a target generation algorithm to initialize artificial target information in the first frame of the original background video sequence, and simulating the artificial target in each subsequent frame of the original background video sequence to obtain a synthetic video sequence; S3, constructing a video continuous frame segmentation training model, which at least includes an encoder and a decoder, wherein the encoder is used to extract features from input data, and the decoder is used to process features output by the encoder; S4, using the synthetic video sequence as a data set sample space to train the constructed video continuous frame segmentation training model; S5. Use the trained video continuous frame segmentation model to analyze the real-time infrared small target detection video and output the target's location information.
2. The infrared target detection method based on generating a data set according to claim 1, Features: In step S1, the following steps are included: First, background videos are collected based on real scenes. The selection of scenes takes into account typical backgrounds in various situations around the clock. During the collection process, ensure that there are no surveillance targets in the video. An affine transformation is performed between each frame and the first frame in the sequence of the collected original background video, that is, based on the same common reference frame, the video stabilization processing is achieved; the key points in the two frames are detected, and the descriptors of the picture frames are calculated using the ORB detector to estimate the affine transformation between the two frames; then, the descriptors are compared using the Hamming distance to find matching key point pairs, and the RANSAC algorithm is used to optimize the robustness of key point matching.
3. The infrared target detection method based on generating a data set according to claim 2, Features: In step S1, the ORB descriptor is used to quickly create feature vector descriptors for key points in the image, and identify objects in the image through feature vector descriptors, which includes two steps: The first step is to use the FAST algorithm to find key points in the image; The second step is to use the BRIEF descriptor to convert the key points found in the first step into feature vectors to distinguish different objects; The Hamming distance is used to calculate the number of different characters in corresponding positions of two strings of equal length. For binary strings A and B, the calculation formula is: Among them, 1(a i ≠b i ) is the indicator function, if a i Not equal to b i , its value is 1; otherwise, it is 0; In the optimization of key point matching robustness, the RANSAC algorithm removes the wrong matches when it is known that there are wrong matches, thereby estimating the homography matrix of the model transformation. The basic steps of the RANSAC algorithm are: 1) Random sampling: randomly select a minimum subset from the data set that is sufficient to fit the model parameters; 2) Model fitting: Fit the model using the randomly selected subset; 3) Calculate inliers: Calculate the distance between each point in the entire data set and the fitted model. If the distance is within a predefined threshold, it is considered an inlier. 4) Evaluate the model: record the number of inliers of the current model. If the number of inliers exceeds that of the previous best model, update the best model. 5) Repeat: Repeat the above steps several times until the number of iterations is met; 6) Model re-estimation: Re-estimate the model parameters using the best set of inliers.
4. The infrared target detection method based on generating a data set according to claim 1, Features: In step S2, the following steps are specifically included: S21, initializing the size, quantity, initial position, initial flight direction and initial speed of the artificial target in the first frame of the original background video; According to different target features, spots with different distributions are used to represent small targets, and Gaussian two-dimensional distribution is used for simulation, that is, the size of the artificial target is initialized by controlling the standard deviation of the two dimensions of the two-dimensional Gaussian distribution; the number, initial position, initial flight direction and initial speed of the artificial targets are randomly selected from the uniform distribution function that meets the specified parameter interval; the texture of the artificial target is modeled using Gaussian noise; the distribution type and value range used to generate the artificial target can be combined with the subsequent CNN training network for comprehensive optimization and value selection; When adding an artificial target to the original image, the pixel value of the artificial target is used as the alpha channel to blend the background and the target; S22. After completing the initialization of the first frame, simulate the movement of the artificial target in each subsequent frame, and the movement direction and movement speed of the artificial target are calculated using the following formula: u t =υ t-1 +△ υ′ i t =θ t-1 +△ θ′ According to the position, speed and direction of the target in the previous frame, the speed and direction of the next frame are simulated, and then the new position of the target is calculated by the new speed and direction, and the artificial targets beyond the image boundary are eliminated; Based on the above method, artificial targets are synthesized from the original background video to form a new video sequence. At the same time, the generated artificial target images are saved separately as labels for subsequent model training. S23, segmenting the generated video, taking n consecutive frames of images as a video sequence, the label corresponding to each video sequence is the label of the last frame, and establishing an initial data set with n-frame video sequences and the last frame label corresponding to each other; A probabilistic method based on the number of objects in the field of view is used to establish the training set. For each video sequence, the probability of it being selected into the final data set is dynamically adjusted according to the number of artificial objects in its field of view. That is, the more objects a video sequence has in its field of view, the greater its probability of being selected. The video background estimation and difference method are used to process the video dataset to emphasize the moving pairs in each frame. For each pixel of each video, the mean μ(x, y) and standard deviation σ(x, y) are calculated frame by frame, and then the z-score normalized data is processed: Among them I (x, y, t), I bs (x, y, t) are the original pixel value at the coordinate x, y position at time t and the pixel value after background difference.
5. The infrared target detection method based on generating a data set according to claim 1, Features: In step S3, the constructed video continuous frame segmentation training model adopts a CNN training network based on an improved U-Net, which includes an encoder and a decoder, wherein the encoder is responsible for feature extraction of input data, and the decoder processes according to the features output by the encoder, wherein the encoder includes at least a number of blocks consisting of a convolutional layer, a batch normalization layer and a ReLU activation function, and a maximum pooling layer following each block; The decoder includes at least several blocks consisting of an upsampling layer, a convolutional layer, a batch normalization layer and a ReLU activation function, and a convolutional layer and a sigmoid activation function in the last layer, wherein the upsampling layer upsamples the obtained feature map using nearest neighbor interpolation; Among them, the convolutional layer outputs of the encoder and decoder use symmetric jump connections with feature maps of the same size to fuse the low-level features with the high-level features; The size of the segmented image finally output by the decoder is half of the original frame image, and the obtained segmentation map is then upsampled using bilinear interpolation to ensure that the output frame is consistent with the original frame image size.
6. The infrared target detection method based on generating a data set according to claim 5, Features: The encoder consists of three blocks consisting of a convolutional layer, a batch normalization layer and a ReLU activation function, and a maximum pooling layer following each block, where the convolution kernel size in the first block is 3*3, the convolution kernel size in the second block is 2*2, and the size in the third block is 1*1.
7. The infrared target detection method based on generating a data set according to claim 5, Features: The decoder consists of two blocks consisting of an upsampling layer, a convolutional layer, a batch normalization layer, and a ReLU activation function.
8. The infrared target detection method based on generating a data set according to claim 5, Features: A convolution layer and a batch normalization layer are set between the encoder and the decoder, which are used to pass the output of the encoder to the decoder.
9. The infrared target detection method based on generating a data set according to claim 1, Features: In step S4, the generated data set sample space is divided into a training set, a validation set, and a test set in proportion. During the training process, the L2 loss function of the model output and the label is used as the optimization target to complete the training and determine the final network weight.
10. The infrared target detection method based on generating a data set according to claim 1, Features: In step S5, the following operations are specifically included: Get real-time infrared small target detection video; Perform video stabilization on n consecutive frames; Perform background estimation and subtraction on n frames of video; Then the video frame is input into the trained continuous frame segmentation based CNN target prediction network for target prediction; The target estimation result is thresholded, and when the target estimation result meets the threshold condition, the coordinate information is output.
Citation Information
Patent Citations
Infrared thermal imaging behavior intention analysis method and system
CN111612815A
Video processing method and device, electronic equipment and storage medium
CN113850108A
Digital human video generation method and device, electronic equipment and storage medium
CN113987268A
Video abnormal behavior detection method and system based on automatic encoder
CN114913599A
Lightweight infrared image pedestrian target detection method
CN116824630A
