Infrared target detection method based on generated dataset

Abstract

The present application relates to a kind of infrared target detection methods based on generating data set, belong to target detection field, it includes: collecting the background video under typical scene, and the background video is stabilized processing;Using target generation algorithm in the first frame of original background video sequence Initialization artificial target information, and by simulating artificial target in each frame of subsequent original background video sequence to obtain synthetic video sequence;Continuous frame segmentation training model of video is constructed, at least including encoder and decoder, wherein, the encoder is used to carry out feature extraction to input data, and the decoder is used to process the feature of encoder output;Synthetic video sequence is used as data set sample space to the continuous frame segmentation training model of video constructed and is trained;Using the continuous frame segmentation model of video trained to analyze real-time infrared small target detection video, and the position information of output target.

Description

Technical Field

[0001] This invention belongs to the field of target detection and relates to an infrared target detection method based on generated datasets. Background Technology

[0002] In certain surveillance scenarios equipped with infrared detectors, the target occupies a very small pixel area in the field of view when it appears, and the target may be moving rapidly. Therefore, to better monitor moving targets within the field of view, it is necessary to fuse information about its appearance and movement. Detection of small moving targets based on infrared detectors is an important research area, with applications including detecting long-range moving targets, studying the behavior of monitored targets, and using the behavior of monitored targets to determine their intentions. However, due to the inherent low resolution and blurred boundaries of infrared images, as well as the lack of unique shape and texture details for small targets, traditional image processing methods and modern target detection methods directly applying CNNs exhibit considerably low performance.

[0003] The prior art disclosed in CN117409192A is an infrared small target detection method and device based on data augmentation. It first uses a background filling method to obtain a clean background image and uses a diffusion model to generate a more complex and diverse background image. Then, it designs a target adaptive fusion method based on generative adversarial network to better fuse the target mask and the generated background image, which can generate a more realistic infrared small target augmented image.

[0004] However, model training typically requires large datasets, and manually labeling tiny moving targets in videos is extremely difficult and time-consuming. Since human observers struggle to detect minute moving objects in recorded videos, manually labeling large amounts of video is a challenging and error-prone task. Currently, there are very few publicly available datasets for labeling small targets in infrared surveillance. Therefore, for specific task scenarios, no training dataset is readily available. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an infrared target detection method based on a generated dataset. This invention proposes a novel method for model building, training, and evaluation. During the training phase, background video is collected and artificial target information is added to the background video to obtain a generated dataset. The model is then trained using this generated dataset. After training, the model is used to analyze the video information acquired by the infrared detector in real time to detect minute moving targets.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An infrared target detection method based on a generated dataset includes the following steps:

[0008] S1. Collect background videos in typical scenarios and stabilize the background videos;

[0009] S2. The artificial target information is initialized in the first frame of the original background video sequence using a target generation algorithm, and the synthetic video sequence is obtained by simulating the artificial targets in each frame of the subsequent original background video sequence.

[0010] S3. Construct a video continuous frame segmentation training model, which includes at least an encoder and a decoder. The encoder is used to extract features from the input data, and the decoder is used to process the features output by the encoder.

[0011] S4. Use the synthesized video sequence as the dataset sample space to train the constructed video continuous frame segmentation training model;

[0012] S5. The trained video continuous frame segmentation model is used to analyze the real-time infrared small target detection video and output the target's position information.

[0013] Furthermore, step S1 includes the following steps:

[0014] First, background videos are collected based on real-world scenarios. The selection of scenarios takes into account typical backgrounds under various conditions throughout the day, and the process of collecting videos ensures that there are no surveillance targets in the videos.

[0015] Affine transformation is performed between each frame of the acquired original background video sequence and the first frame, i.e., based on the same common reference frame, to achieve video stabilization. By detecting key points in the two frames and using an ORB detector to calculate the image frame descriptor, the affine transformation between the two frames is estimated. Then, the descriptors are compared using Hamming distance to find matching key point pairs, and the RANSAC algorithm is used to optimize the robustness of key point matching.

[0016] Furthermore, in step S1, the ORB descriptor is used to quickly create feature vector descriptors for key points in the image, and to identify objects in the image using these feature vector descriptors. This process includes two steps:

[0017] The first step is to use the FAST algorithm to find key points in the image;

[0018] The second step is to use the BRIEF descriptor to convert the key points detected in the first step into feature vectors, in order to distinguish different objects;

[0019] Hamming distance is used to calculate the number of distinct characters at corresponding positions in two strings of equal length. For binary strings A and B, the formula is:

[0020]

[0021] Among them, 1(a i ≠b i ) is an indicator function, if a i Not equal to b i Its value is 1 if it is true, otherwise it is 0.

[0022] In the robustness optimization of keypoint matching, the RANSAC algorithm removes erroneous matches when their existence is known, thereby estimating the homography matrix of the model transformation. The basic steps of the RANSAC algorithm are as follows:

[0023] 1) Random sampling: Randomly select a minimum subset from the dataset that is sufficient to fit the model parameters;

[0024] 2) Model fitting: The model is fitted using a randomly selected subset;

[0025] 3) Calculate inliers: Calculate the distance between each point in the entire dataset and the fitted model. If the distance is within a predefined threshold, it is considered an inlier.

[0026] 4) Evaluate the model: Record the number of interior points in the current model. If the number of interior points exceeds the number of interior points in the previous best model, then update the best model.

[0027] 5) Repeat: Repeat the above steps several times until the required number of iterations is met;

[0028] 6) Model re-estimation: Re-estimate the model parameters using the optimal set of interior points.

[0029] Furthermore, step S2 specifically includes the following steps:

[0030] S21. Initialize the size, number, initial position, initial flight direction and initial speed of the artificial target in the first frame of the original background video;

[0031] Depending on the target characteristics, spots with different distributions are used to represent tiny targets. A Gaussian two-dimensional distribution is used for simulation, that is, the size of the artificial target is initialized by controlling the standard deviation of the two dimensions of the two-dimensional Gaussian distribution. The number, initial position, initial flight direction and initial velocity of the artificial targets are randomly selected from a uniform distribution function that satisfies the specified parameter range. The texture of the artificial targets is modeled using Gaussian noise. The distribution type and value range used to generate the above artificial targets can be optimized and selected by combining with the subsequent CNN training network.

[0032] When adding an artificial object to the original image, the pixel value of the artificial object is used as the alpha channel to blend the background and the object;

[0033] S22. After completing the initialization of the first frame, the motion of the artificial target in each subsequent frame is simulated. The motion direction and velocity of the artificial target are calculated using the following formula:

[0034] υ t =υ t-1 +Δ υ ,

[0035] θ t =θ t-1 +Δ θ ,

[0036] Based on the position, speed and direction of the target in the previous frame, simulate the speed and direction of the target in the next frame, and then calculate the new position of the target using the new speed and direction. Artificial targets that are outside the image limits are removed.

[0037] Based on the above method, artificial targets are synthesized from the collected original background video to form a new video sequence. At the same time, the generated artificial target images are saved separately as labels for subsequent model training.

[0038] S23. Segment the generated video, and take n consecutive frames as a video sequence. The label corresponding to each video sequence is the label of the last frame. Establish an initial dataset with a one-to-one correspondence between the n-frame video sequence and the label of the last frame.

[0039] A probabilistic method based on the number of targets in the field of view is used to build the training set. For each video sequence, the probability of it being selected into the final dataset is dynamically adjusted according to the number of artificial targets in its field of view. That is, the more targets a video sequence has in its field of view, the greater its probability of being selected.

[0040] The video dataset is processed using video background estimation and differencing to emphasize motion pairs in each frame. For each video, the mean μ(x, y) and standard deviation σ(x, y) are calculated frame-by-frame for each pixel, followed by z-score normalization.

[0041]

[0042] where I(x, y, t), I b s(x, y, t) represent the original pixel value at time t and the pixel value after background subtraction, respectively, at coordinates x and y.

[0043] Furthermore, in step S3, the constructed video continuous frame segmentation training model adopts a CNN training network based on the improved U-Net, which includes an encoder and a decoder. The encoder is responsible for extracting features from the input data, and the decoder processes the features output by the encoder. The encoder includes at least several blocks consisting of a convolutional layer, a batch normalization layer and a ReLU activation function, and a max pooling layer following each block.

[0044] The decoder includes at least several blocks consisting of an upsampling layer, a convolutional layer, a batch normalization layer and a ReLU activation function, as well as a convolutional layer and a sigmoid activation function in the last layer. The upsampling layer upsamples the obtained feature map using nearest neighbor interpolation.

[0045] In this system, symmetric skip connections with the same-sized feature maps are used between the outputs of the convolutional layers of the encoder and decoder to fuse low-level features with high-level features.

[0046] The final output image of the decoder is half the size of the original frame image. Then, bilinear interpolation is used to upsample the obtained segmented image to ensure that the output frame is the same size as the original frame image.

[0047] The encoder consists of three blocks, each consisting of a convolutional layer, a batch normalization layer, and a ReLU activation function, followed by a max pooling layer after each block. The kernel size in the first block is 3*3, the kernel size in the second block is 2*2, and the kernel size in the third block is 1*1.

[0048] The decoder consists of two blocks, each consisting of an upsampling layer, a convolutional layer, a batch normalization layer, and a ReLU activation function.

[0049] A convolutional layer and a batch normalization layer are set between the encoder and the decoder, which are used to pass the encoder's output to the decoder.

[0050] Furthermore, in step S4, the generated dataset sample space is divided into training set, validation set and test set according to the proportion. During the training process, the L2 loss function of model output and label is used as the optimization objective to complete the training and determine the final network weights.

[0051] Furthermore, step S5 specifically includes the following operations:

[0052] Acquire real-time infrared small target detection video;

[0053] Perform video stabilization on n consecutive frames;

[0054] Perform background estimation and differencing on n frames of video;

[0055] Then, the video frames are input into the trained CNN target prediction network based on continuous frame segmentation to predict the target.

[0056] The target estimation results are thresholded, and the coordinate information is output when the target estimation results meet the threshold conditions.

[0057] The beneficial effects of this invention are as follows:

[0058] The video synthesis algorithm provided by this invention features adjustable parameters, easily adapting to different types of target objects and real-world backgrounds. By adding artificially synthesized speckled objects to simulate surveillance targets and employing methods such as Gaussian two-dimensional distribution for initialization, it achieves accurate simulation of target motion patterns and flight states. This method not only improves the diversity and realism of training data but also provides strong support for subsequent model training.

[0059] This invention, in simulating artificial targets, not only considers key factors such as the target's size, quantity, initial position, initial flight direction, and initial velocity, but also employs Gaussian noise for texture modeling, further improving the accuracy and realism of the simulation. Furthermore, by calculating the motion direction and velocity of the artificial targets, it achieves accurate simulation of target motion, ensuring the continuity and consistency of the synthesized video sequence.

[0060] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0062] Figure 1 This is the overall process of the infrared target detection method based on generated datasets of the present invention;

[0063] Figure 2 This is a schematic diagram of the structure of the CNN video continuous frame segmentation training model of the present invention;

[0064] Figure 3 This is a flowchart of the real-time detection process of the CNN video continuous frame segmentation training model of the present invention;

[0065] Figure 4 This is the video stabilization process of the present invention. Detailed Implementation

[0066] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0067] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0068] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0069] Please see Figures 1-4 This is an infrared target detection method based on generated datasets.

[0070] Example

[0071] This embodiment provides a detailed description of the specific implementation of the infrared target detection method based on generated datasets proposed in this invention, such as... Figure 1 As shown, it specifically includes the following steps:

[0072] S1. Collect background videos in typical scenarios and stabilize the background videos;

[0073] S2. The artificial target information is initialized in the first frame of the original background video sequence using a target generation algorithm, and the synthetic video sequence is obtained by simulating the artificial targets in each frame of the subsequent original background video sequence.

[0074] S3. Construct a video continuous frame segmentation training model, which includes at least an encoder and a decoder. The encoder is used to extract features from the input data, and the decoder is used to process the features output by the encoder.

[0075] S4. Use the synthesized video sequence as the dataset sample space to train the constructed video continuous frame segmentation training model;

[0076] S5. The trained video continuous frame segmentation model is used to analyze the real-time infrared small target detection video and output the target's position information.

[0077] In step S1, this embodiment first acquires background video based on a real-world scene. The scene selection should consider typical backgrounds under various weather conditions throughout the day, ensuring that there are no monitored targets in the video during the acquisition process. To eliminate potential jitter from the detector, this embodiment performs an affine transformation between each frame in the sequence and the first frame, i.e., based on the same common reference frame, to achieve video stabilization. To estimate the affine transformation between two frames, this embodiment first detects keypoints in the two frames and uses an ORB detector to calculate the image frame descriptors. Then, the descriptors are compared using Hamming distance to find matching keypoint pairs, and the RANSAC algorithm is used to robustly optimize the keypoint matching.

[0078] ORB descriptors are used to quickly create feature vector descriptors for key points in an image, and then identify objects in the image using these feature vector descriptors. This process involves two steps:

[0079] The first step is to use the FAST algorithm to find key points in the image;

[0080] The second step is to use the BRIEF descriptor to convert the key points detected in the first step into feature vectors, in order to distinguish different objects;

[0081] Hamming distance is used to calculate the number of distinct characters at corresponding positions in two strings of equal length. For binary strings A and B, the formula is:

[0082]

[0083] Among them, 1(a i ≠b i ) is an indicator function, if a i Not equal to b i Its value is 1 if it is true, otherwise it is 0.

[0084] In the robustness optimization of keypoint matching, the RANSAC algorithm removes erroneous matches when their existence is known, thereby estimating the homography matrix of the model transformation. The basic steps of the RANSAC algorithm are as follows:

[0085] 1) Random sampling: Randomly select a minimum subset from the dataset that is sufficient to fit the model parameters;

[0086] 2) Model fitting: The model is fitted using a randomly selected subset;

[0087] 3) Calculate inliers: Calculate the distance between each point in the entire dataset and the fitted model. If the distance is within a predefined threshold, it is considered an inlier.

[0088] 4) Evaluate the model: Record the number of interior points in the current model. If the number of interior points exceeds the number of interior points in the previous best model, then update the best model.

[0089] 5) Repeat: Repeat the above steps several times until the required number of iterations is met;

[0090] 6) Model re-estimation: Re-estimate the model parameters using the optimal set of interior points.

[0091] In step S2, the video synthesis algorithm provided in this embodiment generates and synthesizes training data in a way that can easily adapt to different types of target objects and different real-world backgrounds, and has adjustable parameters. Specifically, it includes the following steps:

[0092] S21. Initialize the size, number, initial position, initial flight direction, and initial speed of the artificial target in the first frame of the original background video.

[0093] Specifically, artificial targets refer to artificially synthesized speckled objects used to simulate surveillance targets, aiming to mimic the predictable movement patterns and flight states of the targets.

[0094] Depending on the target characteristics, spots with different distributions are used to represent small targets. Typically, a Gaussian two-dimensional distribution can be used for simulation, that is, the size of the artificial target is initialized by controlling the standard deviation of the two dimensions of the two-dimensional Gaussian distribution.

[0095] In addition, to better simulate artificial targets, the number, initial position, initial flight direction and initial velocity of artificial targets can usually be randomly selected from a uniform distribution function that satisfies a specified parameter range.

[0096] For textures of artificial objects, Gaussian noise is typically used for modeling.

[0097] The distribution type and value range used to generate the artificial targets mentioned above need to be optimized and adjusted in conjunction with the subsequent training of the CNN network.

[0098] When adding an artificial object to the original image, this embodiment uses the pixel value of the artificial object as the alpha channel to blend the background and the object. Thus, artificial objects with lower texture mean will appear lighter, meaning they have higher contrast with the background, while artificial objects with higher texture mean will appear darker, meaning they have lower contrast with the background.

[0099] S22. After completing the initialization of the first frame, it is necessary to simulate the motion of the artificial target in each subsequent frame. The motion direction and speed of the artificial target can be calculated using the following formula:

[0100] υ t =υ t-1 +Δ v ,

[0101] θ t =θ t-1 +Δ θ ,

[0102] Based on the target's position, speed, and direction in the previous frame, the speed and direction of movement in the next frame are simulated. Then, the target's new position is calculated using the new speed and direction. Any artificial targets that exceed the image boundaries during the generation process are removed once they fall outside the image boundaries.

[0103] Based on the above method, artificial targets are synthesized from the collected original background video to form a new video sequence. At the same time, the generated artificial target images are saved separately as labels for subsequent model training.

[0104] S23. Segment the generated video, treating n consecutive frames as a single video sequence. The label for each video sequence is the label of the last frame, creating an initial dataset with a one-to-one correspondence between the n-frame video sequences and the label of the last frame. To effectively alleviate the problem of imbalanced samples in the dataset, this embodiment employs a probabilistic method based on the number of targets within the Field of View (FoV) to construct the training set. The core idea of ​​this strategy is to dynamically adjust the probability of each video sequence being selected into the final dataset based on the number of artificial targets (i.e., the specific objects of interest in this embodiment) in its field of view. Specifically, video sequences with more targets in their field of view have a higher probability of being selected, and vice versa. This approach aims to ensure a more balanced distribution of samples across different classes in the training set, avoiding excessive bias in the model towards one class of samples.

[0105] It is worth noting that, in implementing this process, this embodiment also requires special handling of video sequences that have no targets at all in the field of view. While these sequences may not contain the target information currently of interest in this embodiment, they may, in some cases, contribute to the model's generalization ability or participate in training as negative samples. Therefore, when constructing the dataset, this embodiment needs to handle these targetless sequences flexibly, perhaps by reducing their selection probability or treating them separately as specific types of samples, to ensure the comprehensiveness and effectiveness of the training process.

[0106] To emphasize moving objects in each frame, video background estimation and differencing are used to process the video dataset. For each video, the mean μ(x,y) and standard deviation σ(x,y) are calculated frame by frame for each pixel, and then z-score normalization is performed.

[0107]

[0108] Among them I (x, y, t), I bs (x,y,t) represent the original pixel value at time t and the pixel value after background subtraction at coordinates x and y, respectively.

[0109] In step S3, the video continuous frame segmentation training model constructed in this embodiment uses a CNN training network based on an improved U-Net, such as... Figure 2 As shown, it includes an encoder and a decoder. The encoder is responsible for extracting features from the input data, and the decoder processes the features output by the encoder to obtain an output segmentation map of the required size.

[0110] The encoder comprises at least several blocks consisting of a convolutional layer, a batch normalization layer, and a ReLU activation function, followed by a max-pooling layer after each block. The convolutional layers in the encoder progressively extract low- to high-level features from the input image. Applying a batch normalization layer after the convolutional layers accelerates the training process, helps prevent gradient vanishing or exploding problems, and improves the model's generalization ability. The ReLU activation function introduces non-linearity into the network, enabling it to learn more complex feature representations; applying ReLU after the convolutional layers enhances the network's non-linear modeling capabilities. Using max-pooling layers in the encoder progressively reduces the size of the feature maps, extracting more abstract feature representations. Simultaneously, reducing the number of parameters helps prevent overfitting.

[0111] In this embodiment, the encoder includes three blocks consisting of convolutional layers, batch normalization layers, and ReLU activation functions, with each block followed by a max pooling layer. The convolutional kernel size in the first block is 3*3, the kernel size in the second block is 2*2, and the kernel size in the third block is 1*1.

[0112] In the decoder, the obtained feature map is upsampled using nearest-neighbor interpolation and then input into a block containing a 3x3 kernel convolutional layer, a batch normalization layer, and a ReLU activation function. This process is repeated twice. Considering the small size of the target object, to ensure that detailed information from the input video is used during encoding, a symmetric skip connection with the same-sized feature map is used between the outputs of the convolutional layers in both the encoder and decoder. This fuses low-level and high-level features, improving segmentation accuracy. The final layer of the decoder is a convolutional layer containing a 1x1 kernel and a sigmoid activation function. To reduce the number of weight parameters and improve computation speed, the size of the segmented image output from the final layer is half that of the original frame image. Finally, bilinear interpolation is used to directly upsample the obtained segmented image, ensuring that the output frame is the same size as the original frame image.

[0113] In this embodiment, a 1*1 convolutional layer and a batch normalization layer are set between the encoder and decoder of the CNN video continuous frame segmentation training model, which are used to pass the output of the encoder to the decoder.

[0114] In step S4, the generated dataset sample space is divided into three groups: 50% for training, 25% for validation, and 25% for testing. During training, the L2 loss function of the model output and labels is used as the optimization objective to complete the training and determine the final network weights.

[0115] In step S5, as Figure 3 As shown, in the real-time infrared small target detection process, video stabilization is first performed on n consecutive frames, and the specific method is as follows: Figure 4 As shown, consistent with the aforementioned method for processing the original background video, an affine transformation is performed between each frame in the sequence and the first frame, i.e., based on the same common reference frame, to achieve video stabilization. To estimate the affine transformation between two frames, this embodiment first detects keypoints in both frames and uses an ORB detector to calculate the image frame descriptors. Then, the descriptors are compared using Hamming distance to find matching keypoint pairs, and the RANSAC algorithm is used to robustly optimize the keypoint matching.

[0116] Next, background estimation and differencing are performed on the n-frame video. The processing method is the same as the previous one. The input is a CNN target prediction network based on continuous frame segmentation. In order to output the possible target location more accurately, thresholding processing is added to the output image to output the target location information.

[0117] To further improve the accuracy of target estimation, the system may perform thresholding on the target estimation results. This step typically involves setting certain threshold conditions; only when the target estimation results meet these conditions are they considered valid. In this way, the system can eliminate some false detections or noise interference, improving the reliability of target estimation.

[0118] Finally, the system outputs the processed target coordinates. These coordinates represent the precise location of the target object within the video frame. In video stabilization applications, this coordinate information can be used to adjust the transformation parameters of the video frame (such as translation and rotation) to keep the target object's position relatively stable in the video, thereby achieving video image stabilization.

[0119] By using video generation, the problem of manually labeling tiny moving targets in video datasets can be effectively solved. Furthermore, by simulating the generation of small targets, different small target detection application scenarios can be flexibly addressed by adjusting the simulated distribution function and parameter range, target texture, distribution quantity, motion mode, and other simulated parameters. Finally, the deployment is completed by training the model through video segmentation CNN to achieve parameter tuning.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An infrared target detection method based on a generated data set, characterized by: It includes the following steps: S1. Collect background videos in typical scenarios and stabilize the background videos; S2. The artificial target information is initialized in the first frame of the original background video sequence using a target generation algorithm, and the synthetic video sequence is obtained by simulating the artificial targets in each frame of the subsequent original background video sequence. In the first frame of the background video, the target size is set using a Gaussian two-dimensional distribution, and the quantity, position, direction, and velocity are randomly selected using a uniform distribution. Gaussian noise is used to construct the texture, and the target is fused with the background using the alpha channel to complete the initialization of the artificial target. In subsequent frames, based on the state of the previous frame, the motion parameters of the target are updated step by step according to the velocity increment and direction increment, thereby calculating the new position of each frame and removing out-of-bounds targets, thus generating a complete synthetic video sequence. At the same time, the artificial target image of each frame is saved separately as a training label. The images of n consecutive frames are used as a video sequence, and the label corresponding to each video sequence is the label of the last frame, thus establishing an initial dataset with a one-to-one correspondence between the n-frame video sequence and the label of the last frame. S3. Construct a video continuous frame segmentation training model, which includes at least an encoder and a decoder. The encoder is used to extract features from the input data, and the decoder is used to process the features output by the encoder. S4. Use the synthesized video sequence as the dataset sample space to train the constructed video continuous frame segmentation training model; S5. The trained video continuous frame segmentation model is used to analyze the real-time infrared small target detection video and output the target's position information.

2. The infrared target detection method based on generated datasets according to claim 1, characterized in that: Step S1 includes the following steps: First, background videos are collected based on real-world scenarios. The selection of scenarios takes into account typical backgrounds under various conditions throughout the day, and the process of collecting videos ensures that there are no surveillance targets in the videos. Affine transformation is performed between each frame of the acquired original background video sequence and the first frame, i.e., based on the same reference frame, to achieve video stabilization. By detecting key points in the two frames and using an ORB detector to calculate the image frame descriptor, the affine transformation between the two frames is estimated. Then, the descriptors are compared using Hamming distance to find matching key point pairs, and the RANSAC algorithm is used to optimize the robustness of key point matching.

3. The infrared target detection method based on generated datasets according to claim 2, characterized in that: In step S1, the ORB descriptor is used to quickly create feature vector descriptors for key points in an image, and then identify objects in the image using these feature vector descriptors. This process includes two steps: The first step is to use the FAST algorithm to find key points in the image; The second step is to use the BRIEF descriptor to convert the key points detected in the first step into feature vectors, in order to distinguish different objects; Hamming distance is used to calculate the number of distinct characters at corresponding positions in two strings of equal length. For binary strings... A and B The calculation formula is as follows: in, It is an indicator function, if Not equal to Its value is 1 if it is true, otherwise it is 0. In the robustness optimization of keypoint matching, the RANSAC algorithm removes erroneous matches when their existence is known, thereby estimating the homography matrix of the model transformation. The basic steps of the RANSAC algorithm are as follows: 1) Random sampling: Randomly select a minimum subset from the dataset that is sufficient to fit the model parameters; 2) Model fitting: The model is fitted using a randomly selected subset; 3) Calculate inliers: Calculate the distance between each point in the entire dataset and the fitted model. If the distance is within a predefined threshold, it is considered an inlier. 4) Evaluate the model: Record the number of interior points in the current model. If the number of interior points exceeds the number of interior points in the previous best model, then update the best model. 5) Repeat: Repeat the above steps several times until the required number of iterations is met; 6) Model re-estimation: Re-estimate the model parameters using the optimal set of interior points.

4. The infrared target detection method based on generated datasets according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21. Initialize the size, number, initial position, initial flight direction and initial speed of the artificial target in the first frame of the original background video; Depending on the target characteristics, spots with different distributions are used to represent tiny targets. A Gaussian two-dimensional distribution is used for simulation, that is, the size of the artificial target is initialized by controlling the standard deviation of the two dimensions of the two-dimensional Gaussian distribution. The number, initial position, initial flight direction and initial velocity of the artificial targets are randomly selected from a uniform distribution function that satisfies the specified parameter range. The texture of the artificial targets is modeled using Gaussian noise. The distribution type and value range used to generate the above artificial targets can be optimized and selected by combining with the subsequent CNN training network. When adding an artificial object to the original image, the pixel value of the artificial object is used as the alpha channel to blend the background and the object; S22. After completing the initialization of the first frame, the motion of the artificial target in each subsequent frame is simulated. The motion direction and velocity of the artificial target are calculated using the following formula: Based on the position, speed and direction of the target in the previous frame, simulate the speed and direction of the target in the next frame, and then calculate the new position of the target using the new speed and direction. Artificial targets that are outside the image limits are removed. Based on the above method, artificial targets are synthesized from the collected original background video to form a new video sequence. At the same time, the generated artificial target images are saved separately as labels for subsequent model training. S23. Segment the generated video, and take n consecutive frames as a video sequence. The label corresponding to each video sequence is the label of the last frame. Establish an initial dataset with a one-to-one correspondence between the n-frame video sequence and the label of the last frame. A probabilistic method based on the number of targets in the field of view is used to build the training set. For each video sequence, the probability of it being selected into the final dataset is dynamically adjusted according to the number of artificial targets in its field of view. That is, the more targets a video sequence has in its field of view, the greater its probability of being selected. The video dataset is processed using video background estimation and differencing to emphasize moving objects in each frame. The mean is calculated frame-by-frame for each pixel in each video. and standard deviation Then, z-score normalization data processing is performed: in , Coordinates x , y Position Time t Original pixel values, pixel values ​​after background subtraction.

5. The infrared target detection method based on generated datasets according to claim 1, characterized in that: In step S3, the constructed video continuous frame segmentation training model adopts a CNN training network based on the improved U-Net, which includes an encoder and a decoder. The encoder is responsible for extracting features from the input data, and the decoder processes the features output by the encoder. The encoder includes at least several blocks consisting of a convolutional layer, a batch normalization layer and a ReLU activation function, and a max pooling layer following each block. The decoder includes at least several blocks consisting of an upsampling layer, a convolutional layer, a batch normalization layer and a ReLU activation function, as well as a convolutional layer and a sigmoid activation function in the last layer. The upsampling layer upsamples the obtained feature map using nearest neighbor interpolation. In this system, symmetric skip connections with the same-sized feature maps are used between the outputs of the convolutional layers of the encoder and decoder to fuse low-level features with high-level features. The final segmented image output by the decoder is half the size of the original frame image. Then, bilinear interpolation is used to upsample the obtained segmented image to ensure that the output frame is the same size as the original frame image.

6. The infrared target detection method based on generated datasets according to claim 5, characterized in that: The encoder consists of three blocks, each consisting of a convolutional layer, a batch normalization layer, and a ReLU activation function, followed by a max pooling layer after each block. The kernel size in the first block is 3*3, the kernel size in the second block is 2*2, and the kernel size in the third block is 1*1.

7. The infrared target detection method based on a generated dataset according to claim 5, characterized in that: The decoder consists of two blocks, each consisting of an upsampling layer, a convolutional layer, a batch normalization layer, and a ReLU activation function.

8. The infrared target detection method based on generated datasets according to claim 5, characterized in that: A convolutional layer and a batch normalization layer are set between the encoder and the decoder, which are used to pass the encoder's output to the decoder.

9. The infrared target detection method based on generated datasets according to claim 1, characterized in that: In step S4, the generated dataset sample space is divided into training set, validation set and test set according to the proportion. During the training process, the L2 loss function of model output and label is used as the optimization objective to complete the training and determine the final network weights.

10. The infrared target detection method based on generated datasets according to claim 1, characterized in that: Step S5 specifically includes the following operations: Acquire real-time infrared small target detection video; Perform video stabilization on n consecutive frames; Perform background estimation and differencing on n frames of video; Then, the video frames are input into the trained CNN target prediction network based on continuous frame segmentation to predict the target. The target estimation results are thresholded, and the coordinate information is output when the target estimation results meet the threshold conditions.

Images