Infrared image small target tracking method and system based on image local enhancement and 3D sparse convolutional network

Abstract

The invention discloses an infrared image small target tracking method and system based on image local enhancement and a 3D sparse convolutional network, and relates to the technical field of infrared small target tracking detection. The objective of the invention is to solve the problem that the current infrared small target features are not obvious and the background is complex and difficult to detect, and to improve the tracking precision and shorten the tracking time. The method comprises the following steps of: (1) enhancing an infrared target of an infrared image through an image spatial filter and simultaneously suppressing a background; (2) carrying out sparse sampling on the enhanced infrared image to generate sparse data; (3) stacking the sparse data into a three-dimensional tensor according to a time dimension; (4) carrying out cutting, superposition and rotation on the three-dimensional tensor to carry out data enhancement; and (5) segmenting the input sparse data by using the 3D sparse convolutional network to extract the trajectory. According to the method, high-precision tracking can be carried out on the small infrared target with track shielding under the conditions that no initial position mark exists and the background is complex.

Description

Technical Field

[0001] The present invention relates to the field of image processing and deep learning technology, and in particular to a method and system for tracking small targets in infrared images based on local image enhancement and a 3D sparse convolutional network. Background Art

[0002] Infrared small target tracking is an important research direction in the field of computer vision and image processing. With the continuous advancement of infrared imaging technology, infrared cameras have become important equipment widely used in military, security, industrial and medical fields. These infrared cameras can capture the infrared radiation emitted by the target, so they are widely used in target detection and tracking in low light, night and harsh environments. Traditional visible light target tracking technology faces some challenges in infrared small target tracking, such as low contrast between the target and the background, blurred target outline, dynamic occlusion and other problems. Therefore, the research on infrared small target tracking algorithms has become particularly important. These algorithms need to combine the characteristics of infrared images, such as temperature distribution, radiation intensity, etc., to design effective target detection and tracking methods to achieve accurate tracking of infrared small targets.

[0003] In recent years, with the development of deep learning technology, infrared small target tracking algorithms based on deep learning have made significant progress. Deep learning technology can automatically learn the feature representation of the target from the data, thereby improving the accuracy and robustness of target detection and tracking. At the same time, various novel neural network structures and training methods have also emerged, providing rich possibilities for the improvement and optimization of infrared small target tracking algorithms. However, most of the existing tracking methods require marking the initial position of the infrared small target to obtain the subsequent trajectory, which is inconvenient in practical applications. Summary of the invention

[0004] The present invention mainly solves the problem that the unknown target position is difficult to track and the background is complex and occluded and difficult to detect in the prior art, and further provides an infrared image small target tracking method and system based on image local enhancement and 3D sparse convolution network. The present invention can accurately extract the infrared small target and effectively suppress the background.

[0005] The technical solution adopted by the present invention is: a method for tracking small targets in infrared images based on local image enhancement and 3D sparse convolutional network, comprising the following steps:

[0006] Step 1, the infrared image is passed through an image spatial filter to enhance the infrared target and suppress the background;

[0007] Step 2, sparse sampling the enhanced infrared image to generate sparse data;

[0008] Step 3, stack the sparse data into a three-dimensional tensor according to the time dimension;

[0009] Step 4: crop, stack, and rotate the three-dimensional tensor for data enhancement;

[0010] Step 5: Use the 3D sparse convolutional network to segment the input sparse data and extract the trajectory.

[0011] Furthermore, in step 1, the specific implementation method of enhancing the infrared target and suppressing the background by passing the infrared image through the image space filter is as follows:

[0012] Step 1.1: The grayscale value of the infrared small target is high, often much higher than the background, but there is a large bright background. Local contrast suppression is used. An N×N sliding window is used to traverse the infrared image from left to right and from top to bottom according to the step length of step to obtain a set of slice data. Step is usually 3 and N is usually 9.

[0013] Y1,Y2,…,Y n (1)

[0014] where Y n is the nth slice, of size N×N

[0015] Step 1.2: Perform local target enhancement and background suppression on each slice data and restore the original image size:

[0016]

[0017] Where i∈1,…,n, δ is a minimum value, usually 0.000001, α and θ are two hyperparameters, usually 2 and 1, and finally the original image size Y is obtained through resizing operation.

[0018] Step 1.3, obtain the feature data of the global image, and perform target enhancement and background suppression on the global image:

[0019]

[0020] where Y global is the globally enhanced image, and Laplacian(Y) is the Laplacian operator.

[0021] Furthermore, the specific implementation method of sparse sampling the enhanced infrared image in step 2 to generate sparse data is as follows:

[0022] Step 2.1, calculate the average background intensity threshold of the image obtained in step 1.3:

[0023]

[0024] Where λ is a hyperparameter, usually set to 2;

[0025] Step 2.2: Use the average background intensity obtained in step 2.1 as the threshold to obtain the sparse image data Y sparse :

[0026]

[0027] Furthermore, the specific implementation method of stacking the sparse data into a three-dimensional tensor according to the time dimension in step 3 is as follows:

[0028] Step 3.1, stack all sparse data of the same sequence into a three-dimensional sparse tensor according to the time dimension, and the points with values ​​in the tensor account for less than 10% of the entire tensor;

[0029] Step 3.2, resample the W and H dimensions of the sparse data to 256 as required;

[0030] Furthermore, the specific implementation method of cropping, superimposing, and rotating the three-dimensional tensor for data enhancement in step 4 is as follows:

[0031] Step 4.1: Cut the stacked three-dimensional sparse tensors of the same sequence in the time dimension according to the step length S to obtain a set of three-dimensional tensors of size W×H×S, where W is the width of the original image, H is the length of the original image, both are 256, and S is usually 50;

[0032] Step 4.2, randomly extract and rotate the tensor obtained in step 4.1, and extract two-thirds of the tensor, and superimpose them two by two to obtain more complex sparse data;

[0033] Furthermore, the specific implementation method of using the 3D sparse convolutional network to segment the input sparse data and extract the trajectory in step 5 is as follows:

[0034] Step 5.1, encode the tensors of multiple trajectories obtained in step 4.2, retain only the intensity values ​​and coordinates of the locations with data, and save them as a dictionary;

[0035] In step 5.2, the dictionary is input into the downsampling part of the 3D sparse convolutional network in batches of size batch_size for encoding. The batch_size is 2. The encoding part consists of four convolutional modules, each of which consists of several sparse convolutional modules. A sparse convolutional module consists of a convolutional layer, a BN layer, and a Relu activation layer. The first convolutional module consists of a sparse convolutional module, the convolutional layer input channel is 1, the output channel is 16, the convolution kernel size is 3, and the stride is 1. The second convolutional module consists of a sparse convolutional module, the convolution kernel side length is 3, the stride is 1, the padding is 1, and the input and output channels are 16. The third convolution module consists of 3 sparse convolution modules. The first sparse convolution module has a convolution kernel side length of 3, a stride of 2, a padding of 1, an input channel of 16, and an output channel of 32; the second sparse convolution module has a convolution kernel side length of 3, a stride of 1, a padding of 1, an input channel of 32, and an output channel of 32; the third sparse convolution module has a convolution kernel side length of 3, a stride of 1, a padding of 1, an input channel of 32, and an output channel of 32. The fourth convolution module consists of 3 sparse convolution modules. The first sparse convolution module has a convolution kernel side length of 3, a stride of 2, a padding of 1, an input channel of 32, and an output channel of 64; the second sparse convolution module has a convolution kernel side length of 3, a stride of 1, a padding of 1, an input channel of 64, and an output channel of 64; the third sparse convolution module has a convolution kernel side length of 3, a stride of 1, a padding of 1, an input channel of 64, and an output channel of 64. Get high-level features after dimensionality upgrade;

[0036] Step 5.3, input the high-level features after dimensionality upgrade into the upsampling module, of which there are four upsampling modules in total, each of which includes splicing and convolution parts, and also includes three deconvolution and three splicing parts, where the features output in each deconvolution are spliced ​​with the convolution output features of the same size obtained in step 5.2 and input into the next deconvolution layer to obtain output feature values ​​of the same size. The first upsampling module includes three sparse convolution modules, the first is a residual convolution module, which includes two layers of convolution layers with 64 input and output channels, a convolution kernel size of 3, and a padding of 1; the second is a 3D sparse convolution module with an input channel of 128, an output of 64, a convolution kernel size of 3, a step size of 1, and a padding of 1; the third layer is a 3D desparse convolution module with an input channel of 64, an output channel of 64, a convolution kernel size of 3, and a step size of 1. The second upsampling module contains three sparse convolution modules. The first one is a residual convolution module, which contains two layers of convolution layers with 64 input and output channels, a convolution kernel size of 3, and a padding of 1; the second one is a 3D sparse convolution module with 128 input channels, 64 output channels, a convolution kernel size of 3, a stride of 1, and a padding of 1; the third layer is a 3D desparse convolution module with 64 input channels, 32 output channels, a convolution kernel size of 3, and a stride of 1. The third upsampling module contains three sparse convolution modules. The first one is a residual convolution module, which contains two layers of convolution layers with 32 input and output channels, a convolution kernel size of 3, and a padding of 1; the second one is a 3D sparse convolution module with 64 input channels, 32 output channels, a convolution kernel size of 3, a stride of 1, and a padding of 1; the third layer is a 3D desparse convolution module with 32 input channels, 16 output channels, a convolution kernel size of 3, and a stride of 1. The fourth upsampling module contains two sparse convolution modules. The first one is a residual convolution module, which contains two layers of convolution layers with 16 input and output channels, a convolution kernel size of 3, and a padding of 1. The second one is a 3D sparse convolution module with 32 input channels, 16 output channels, a convolution kernel size of 3, a stride of 1, and a padding of 1. Finally, the dictionary is output.

[0037] In step 5.4, the obtained dictionary is decoded into sparse data of the original size, and sliced ​​to obtain each image containing only the target, thereby obtaining the target trajectory.

[0038] Step 5.5: Set the network learning rate to 0.0001 and decay it exponentially after each iteration. Initialize the parameters with alpha parameter of 0.25 and gamma of 2.0. Optimize the model using the Focal loss function. Train the training set for 200 iterations.

[0039] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0040] The infrared image small target tracking method based on local image enhancement and 3D sparse convolutional network proposed by the present invention first decomposes the original image into image blocks, suppresses the local background in each image block, calculates the features of the global image, and performs target enhancement and background suppression on the global image; removes the background of the enhanced image to obtain sparse data; then stacks the obtained sparse data according to the time dimension to obtain a 3D sparse tensor; cuts the tensor according to the time dimension to obtain a series of tensor blocks of the same length, and rotates and superimposes a part of the tensor blocks for data enhancement; outputs the data-enhanced tensor in batches of a certain size to the 3D sparse network, encodes it to obtain high-level features and then decodes it, and finally outputs a sparse tensor containing only the target, slices the tensor to achieve target detection, and obtains the infrared small target trajectory. The present invention enhances the target through local global filtering, samples the data to sparseness, uses a neural network for detection, effectively utilizes the sparsity of the data, and improves the detection speed and detection accuracy. At the same time, the present invention directly detects the time series without the need to mark the initial position of the target in advance, which greatly improves the practicality. The present invention can accurately extract small infrared targets and effectively suppress the background, and can perform high-precision tracking of small infrared targets with trajectory obstruction in the absence of initial position marks and under complex backgrounds.

[0041] The present invention adopts advanced infrared small target tracking technology, and does not need to determine the initial position of the target. Traditional target tracking technology usually requires knowing the location information of the target in advance before tracking. However, the present invention uses time dimension information to track the target immediately after it appears, without determining the location of the target in advance, thereby greatly improving the efficiency and accuracy of tracking. Secondly, the present invention adopts 3D sparse convolution, which can quickly and accurately capture the location information of the target and reduce the computing power requirements. Traditional target tracking technology is often affected by factors such as ambient light and background interference, resulting in low tracking accuracy. However, the present invention can effectively reduce the influence of these interference factors by adopting a deep learning algorithm, and improve the accuracy and stability of tracking. In addition, the present invention has strong anti-interference ability and adaptability. Since infrared technology has strong penetration and anti-interference ability, even in complex environments, such as at night, in bad weather or under complex backgrounds, the present invention can still stably and reliably track targets, and has strong applicability and practicality. In summary, the present invention not only overcomes the limitation of traditional target tracking technology that requires the target position to be determined in advance, but also has high precision, high stability and strong anti-interference ability. It can realize fast and accurate tracking of small targets in various complex environments, and has broad application prospects and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the method of the present invention;

[0043] Figure 2 It is a schematic diagram of the 3D sparse convolutional network structure;

[0044] Figure 3 This is the detection result diagram of the infrared small target;

[0045] Figure 4 This is the infrared target tracking result diagram. DETAILED DESCRIPTION

[0046] Below is a combination of the embodiments and the attached Figure 1-3 The present invention is described in further detail, but the embodiments of the present invention are not limited thereto.

[0047] This embodiment takes the infrared time-sensitive target detection and tracking dataset for air-to-ground applications as an example to explain in detail the process of using the present invention to detect infrared small targets. The dataset contains 87 infrared sequences, each with 250 images. This dataset contains 21,750 frames of images, 89,174 targets, 87 video sequences, and 393 target trajectories, which is relatively large in scale. Each infrared image in the dataset has a resolution of 640×480 pixels, 8-bit bit depth, and a size of approximately 300KB.

[0048] according to Figure 1 As shown in the flowchart, a method for tracking small targets in infrared images based on local image enhancement and 3D sparse convolutional network includes the following steps:

[0049] Step 1: local contrast suppression is adopted, and an N×N sliding window is used to traverse the infrared image from left to right and from top to bottom according to the step length of step to obtain a set of slice data, where N is usually 9 and step is usually 3;

[0050] Y1,Y2,…,Y n (1)

[0051] where Y n For the nth slice, the size is N×N, local target enhancement and background suppression are performed on each slice data, and the original image size is restored:

[0052]

[0053] Among them, i∈1,…,n, δ is a minimum value, usually 0.000001, α,θ are two hyperparameters, usually 2 and 1, and finally the original image size Y is obtained through the resize operation. The feature data of the global image is obtained, and the target is enhanced and the background is suppressed:

[0054]

[0055] where Yglobal is the globally enhanced image, and Laplacian(Y) is the Laplacian operator.

[0056] Step 2: Calculate the average background intensity threshold:

[0057]

[0058] Where λ is 2, and the average background intensity obtained in step 2.1 is used as the threshold to obtain the sparse image data Y sparse :

[0059]

[0060] Step 3: stack all sparse data of the same sequence into a three-dimensional sparse tensor according to the time dimension. The points with values ​​in the tensor account for less than 10% of the entire tensor. Resample the W and H dimensions of the sparse data to 256 as required.

[0061] Step 4: Cut the stacked three-dimensional sparse tensors of the same sequence in the time dimension according to the step length S to obtain a set of three-dimensional tensors of size W×H×S, where W is the width of the original image, H is the length of the original image, both are 256, and S is usually 50; randomly extract the tensors obtained in step 4.1 and rotate them, and extract two-thirds of the tensors, and stack them two by two to obtain more complex sparse data;

[0062] Step 5: Encode the tensors of multiple trajectories obtained in step 4.2, retain only the intensity values ​​and coordinates of the locations with data, and save them as dictionaries; input the dictionaries into the downsampling part of the 3D sparse convolutional network in batches of size batch_size for encoding, batch_size is 2, and the encoding part consists of four convolution modules, each of which consists of several sparse convolution modules, and a sparse convolution module consists of a convolution layer, a BN layer, and a Relu activation layer. The first convolution module consists of a sparse convolution module, the convolution layer input channel is 1, the output channel is 16, the convolution kernel size is 3, and the stride is 1. The second convolution module consists of a sparse convolution module, the convolution kernel side length is 3, the stride is 1, the padding is 1, and the input and output channels are 16. The third convolution module consists of 3 sparse convolution modules. The first sparse convolution module has a convolution kernel side length of 3, a stride of 2, a padding of 1, an input channel of 16, and an output channel of 32; the second sparse convolution module has a convolution kernel side length of 3, a stride of 1, a padding of 1, an input channel of 32, and an output channel of 32; the third sparse convolution module has a convolution kernel side length of 3, a stride of 1, a padding of 1, an input channel of 32, and an output channel of 32. The fourth convolution module consists of 3 sparse convolution modules. The first sparse convolution module has a convolution kernel side length of 3, a stride of 2, a padding of 1, an input channel of 32, and an output channel of 64; the second sparse convolution module has a convolution kernel side length of 3, a stride of 1, a padding of 1, an input channel of 64, and an output channel of 64; the third sparse convolution module has a convolution kernel side length of 3, a stride of 1, a padding of 1, an input channel of 64, and an output channel of 64. Get the high-level features after dimensionality upgrade. Input the high-level features after dimensionality upgrade into the upsampling module, which has a total of four upsampling modules. Each upsampling module includes splicing and convolution parts, and also includes three deconvolution and three splicing parts. The features output in each deconvolution are spliced ​​with the convolution output features of the same size obtained in step 5.2 and input into the next deconvolution layer to obtain output feature values ​​of the same size. The first upsampling module includes three sparse convolution modules. The first is a residual convolution module, which includes two layers of convolution layers with 64 input and output channels, a convolution kernel size of 3, and a padding of 1; the second is a 3D sparse convolution module with an input channel of 128, an output of 64, a convolution kernel size of 3, a step size of 1, and a padding of 1; the third layer is a 3D desparse convolution module with an input channel of 64, an output channel of 64, a convolution kernel size of 3, and a step size of 1.The second upsampling module contains three sparse convolution modules. The first one is a residual convolution module, which contains two layers of convolution layers with 64 input and output channels, a convolution kernel size of 3, and a padding of 1; the second one is a 3D sparse convolution module with 128 input channels, 64 output channels, a convolution kernel size of 3, a stride of 1, and a padding of 1; the third layer is a 3D desparse convolution module with 64 input channels, 32 output channels, a convolution kernel size of 3, and a stride of 1. The third upsampling module contains three sparse convolution modules. The first one is a residual convolution module, which contains two layers of convolution layers with 32 input and output channels, a convolution kernel size of 3, and a padding of 1; the second one is a 3D sparse convolution module with 64 input channels, 32 output channels, a convolution kernel size of 3, a stride of 1, and a padding of 1; the third layer is a 3D desparse convolution module with 32 input channels, 16 output channels, a convolution kernel size of 3, and a stride of 1. The fourth upsampling module contains two sparse convolution modules. The first is a residual convolution module, which contains two layers of convolution layers with 16 input and output channels, a convolution kernel size of 3, and a padding of 1. The second is a 3D sparse convolution module with 32 input channels, 16 outputs, a convolution kernel size of 3, a stride of 1, and a padding of 1. Finally, the dictionary is output. The obtained dictionary is decoded into sparse data of the original size, sliced, and each image containing only the target is obtained to obtain the target trajectory.

[0063] Step 6: Divide the 1st to 76th segments of the infrared time-sensitive target detection and tracking dataset for air-to-ground applications into training sets and validation sets, and divide segments 77 to 87 into test sets. Set the network learning rate to 0.0001 and decay exponentially after each iteration. Initialize the parameters with alpha parameter of 0.25 and gamma of 2.0. Use the focal loss function to optimize the model. Invest the training set in 200 iterations of training.

[0064] The experimental environment used is as follows:

[0065] Hardware environment: Intel Xeon CPU, Quadro GV100 GPU

[0066] Software environment: Python 3.8, pytorch 1.10.0

[0067] The model obtained by the validation set in 200 iterations of training is verified, and the model with the lowest loss function value is used as the optimal network parameter model. The test set is then used to output the detection results to verify the actual detection performance of the model. The test set image is put into the optimal network parameter model to obtain a dictionary containing only the target. The dictionary is decoded according to the structure of the original tensor and restored to the size of the original image, and the detected small target image is obtained as shown in the figure. Figure 3 , Figure 4As shown, it can be seen that the present invention has good performance in tracking and detecting infrared weak targets.

[0068] The above are preferred embodiments of the present invention, but it is worth emphasizing that the implementation of the present invention is not limited to the above examples. Any adjustment, modification, substitution, combination or simplification that does not deviate from the core essence and principle of the present invention shall be regarded as an equivalent alternative and shall be included in the protection scope of the present invention.

Claims

1. A method for tracking small targets in infrared images based on local image enhancement and 3D sparse convolutional network, characterized in that: The following steps are involved: Step 1, the infrared image is passed through an image spatial filter to enhance the infrared target and suppress the background; Step 2, sparse sampling the enhanced infrared image to generate sparse data; Step 3, stack the sparse data into a three-dimensional tensor according to the time dimension; Step 4: crop, stack, and rotate the three-dimensional tensor for data enhancement; Step 5: Use the 3D sparse convolutional network to segment the input sparse data and extract the trajectory.

2. The infrared image small target tracking method based on local image enhancement and 3D sparse convolutional network as claimed in claim 1, characterized in that: The specific implementation of step 1 is as follows: Step 1.1: The grayscale value of the infrared small target is high, often much higher than the background, but there is a large area of ​​bright background. Local contrast suppression is used, and an N×N sliding window is used to traverse the infrared image from left to right and from top to bottom according to the step length. The step is usually 3, and a set of slice data is obtained; Y1,Y2,…,Y n (1) where Y n It is the nth slice, with a size of N×N, where N is usually 9. Step 1.2: Perform local target enhancement and background suppression on each slice data and restore the original image size: Where i∈1,…,n, δ is a minimum value, usually 0.000001, α and θ are two hyperparameters, usually 2 and 1, and finally the original image size Y is obtained through resizing operation; Step 1.3, obtain the feature data of the global image, and perform target enhancement and background suppression on the global image: where Y global is the globally enhanced image, and Laplacian(Y) is the Laplacian operator.

3. The infrared image small target tracking method based on local image enhancement and 3D sparse convolutional network as claimed in claim 1, characterized in that: The specific implementation of step 2 is as follows: Step 2.1, calculate the average background intensity threshold of the image obtained in step 1.3: Where λ is a hyperparameter, usually 2 Step 2.2: Use the average background intensity threshold obtained in step 2.1 as the threshold to obtain the sparse image data Y sparse :

4. The infrared image small target tracking method based on local image enhancement and 3D sparse convolutional network as claimed in claim 1, characterized in that: The specific implementation of step 3 is as follows: Step 3.1, stack all sparse data of the same sequence into a three-dimensional sparse tensor according to the time dimension, and the points with values ​​in the tensor account for less than 10% of the entire tensor. Step 3.2, resample the W and H dimensions of the sparse data to 256 as required.

5. The infrared image small target tracking method based on local image enhancement and 3D sparse convolutional network as claimed in claim 1, characterized in that: The specific implementation of step 4 is as follows: Step 4.1: Cut the stacked three-dimensional sparse tensors of the same sequence in the time dimension according to the step length S to obtain a set of three-dimensional tensors of size W×H×S, where W is the width of the original image, H is the length of the original image, both are 256, and S is usually 50; In step 4.2, the tensor obtained in step 4.1 is randomly extracted and rotated, and two-thirds of the tensors are extracted and superimposed two by two to obtain more complex sparse data.

6. The infrared image small target tracking method based on local image enhancement and 3D sparse convolutional network as claimed in claim 1, characterized in that: The specific implementation of step 5 is as follows: Step 5.1, encode the tensors of multiple trajectories obtained in step 4.2, retain only the intensity values ​​and coordinates of the locations with data, and save them as a dictionary; Step 5.2, input the dictionary into the downsampling part of the 3D sparse convolutional network in batches of size batch_size for encoding, batch_size is 2, where the encoding part consists of four convolutional modules, each of which consists of several sparse convolutional modules, and a sparse convolutional module consists of a convolutional layer, a BN layer, and a Relu activation layer; the first convolutional module consists of a sparse convolutional module, the convolutional layer input channel is 1, the output channel is 16, the convolution kernel size is 3, and the step size is 1; the second convolutional module consists of a sparse convolutional module, the convolution kernel side length is 3, the step size is 1, the padding is 1, and the input and output channels are 16; the third convolutional module consists of 3 sparse convolutional modules, the first sparse convolutional module has a convolution kernel side length of 3 and a step size of 2 , padding is 1, the input channel is 16, and the output channel is 32; the second sparse convolution module has a convolution kernel side length of 3, a step size of 1, a padding of 1, an input channel of 32, and an output channel of 32; the third sparse convolution module has a convolution kernel side length of 3, a step size of 1, a padding of 1, an input channel of 32, and an output channel of 32; the fourth convolution module consists of 3 sparse convolution modules, the first sparse convolution module has a convolution kernel side length of 3, a step size of 2, a padding of 1, an input channel of 32, and an output channel of 64; the second sparse convolution module has a convolution kernel side length of 3, a step size of 1, a padding of 1, an input channel of 64, and an output channel of 64; the third sparse convolution module has a convolution kernel side length of 3, a step size of 1, a padding of 1, an input channel of 64, and an output channel of 64; obtain the advanced features after dimensionality increase; Step 5.3, input the high-level features after dimensionality upgrade into the upsampling module, of which there are four upsampling modules in total, each of which includes splicing and convolution parts, and also includes three deconvolution and three splicing parts. The features output in each deconvolution are spliced ​​with the convolution output features of the same size obtained in step 5.2 and input into the next deconvolution layer to obtain output feature values ​​of the same size; the first upsampling module contains three sparse convolution modules, the first of which is a residual convolution module, which contains two layers of input and output The first convolutional layer has 64 channels, a kernel size of 3, and a padding of 1; the second is a 3D sparse convolutional module with 128 input channels, 64 output channels, a kernel size of 3, a stride of 1, and a padding of 1; the third layer is a 3D anti-sparse convolutional module with 64 input channels, 64 output channels, a kernel size of 3, and a stride of 1; the second upsampling module contains three sparse convolutional modules, the first of which is a residual convolutional module, which contains two layers of convolutional layers with 64 input and output channels, a kernel size of 3, and a padding of 1; the The second one is a 3D sparse convolution module with an input channel of 128, an output of 64, a convolution kernel size of 3, a step size of 1, and a padding of 1; the third layer is a 3D anti-sparse convolution module with an input channel of 64, an output channel of 32, a convolution kernel size of 3, and a step size of 1; the third upsampling module contains three sparse convolution modules, the first one is a residual convolution module, which contains two layers of convolution layers with input and output channels of 32, a convolution kernel size of 3, and a padding of 1; the second one is a 3D sparse convolution module with an input channel of 64 and an output of 32 , the convolution kernel size is 3, the stride is 1, and the padding is 1; the third layer is a 3D anti-sparse convolution module, with an input channel of 32, an output channel of 16, a convolution kernel size of 3, and a stride of 1; the fourth upsampling module contains two sparse convolution modules, the first is a residual convolution module, which contains two layers of convolution layers with input and output channels of 16, a convolution kernel size of 3, and a padding of 1; the second is a 3D sparse convolution module, with an input channel of 32, an output of 16, a convolution kernel size of 3, a stride of 1, and a padding of 1; finally, the dictionary is output; Step 5.4, decode the obtained dictionary into sparse data of the original size, slice it, obtain each image containing only the target, and obtain the target trajectory; Step 5.5: Set the network learning rate to 0.0001 and decay it exponentially after each iteration; The parameters are initialized, the alpha parameter is 0.25, and the gamma is 2.0; the focal loss function is used to optimize the model; and the training set is used for 200 iterations of training.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the infrared image small target tracking method based on image local enhancement and 3D sparse convolutional network described in any one of claims 1 to 6 when called by a processor.

8. An infrared image small target tracking system based on local image enhancement and 3D sparse convolutional network, characterized by: The system has a program module corresponding to the steps of any one of claims 1 to 6 above, and executes the steps in the infrared image small target tracking method based on local image enhancement and 3D sparse convolutional network when running.

Images