A multispectral small target recognition and tracking method based on deep learning

Abstract

The application discloses a multispectral small target recognition and tracking method based on deep learning, and relates to the field of computer vision, and comprises the following steps: step one, different wave bands are divided through optical elements in a multispectral camera, and multiple multispectral sensors are allowed to capture multispectral image data of a target area at the same time; different wave bands are divided through optical elements of the multispectral camera, multiple sensors capture multispectral image data of the target area, the image data is loaded by using an OpenCV library, small targets are highlighted by adjusting size, filling and cropping, a mean filter is adopted to perform smoothing processing on central pixels of the small targets, so that the influence of background noise is reduced, a fly bionic attention mechanism is constructed and combined with a convolutional neural network, small targets in multispectral images can be dynamically focused and recognized, and through geometric rectification and optical flow analysis technology, Kalman filtering and a feature matching algorithm are combined, so that accurate tracking of the small targets in different capture scenes is realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and specifically to a method for recognizing and tracking multispectral small targets based on deep learning. Background Technology

[0002] With the rapid development of fields such as intelligent monitoring, environmental monitoring, and automated agriculture, the demand for accurate identification and tracking capabilities of small targets is increasing. Small targets usually occupy a small area in vision and are often in complex backgrounds. Accurate identification and tracking of these targets are crucial for achieving the goals of various applications.

[0003] Existing technologies suffer from the following shortcomings: Traditional target recognition methods rely heavily on visible light images. However, the limited information in visible light images in certain scenarios directly impacts the detection of small targets. They often overlook the rich information provided by multispectral images, making it difficult to accurately identify small targets in complex backgrounds and with environmental interference. This underutilization of information not only reduces the accuracy of target detection but also worsens the algorithm's adaptability to background conditions, increasing the likelihood of misidentification. When small targets constitute a very small proportion, they are severely affected by image noise and background complexity, causing their features to be submerged and difficult to separate from the background. When the target and background brightness are similar or the background noise is high, the ability to effectively extract and label targets is greatly reduced. Traditional image processing methods lack the ability to learn complex features and are ill-suited to various changing conditions. Small targets may be affected by multiple factors; if the feature extraction method cannot adaptively adjust, the recognition accuracy will be greatly affected, making effective tracking of small targets in complex environments extremely difficult. Traditional tracking algorithms often fail to accurately capture the dynamic changes and state updates of small targets.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning-based multispectral small target recognition and tracking method to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a multispectral small target recognition and tracking method based on deep learning, comprising the following steps:

[0007] Step 1: Divide the target area into different bands using the optical elements inside the multispectral camera and allow multiple multispectral sensors to simultaneously capture multispectral image data of the target area;

[0008] Step 2: Load the multispectral image data using the OpenCV library, adjust the size, apply padding and cropping operations, highlight small targets in the multispectral image data through image processing and perform iterative marking, and use a mean filter to smooth the center pixels of the small targets in the multispectral image data.

[0009] Step 3: Establish a fly-inspired attention mechanism and integrate it into a convolutional neural network to dynamically focus on and identify small targets in multispectral image data;

[0010] Step 4: By performing geometric correction and optical flow analysis on multispectral image data, combined with Kalman filtering and feature matching algorithms, accurate tracking and localization of small targets in different capture scenarios can be achieved.

[0011] Preferably, the input beam is distributed to multiple optical fibers according to different wavelength ranges by an internal fiber optic beam splitter of the multispectral camera; different wavelengths of the input beam are separated into different spectral bands by an internal optical prism of the multispectral camera; different wavelengths of the input beam are separated by an internal grating beam splitter of the multispectral camera and distributed to multispectral sensors in wavelength order; multiple multispectral sensors are installed corresponding to specific wavelengths and capture multispectral image data of the target area; the parameters and settings of the multispectral camera are adjusted according to the characteristics of the capture scene under different angles and lighting conditions, including exposure adjustment, focus adjustment, white balance adjustment, and ISO setting; a data storage unit is created and the multispectral image data of the target area is stored in the data storage unit during the capture process; the multispectral image data of the target area is marked with the capture time, target area, acquisition angle, and lighting conditions; exposure is adjusted by adjusting the aperture size of the multispectral camera; focus is adjusted by subject autofocus and manual focus by adjusting the camera focal length; white balance is adjusted by preset white balance mode and manual white balance setting; and ISO value is set according to the lighting conditions of the capture scene.

[0012] Preferably, the size of the multispectral image data is fixed at 224*224 pixels. The multispectral image data is loaded using the OpenCV library. The target size parameter is specified by the `target_size` command of the `resize` function. If the target size parameter does not match the size of the multispectral image data, padding and cropping operations are applied for adjustment. This includes adding pixels to insufficient areas of the multispectral image data, cropping pixels to excessive areas, converting the RGB color space of the multispectral image data to grayscale space using the RGB channels, adjusting the brightness and contrast of the multispectral image data through grayscale stretching, highlighting details of the multispectral image data using sharpening and edge enhancement, iteratively marking small targets in the multispectral image data, and averaging the pixels around the center pixel of the small target using a mean filter. The specific formula is as follows:

[0013]

[0014] Among them I new (x,y) represents the new pixel value at position (x,y) on the multispectral image data after mean filtering, N represents the size of the mean filter, i and j represent the position of each pixel within the mean filter, and x and y represent each position of the mean filter on the multispectral image data.

[0015] Preferably, a convolutional layer is added, using multiple convolutional kernels to perform sliding local sensing on the multispectral image data. ReLU is used as the activation function layer to introduce nonlinearity, and dot product operations are performed on the multispectral image data to generate multispectral feature maps. The specific formula is as follows:

[0016] S(a,b)=∑∑I(m,n)*K(am,bn)

[0017] Where S(a,b) represents a pixel in the multispectral feature map, I(m,n) represents the pixel value of the multispectral image data, K(am,bn) represents the weight value of the multispectral feature map, m and n represent the row and column indices within the convolution kernel, respectively, and a and b represent the row and column indices of the multispectral feature map, respectively. Based on the fly-inspired attention mechanism, key regions of the target feature map are selected through attention weights. The specific formula is as follows:

[0018]

[0019] Among them, A pq F represents the attention weight of the q-th pixel in the p-th target feature map. pq represents the single feature value of the q-th pixel in the p-th target feature map after processing by the convolutional layer, and o represents all pixels in the key region of the target feature map. The pooling layer performs downsampling operation only on the target feature map according to the fly-inspired attention mechanism. The convolution and pooling are repeated iteratively. The target feature map is flattened into a one-dimensional vector through the fully connected layer. The target feature map is mapped to the category of different small targets according to the attention weights, and the final recognition result is output.

[0020] Preferably, the multispectral image data is registered with capture scenes under different angles and lighting conditions. Geometric correction is performed on the multispectral image data based on the pose changes of the multispectral camera. The trajectory of a small target in the multispectral image data is predicted based on the optical flow field of the previous frame using a raptor-inspired hunting mechanism. The motion vector of the center pixel of the small target in the multispectral image data is estimated using the optical flow equation based on the spatial and temporal gradients of the multispectral image data. The specific formula is as follows:

[0021] I cu+I d v+I t =0

[0022] Among them, I c I represents the gradient of multispectral image data in the horizontal direction. d I represents the gradient of multispectral image data in the vertical direction. t The gradient of the multispectral image data in the time direction is represented by u and v, which represent the optical flow components in the horizontal and vertical directions, respectively. The future state of the small target is predicted based on the multispectral image data of the previous frame by Kalman filtering. The capture scene is finely tracked by using a local optical flow weighting strategy to locate the position of the small target in the multispectral image data. This position is used as the initial position. Key points of the small target are extracted from the capture area around the initial position. The target feature map of the subsequent frame is matched with the target feature map of the initial position to continuously estimate and update the small target tracking.

[0023] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0024] 1. By utilizing the fiber optic beam splitter, optical prism, and grating beam splitter within the multispectral camera, effective allocation and separation of beams of different wavelengths are achieved. This allows the camera to capture richer multispectral image data. Camera parameters, including exposure, focus, white balance, and ISO settings, are automatically adjusted according to different angles and lighting conditions, ensuring high-quality multispectral images even in varying environments. This guarantees the capture of details and features of the target area. During capture, multispectral image data is stored in a data storage unit, and each image data point is marked with time, target area, acquisition angle, and lighting conditions. Image processing using the OpenCV library enables rapid loading and adjustment of multispectral image data dimensions, along with grayscale conversion, brightness and contrast adjustment, and detail enhancement. For small targets in the multispectral image, a mean filter is used to average the pixels surrounding the center pixel, effectively reducing noise and improving the recognition accuracy of small targets.

[0025] 2. Through optical flow analysis and a raptor-inspired hunting mechanism, the system can predict the trajectory of small targets based on the optical flow field of the previous frame. This not only improves the sensitivity to the movement of small targets but also enhances adaptability in complex environments. During the processing of multispectral image data, geometric correction can be performed based on camera pose changes, ensuring that image data from different capture scenes can be analyzed in the same coordinate system. Kalman filtering effectively handles noise and errors during small target tracking, providing the best estimate of the small target's position. It maintains stability in fast motion and complex backgrounds. Employing a local optical flow weighting strategy, it can perform refined tracking of the capture scene, ensuring more accurate positioning of small targets. This allows for timely adjustment of the tracking strategy when handling the rapid movement of small targets, preventing target loss. Through feature matching algorithms, it can match the target feature map of subsequent frames with the target feature map of the initial position, continuously updating the position of the small target and ensuring that the tracking of small targets remains efficient and accurate under different scenes and conditions. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0027] Figure 1 This is a flowchart of a deep learning-based multispectral small target recognition and tracking method according to the present invention. Detailed Implementation

[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0029] Example 1

[0030] This invention provides, for example Figure 1 The method for identifying and tracking multispectral small targets based on deep learning, as shown, includes the following steps:

[0031] Step 1: Divide the target area into different bands using the optical elements inside the multispectral camera and allow multiple multispectral sensors to simultaneously capture multispectral image data of the target area;

[0032] The multispectral camera uses an internal fiber optic splitter to distribute the input beam into multiple optical fibers according to different wavelength ranges. An internal optical prism separates the input beams of different wavelengths into different spectral bands. An internal grating splitter further separates the input beams of different wavelengths and distributes them sequentially to the multispectral sensors. Multiple multispectral sensors are installed to capture multispectral image data of the target area at specific wavelengths. The parameters and settings of the multispectral camera are adjusted according to the characteristics of the capture scene under different angles and lighting conditions, including exposure adjustment, focus adjustment, white balance adjustment, and ISO settings. A data storage unit is created, and the multispectral image data of the target area is stored in the data storage unit during the capture process. The multispectral image data of the target area is marked with the capture time, target area, acquisition angle, and lighting conditions. Exposure is adjusted by adjusting the aperture size of the multispectral camera; focus is adjusted by using subject autofocus and manual focus adjustment by adjusting the camera focal length; white balance is adjusted by using preset white balance modes and manual white balance settings; and the ISO value is set according to the lighting conditions of the capture scene.

[0033] In this embodiment, the first step specifically involves the fiber optic beam splitter, which distributes the input beam to multiple optical fibers according to different wavelength ranges, ensuring that optical signals in each band can be transmitted independently. The optical prism separates the input beams of different wavelengths into different spectral bands, thereby achieving precise band division. The grating beam splitter further separates the beams of different wavelengths and distributes them to various multispectral sensors in wavelength order. By installing multiple multispectral sensors, each corresponding to a specific wavelength, multispectral image data of the target area can be captured simultaneously, improving the efficiency and quality of data acquisition. The amount of light entering the camera is controlled by adjusting the aperture size to adapt to different lighting conditions and ensure that the brightness of the multispectral image data is moderate. A combination of subject autofocus and manual focus is used to ensure that the multispectral image data is clear. Through efficient optical design and flexible parameter adjustment, the multispectral camera can accurately and comprehensively collect multispectral image data of the target area under different capture scenarios.

[0034] Step 2: Load the multispectral image data using the OpenCV library, adjust the size, apply padding and cropping operations, highlight small targets in the multispectral image data through image processing and perform iterative marking, and use a mean filter to smooth the center pixels of the small targets in the multispectral image data.

[0035] The size of the multispectral image data is fixed at 224*224 pixels. The multispectral image data is loaded using the OpenCV library. The target size parameter is specified by the `target_size` command of the `resize` function. If the target size parameter does not match the size of the multispectral image data, padding and cropping operations are applied for adjustment. This includes adding pixels to insufficient areas of the multispectral image data, cropping pixels to excessive areas, converting the RGB color space of the multispectral image data to grayscale space using the RGB channels, adjusting the brightness and contrast of the multispectral image data through grayscale stretching, highlighting details using sharpening and edge enhancement, iteratively marking small targets in the multispectral image data, and averaging the pixels around the center pixel of the small target using a mean filter. The specific formula is as follows:

[0036]

[0037] Among them I new (x,y) represents the new pixel value at position (x,y) on the multispectral image data after mean filtering, N represents the size of the mean filter, i and j represent the position of each pixel within the mean filter, and x and y represent each position of the mean filter on the multispectral image data.

[0038] In this embodiment, step two specifically needs to be explained: Multispectral image data is loaded using the OpenCV library, ensuring the data format is correct. A fixed output size of 224x224 pixels is set. The target size parameter is specified by the `target_size` command of the `resize` function. When the size of the loaded multispectral image data does not match the target size, the following operations are performed: Padding: For multispectral image data smaller than 224x224 pixels, surrounding pixels are added to expand it to the target size; Cropping: For multispectral image data larger than 224x224 pixels, the excess portion is cropped to ensure compliance with... To achieve the desired target size, the multispectral image data is converted to grayscale using the RGB channels to make it easier to identify small targets. The brightness and contrast of the multispectral image data are adjusted using grayscale stretching technology to enhance the visibility of small targets in the multispectral image data. Sharpening and edge enhancement techniques are used to highlight the details of the multispectral image data. Small targets in the image are iteratively marked for subsequent recognition and tracking. After filling and cropping the multispectral image, grayscale conversion and brightness and contrast adjustments should be performed first. Subsequent sharpening and mean filtering operations can be performed while the multispectral image is clearly visible to ensure the consistency and effectiveness of multispectral image processing.

[0039] Step 3: Establish a fly-inspired attention mechanism and integrate it into a convolutional neural network to dynamically focus on and identify small targets in multispectral image data;

[0040] A convolutional layer is added, using multiple convolutional kernels to perform sliding local sensing on the multispectral image data. ReLU is used as the activation function layer to introduce nonlinearity, and dot product operations are performed on the multispectral image data to generate multispectral feature maps. The specific formula is as follows:

[0041] S(a,b)=∑∑I(m,n)*K(am,bn)

[0042] Where S(a,b) represents a pixel in the multispectral feature map, I(m,n) represents the pixel value of the multispectral image data, K(am,bn) represents the weight value of the multispectral feature map, m and n represent the row and column indices within the convolution kernel, respectively, and a and b represent the row and column indices of the multispectral feature map, respectively. Based on the fly-inspired attention mechanism, key regions of the target feature map are selected through attention weights. The specific formula is as follows:

[0043]

[0044] Among them, A pq F represents the attention weight of the q-th pixel in the p-th target feature map. pq represents the single feature value of the q-th pixel in the p-th target feature map after processing by the convolutional layer, and o represents all pixels in the key region of the target feature map. The pooling layer performs downsampling operation only on the target feature map according to the fly-inspired attention mechanism. The convolution and pooling are repeated iteratively. The target feature map is flattened into a one-dimensional vector through the fully connected layer. The target feature map is mapped to the category of different small targets according to the attention weights, and the final recognition result is output.

[0045] In this embodiment, step three specifically needs to be explained: a fly-inspired attention mechanism is established based on the visual characteristics of flies to improve the attention and responsiveness of the convolutional neural network to small targets. By calculating attention weights for each pixel of the feature map, the focus is dynamically placed on the key regions of the target feature map to enhance the salience of small targets in the target feature map. Multiple convolutional layers and kernels are introduced, and multispectral features are extracted through sliding local perception. Each convolutional layer uses the ReLU activation function to improve the nonlinear expressive power of the model.

[0046] Step 4: By performing geometric correction and optical flow analysis on multispectral image data, combined with Kalman filtering and feature matching algorithms, accurate tracking and localization of small targets in different capture scenarios can be achieved.

[0047] The multispectral image data is registered to capture scenes under different angles and lighting conditions. Geometric correction is performed on the multispectral image data based on the pose changes of the multispectral camera. The trajectory of a small target in the multispectral image data is predicted based on the optical flow field of the previous frame using a raptor-inspired hunting mechanism. The motion vector of the center pixel of the small target in the multispectral image data is estimated using the optical flow equation based on the spatial and temporal gradients of the multispectral image data. The specific formula is as follows:

[0048] I c u+I d v+I t =0

[0049] Among them, I c I represents the gradient of multispectral image data in the horizontal direction. d I represents the gradient of multispectral image data in the vertical direction. t The gradient of the multispectral image data in the time direction is represented by u and v, which represent the optical flow components in the horizontal and vertical directions, respectively. The future state of the small target is predicted based on the multispectral image data of the previous frame by Kalman filtering. The capture scene is finely tracked by using a local optical flow weighting strategy to locate the position of the small target in the multispectral image data. This position is used as the initial position. Key points of the small target are extracted from the capture area around the initial position. The target feature map of the subsequent frame is matched with the target feature map of the initial position to continuously estimate and update the small target tracking.

[0050] In this embodiment, step four specifically needs to be explained: geometric correction is used to eliminate the influence of camera pose changes; images from different capture scenes are registered to ensure that multispectral images are analyzed in the same coordinate system; optical flow is used to analyze the motion trajectory of the small target; based on the optical flow field of the previous frame and combined with the raptor-inspired hunting mechanism, the position of the small target in the current frame is predicted; and the Kalman filter algorithm is used to predict the future state of the small target based on the multispectral image data of the previous frame. Kalman filtering can effectively handle system noise and measurement noise, providing the best estimate of the position of the small target. In optical flow analysis, a local weighting strategy is used to perform fine tracking of the capture scene, which can better adapt to the rapid movement of the small target and improve the tracking accuracy.

[0051] By utilizing the fiber optic beam splitter, optical prism, and grating beam splitter within the multispectral camera, effective allocation and separation of beams of different wavelengths are achieved. This allows the camera to capture richer multispectral image data. Camera parameters, including exposure, focus, white balance, and I / O settings, are automatically adjusted according to different angles and lighting conditions, ensuring high-quality multispectral images even in varying environments and guaranteeing the capture of details and features of the target area. During capture, multispectral image data is stored in a data storage unit, and each image data point is tagged with time, target area, acquisition angle, and lighting conditions. Image processing using the OpenCV library enables rapid loading and adjustment of the multispectral image data size, along with grayscale conversion, brightness and contrast adjustment, and detail enhancement. For small targets in the multispectral image, a mean filter is used to average the pixels surrounding the center pixel, effectively reducing noise and improving the recognition accuracy of small targets.

[0052] By employing optical flow analysis and a raptor-inspired hunting mechanism, the system can predict the trajectory of small targets based on the optical flow field of the previous frame. This not only improves the sensitivity to the movement of small targets but also enhances adaptability in complex environments. During the processing of multispectral image data, geometric correction can be performed based on camera pose changes, ensuring that image data from different capture scenes can be analyzed in the same coordinate system. Kalman filtering effectively handles noise and errors during small target tracking, providing the best estimate of the small target's position. It maintains stability in fast motion and complex backgrounds. By adopting a local optical flow weighting strategy, it can perform refined tracking of the capture scene, ensuring more accurate positioning of small targets. This allows for timely adjustment of the tracking strategy when handling the rapid movement of small targets, preventing target loss. Through feature matching algorithms, the target feature map of subsequent frames can be matched with the target feature map of the initial position, continuously updating the position of the small target and ensuring that the tracking of small targets remains efficient and accurate under different scenes and conditions.

[0053] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A deep learning-based method for multispectral small target recognition and tracking, characterized in that, Includes the following steps: Step 1: Divide the target area into different bands using the optical elements inside the multispectral camera and allow multiple multispectral sensors to simultaneously capture multispectral image data of the target area; Step 2: Load the multispectral image data using the OpenCV library, adjust the size, apply padding and cropping operations, highlight small targets in the multispectral image data through image processing and perform iterative marking, and use a mean filter to smooth the center pixels of the small targets in the multispectral image data. Step 3: Establish a fly-inspired attention mechanism and integrate it into a convolutional neural network to dynamically focus on and identify small targets in multispectral image data; Step 4: By performing geometric correction and optical flow analysis on multispectral image data, combined with Kalman filtering and feature matching algorithms, accurate tracking and localization of small targets in different capture scenarios can be achieved. In step one, the optical components inside the multispectral camera include: an optical fiber splitter for distributing the input beam to multiple optical fibers according to different wavelength ranges; an optical prism for separating the input beam of different wavelengths into different spectral bands through the refraction effect of the prism; and a grating splitter for separating the input beam of different wavelengths according to the diffraction effect of the grating and distributing it to the multispectral sensor in wavelength order. In step two, the size of the multispectral image data is fixed at 224*224 pixels. The multispectral image data is loaded using the OpenCV library. The target size parameter is specified by the target_size command of the resize function. If the target size parameter does not match the size of the multispectral image data, padding and cropping operations are applied for adjustment. This includes adding pixels in the insufficient areas of the multispectral image data, cropping pixels in the excessive areas of the multispectral image data, converting the RGB color space of the multispectral image data to grayscale space using the RGB channel, adjusting the brightness and contrast of the multispectral image data through grayscale stretching, highlighting the details of the multispectral image data using sharpening and edge enhancement, iteratively marking small targets in the multispectral image data, and averaging the pixels around the center pixel of the small target in the multispectral image data using a mean filter. The specific formula for the mean filter is as follows: ； in This indicates the location on the multispectral image data after mean filtering. The new pixel value at that location, Indicates the size of the mean filter. This indicates the position of each pixel within the mean filter. Let N represent each position of the mean filter in the multispectral image data. N is a positive odd number, greater than or equal to 3. The offsets i and j within the summation interval are both integers.

2. The method for multispectral small target recognition and tracking based on deep learning according to claim 1, characterized in that: In step three, a convolutional layer is added, and multiple convolutional kernels are used to perform sliding local perception on the multispectral image data. ReLU is used as the activation function layer to introduce nonlinearity and perform dot product operation on the multispectral image data to generate a multispectral feature map. Based on the fly-inspired attention mechanism, key regions of the target feature map are selected through attention weights. Through the pooling layer, based on the fly-inspired attention mechanism, downsampling operation is performed only on the target feature map. Convolution and pooling are repeated iteratively. The target feature map is flattened into a one-dimensional vector through a fully connected layer. Based on the attention weights, the target feature map is mapped to the category of different small targets, and the final recognition result is output.

3. The method for multispectral small target recognition and tracking based on deep learning according to claim 2, characterized in that: The specific formula for selecting the key region of the target feature map is as follows: ； in, This represents the attention weight of the q-th pixel in the p-th target feature map. This represents the single feature value of the q-th pixel in the p-th target feature map after processing by the convolutional layer. This represents all pixels in the key region of the target feature map.

4. The method for multispectral small target recognition and tracking based on deep learning according to claim 1, characterized in that: In step four, the multispectral image data is registered with capture scenes at different angles and under different lighting conditions. Geometric correction is performed on the multispectral image data based on the attitude changes of the multispectral camera. The motion trajectory of the small target in the multispectral image data is predicted based on the optical flow field of the previous frame of the multispectral image data according to the raptor-inspired hunting mechanism. The motion vector of the center pixel of the small target in the multispectral image data is estimated by optical flow equation based on the spatial and temporal gradients of the multispectral image data. The future state of the small target is predicted based on the multispectral image data of the previous frame using Kalman filtering. The capture scene is finely tracked using a local optical flow weighting strategy to locate the position of the small target in the multispectral image data. This position is used as the initial position. Key points of the small target are extracted from the capture area around the initial position. The target feature map of the subsequent frame is matched with the target feature map of the initial position using a feature matching algorithm to continuously estimate and update the small target tracking.

5. The method for multispectral small target recognition and tracking based on deep learning according to claim 4, characterized in that: The specific formula for the optical flow equation is as follows: ； in, This represents the gradient of multispectral image data in the horizontal direction. This represents the gradient of multispectral image data in the vertical direction. This represents the gradient of multispectral image data in the time direction. These represent the optical flow components in the horizontal and vertical directions, respectively.

Images