Trajectory calculation method based on tiny target recognition and tracking and storage medium

Abstract

The invention discloses a trace calculation method based on tiny target identification and tracking and a storage medium. The trace calculation method comprises the following steps: extracting an original image of a target video as input, performing image processing, improving the quality of a target video image by adopting a background-based adaptive prediction algorithm, and comparing with the input original image to obtain a target detection image; the target detection image is input into a target extraction algorithm module, a connected domain marking algorithm is adopted to separate motion and noisy point targets, a distribution area is determined, and a feature image is output; tracking and positioning the feature image, calculating by adopting a centroid positioning algorithm, and outputting a centroid coordinate and a pixel gray value of the target; and filtering static noisy points in the video image through threshold segmentation, performing trajectory prediction and calculation on a moving target based on a Kalman filtering method, and outputting the trajectory of the moving target in an image plane coordinate system. The method has higher trajectory calculation precision and calculation speed, and can be used in the field of motion trajectory prediction of small targets such as unmanned aerial vehicles.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a trajectory calculation method and storage medium based on small target recognition and tracking. Background Art

[0002] Noise in video images poses a significant technical challenge when identifying, tracking, and calculating the trajectories of small targets. Images often contain random noise and interference from varying lighting conditions. This noise often severely impacts image quality. These noise factors not only reduce target visibility but also directly affect the accuracy of small target recognition and location. For example, background noise can affect the recognition of small targets, leading to false detections and missed detections of target sizes. Small targets can also be confused with noise in images, making it difficult for traditional detection algorithms to accurately identify them. The detection and tracking of small targets has become a research hotspot in computer vision and image processing. Effective detection of these small targets typically requires image preprocessing to enhance target visibility and the application of specific target recognition algorithms to extract the target.

[0003] In image preprocessing, traditional image enhancement techniques such as contrast enhancement, histogram equalization, and gamma correction are often used for small target recognition. However, these enhancement techniques only improve overall image quality and fail to effectively enhance the features of small targets. Consequently, in the presence of strong noise, the contrast between small targets and noise points is still insufficient, making it difficult to separate the targets. In target recognition and tracking, with the development of artificial intelligence, machine learning and deep learning-based methods have been gradually introduced into target detection. For example, models such as support vector machines (SVMs) and convolutional neural networks (CNNs) can distinguish targets from noise to a certain extent. However, deep learning methods typically rely on large annotated datasets. In practical applications of small target detection, especially in nighttime scenes, annotated data is very limited, placing high demands on hardware resources and making them unsuitable for small target tracking tasks that require real-time performance. In trajectory calculation, the commonly used optical flow method calculates the motion of image pixels to obtain the target's trajectory. This method is suitable for tracking larger targets. However, the motion features of small targets are not obvious, and in noisy nighttime images, optical flow methods struggle to accurately capture the direction and speed of small targets.

[0004] Therefore, in order to improve the anti-noise ability and real-time performance of video images and reduce the computational complexity, so as to more accurately calculate the trajectory of small targets, the present invention proposes a trajectory calculation method based on small target recognition and tracking to meet the above requirements. Summary of the Invention

[0005] The present invention aims to address the problem of low precision in trajectory calculation for small targets by providing a trajectory calculation method based on small target identification and tracking. This method utilizes an image background adaptation algorithm and a connected component labeling algorithm for image preprocessing and target localization. Compared with traditional methods, this method eliminates the impact of dynamic background changes, mitigates noise issues from the observation equipment and background, and improves the accuracy of trajectory prediction.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A trajectory calculation method based on small target recognition and tracking specifically includes the following steps:

[0008] Step (1), image preprocessing and target detection

[0009] First, a video of a moving target is imported and the original images are extracted as input. In these images, the moving target becomes blurred and difficult to discern due to noise interference. Therefore, the images are processed using a background adaptive prediction algorithm. This algorithm automatically analyzes the image's background features and adapts to background changes in real time, effectively improving image quality and enhancing the saliency of the target. By comparing the original input image with the processed image, we obtain a target detection image in which background noise is significantly reduced and the features of the moving target are highlighted, laying the foundation for further processing.

[0010] Step (2), target feature extraction

[0011] The object detection image generated in the first step is fed into the object extraction algorithm module. This module employs a connected component labeling algorithm, which analyzes pixel connectivity to separate individual noise points from moving objects. The algorithm automatically identifies the distribution area of ​​each noise point and moving object and labels its range. The output is a feature image containing the location information of all noise points and moving objects. This step ensures the separation of noise points from actual moving objects, allowing us to accurately locate the moving object area.

[0012] Step (3), tracking positioning and center of mass calculation

[0013] Each connected region in the feature image is precisely tracked and located. A centroid localization algorithm is used to calculate the center of mass position and pixel grayscale value of each target region. Specifically, the algorithm calculates the target's center of mass coordinates based on the pixel value distribution of the connected region, obtaining the center of mass position and corresponding grayscale value of the moving target and noise points in the image plane coordinate system. This information provides high-precision data input for subsequent trajectory prediction while effectively removing interference from irrelevant noise.

[0014] Step (4), trajectory prediction calculation

[0015] First, the center of mass position of the current frame is compared with that of the previous frame. If the change in center of mass position exceeds a certain threshold, the target is considered to be in motion; otherwise, the target is considered stationary, and only moving targets are retained. Next, the extended Kalman filter algorithm is applied to these moving targets to predict their future trajectories based on their current and historical positions. The extended Kalman filter iteratively updates the target's state estimate at each moment, minimizing noise interference and thus producing a smooth trajectory for the moving target.

[0016] A computer-readable storage medium stores a computer program, which is executed by a processor to implement the steps of the method for calculating the trajectory of a small target based on identification and tracking described in the present invention.

[0017] Beneficial effects of the present invention:

[0018] (1) Higher trajectory calculation accuracy

[0019] In trajectory calculations before denoising, the mean square error (MSE) of the measured values ​​was approximately 1.944. After denoising using the method of the present invention, the calculated mean square error dropped to 0.664, resulting in a 68.4% improvement in accuracy. Furthermore, for the measurement residuals, the mean absolute error (MAE) was approximately 1.109 before denoising, but dropped to 0.459 after denoising, resulting in a 58.62% improvement in residual accuracy. Therefore, the present invention achieves higher trajectory calculation accuracy.

[0020] (2) Stronger real-time performance

[0021] The present invention uses an extended Kalman filter (ECF) in its trajectory prediction process. This algorithm relies on current and historical observation data and requires only first-order derivatives and matrix operations. Compared to traditional filtering algorithms (such as complex nonlinear algorithms like particle filters), its operation speed is increased by approximately 25-30%, ensuring a stable processing frame rate of over 30 frames per second, thus meeting the requirements of real-time video processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 : Flowchart of a trajectory calculation method based on small target recognition and tracking.

[0023] Figure 2 : Video images of tiny moving targets, (a), (b), and (c) are three frames of images in the video respectively.

[0024] Figure 3 : Trajectory prediction curve of small moving targets. DETAILED DESCRIPTION

[0025] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0026] Those skilled in the art will understand that the following examples are only used to illustrate the present invention and should not be considered as the scope of the present invention. Where specific techniques or conditions are not specified in the examples, the techniques or conditions described in the literature in the art shall be followed.

[0027] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0028] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which the various embodiments of the present invention pertain. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meaning as in the context of the relevant technical field.

[0029] See also Figure 1 As shown, a trajectory calculation method based on small target recognition and tracking includes the following steps:

[0030] Step (1): pre-process the target video image

[0031] The original image of the small moving target video frame is extracted as input, denoted as I origin_in , use the grayscale values ​​of surrounding pixels to predict the image, including:

[0032] Step (1.1) is as follows:

[0033]

[0034] I origin_in is the input original image of M×N, X(i,j) is the predicted image, W represents the weight matrix, and the size of the prediction window S is k×l. The residual image E(i,j) is obtained by subtracting the input original image and the predicted image, as shown in the following formula:

[0035] E(i,j)=I origin_in (i,j)-X(i,j)

[0036] In step (1.2), for the target video image, the background usually has some fluctuations. When predicting the background, the regional maximization prediction method is usually used. That is, with the current pixel as the center, four regions are divided in the surrounding pixels to obtain the corresponding background prediction value, namely:

[0037]

[0038] The maximum value is used as the final background grayscale prediction value, as shown in the following formula:

[0039] X(i,j)=max{X m (i,j)},m=1,2,3,4

[0040] In step (1.3), the key to background prediction is to determine the weight matrix. The grayscale average of all pixels in the prediction window S is used as the predicted value of the current background pixel. In order to improve the accuracy of the prediction, the closer to the prediction point, the smaller the corresponding weight. Therefore, the value of the weight matrix is ​​as shown in the following formula:

[0041]

[0042] Step (2), extracting target and noise distribution areas, including:

[0043] After completing the background prediction in step (1), the residual image E(i, j) is obtained, which includes two steps: segmentation between the target and the background, and segmentation between the target and the noise. After completing steps (2.1) and (2.2), the binary image BW(i, j) is output.

[0044] In step (2.1), the segmentation between the target and the background is performed using the threshold segmentation method. The threshold is calculated by the following formula:

[0045] Y=μ+kσ

[0046] Among them, μ represents the mean of the residual image, σ is the mean square error of the residual image, and k is the weight coefficient, which usually ranges from 5 to 15.

[0047] In step (2.2), the segmentation between the target and the noise is performed using the connected domain labeling method. The image after the threshold segmentation in step (2.1) is labeled using the morphological four-connected domain labeling method. Each connected domain is assigned a label number. To eliminate additional noise, targets with pixels smaller than the threshold are discarded.

[0048] Step (3), after obtaining the target distribution area, the centroid positioning algorithm can be used to locate the small target. The square weighted centroid method is used for calculation, that is, the square of the gray value is used instead of the gray value as the weight to calculate the centroid. This can solve the error problem caused by the uneven gray distribution of the image. The calculation formula is shown as follows:

[0049]

[0050] Among them, E(x,y) represents the grayscale value occupied by the current target, and (x0,y0) is the center of mass coordinate of the small target.

[0051] Step (4): calculate the motion trajectory of the small target

[0052] The center of mass coordinates obtained by step (3) are output, and the center of mass position of the current frame is compared with the center of mass position of the previous frame. If the distance of the center of mass position change exceeds a certain threshold, the target is considered to be moving; otherwise, the target is considered to be stationary. The moving target is extracted and the estimated value of the previous moment is predicted and corrected based on the real-time observation value of the system through Kalman filtering, including:

[0053] Step (4.1), prediction step

[0054] In the prediction step, the nonlinear state transfer equation f(x) is used to predict the state and covariance. The state prediction equation and covariance prediction equation are written as shown below:

[0055]

[0056] Among them, F k is the Jacobian matrix of the state transfer function, and linearizing f(x) yields:

[0057]

[0058] Step (4.2), update step

[0059] In the update step, the nonlinear observation equation h(x) and the actual observation value z k Update the state estimates and covariance. First

[0060] Calculate the Kalman gain:

[0061]

[0062] Then update the status:

[0063]

[0064] in is the measurement residual, which represents the difference between the observed value and the predicted value.

[0065] Finally update the covariance:

[0066] P k|k =(IK k H k )P k|k-1

[0067] Among them, P k|k is the updated covariance matrix, which indicates the accuracy of the current state estimation. Repeating the above process iteratively will get the predicted trajectory.

[0068] Example 1

[0069] There is a video of a moving target, such as Figure 2 As shown in FIG, the small target is 100 meters above the ground and the target trajectory contains a lot of noise. The algorithm of the present invention is used to process the video as follows:

[0070] Directly using the measurement values ​​in the original image to draw the trajectory of the moving target, we can see that the trajectory contains many noise points, and the trajectory curve is irregular with obvious fluctuations.

[0071] The extended Kalman filter algorithm is used to process the noise and obtain the denoised trajectory curve.

[0072] The denoised trajectory is smoother and truly reflects the target's motion path in the two-dimensional plane. Finally, the trajectory of the moving target before and after denoising is compared and output in the form of a coordinate graph.

[0073] The output is the predicted trajectory of the moving target in the two-dimensional plane (image plane coordinate system), such as Figure 3 As shown, the blue dotted line represents the ideal target motion trajectory, the gray dots represent the motion trajectory of the target in the image without denoising, and the red solid line represents the target motion trajectory after denoising using the method of the present invention.

[0074] In the trajectory calculation before denoising, the mean square error (MSE) of the measured value was approximately 1.944. After denoising using the method of the present invention, the calculated mean square error was reduced to 0.664. This denoising process brought a 68.4% improvement in accuracy. In addition, for the measurement residual, the mean absolute error (MAE) was approximately 1.109 before denoising, and was reduced to 0.459 after denoising, bringing about a 58.62% improvement in residual accuracy. The specific calculation process of the above results is as follows:

[0075] The mean square error (MSE) of the measured value and the mean absolute error (MAE) before denoising are used to evaluate the accuracy of trajectory calculation. The calculation formula of MSE is as follows:

[0076]

[0077] Among them, n is the total number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample, given by Figure 3 Comprehensive calculations show that the mean square error (MSE) of the measured values ​​is approximately 1.944. After denoising using the method of the present invention, the calculated mean square error is reduced to 0.664. This denoising process brings a 68.4% improvement in accuracy, and the model prediction results are closer to the true values. In addition, the mean absolute error (MAE) is introduced to evaluate the average value of the absolute error between the predicted value and the observed value. The smaller the value, the closer the predicted value is to the true value. The MAE calculation formula is as follows:

[0078]

[0079] Among them, n is the total number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample, given by Figure 3 Analysis and calculation show that the mean absolute error (MAE) before denoising is about 1.109, which is reduced to 0.459 after denoising, resulting in an improvement of the residual accuracy by about 58.62%. Therefore, the present invention has higher trajectory calculation accuracy.

Claims

1. A trajectory calculation method based on small target recognition and tracking, characterized in that: The following steps are involved: Step (1), extracting the original image of the target video as input, performing image processing, using a background adaptive prediction algorithm to improve the quality of the target video image, and comparing it with the input original image to obtain a target detection image; Step (2), inputting the target detection image output in step (1) into the target extraction algorithm module, using the connected domain labeling algorithm to separate the moving target and the noise target, determining their distribution areas, and outputting the feature image; Step (3), tracking and locating the feature image output in step (2), calculating each connected domain using a centroid positioning algorithm, and outputting the centroid coordinates and pixel grayscale values ​​of the target in the image plane coordinate system; Step (4) is to filter out static noise points in the video image by threshold segmentation, predict and calculate the trajectory of the moving target based on the Kalman filter method, and output the trajectory of the moving target in the image plane coordinate system.

2. The method for calculating the trajectory of a small target based on identification and tracking according to claim 1, characterized in that: Step (1) further includes: The original image of the small moving target video frame is extracted as input, denoted as I origin_in , use the grayscale values ​​of surrounding pixels to predict the image, including: Step (1.1), as shown in formula (1): I origin_in is the input original image of M×N, X(i, j) is the predicted image, W represents the weight matrix, and the size of the prediction window S is k×l; the difference between the input original image and the predicted image is obtained to obtain the residual image E(i, j), which is expressed by formula (2): E(i,j)=I origin_in (i,j)-X(i,j) (2) In step (1.2), use formula (1) to obtain the corresponding background prediction value as shown in formula (3): The maximum value among them is used as the final background grayscale prediction value, which is expressed by formula (4): X(i,j)=max{X m (i,j)},m=1,2,3,4 (4) In step (1.3), the grayscale average of all pixels in the prediction window S is used as the predicted value of the current background pixel. The closer to the prediction point, the smaller the corresponding weight. The value of the weight matrix is ​​shown in formula (5):

3. The method for calculating the trajectory of a small target based on identification and tracking according to claim 2, characterized in that: Step (2) further includes: In step (2.1), the segmentation between the target and the background adopts the threshold segmentation method, and the threshold is calculated by formula (6): Y=μ+kσ (6) Among them, μ represents the mean of the residual image, σ is the mean square error of the residual image, and k is the weight coefficient.

4. The method for calculating the trajectory of a small target based on identification and tracking according to claim 3, characterized in that: Step (2) further includes: In step (2.2), the segmentation between the target and the noise is performed using the connected domain labeling method. The image after the threshold segmentation in step (2.1) is labeled using the morphological four-connected domain labeling method. Each connected domain is assigned a label number. In order to eliminate additional noise, the target with a pixel value smaller than the threshold is discarded.

5. The method for calculating the trajectory of a small target based on identification and tracking according to claim 4, characterized in that: Step (3) further includes: The centroid positioning algorithm is used to locate the small target, and the square weighted centroid method is used for calculation. The calculation formula is shown in formula (7): Among them, E(x,y) represents the grayscale value occupied by the current target, and (x0,y0) is the center of mass coordinate of the small target.

6. The method for calculating the trajectory of a small target based on identification and tracking according to claim 5, characterized in that: Step (4) further includes: The center of mass coordinates obtained by the output of step (3) are used to compare the center of mass position of the current frame with the center of mass position of the previous frame. If the distance of the center of mass position change exceeds a certain threshold, the target is considered to be moving; otherwise, the target is considered to be stationary. The moving target is extracted and the estimated value of the previous moment is predicted and corrected based on the real-time observation value of the system through Kalman filtering.

7. The method for calculating the trajectory of a small target based on identification and tracking according to claim 6, characterized in that: Step (4) further includes: Step (4.1), prediction step In the prediction step, the nonlinear state transfer equation f(x) is used to predict the state and covariance. The state prediction equation and covariance prediction equation are written as shown in Equation (8) and Equation (9), respectively: Among them, F k is the Jacobian matrix of the state transfer function, and linearizing f(x) yields Equation (10):

8. The method for calculating the trajectory of a small target based on identification and tracking according to claim 7, characterized in that: Step (4) further includes: Step (4.2), update step In the update step, the nonlinear observation equation h(x) and the actual observation value z k Update state estimates and covariances; First, calculate the Kalman gain, as shown in formula (11): Then update the state as shown in formula (12): in is the measurement residual, which represents the difference between the observed value and the predicted value; Finally, the covariance is updated as shown in formula (13): P k|k =(I-K k H k )P k|k-1 (13) Among them, P k|k is the updated covariance matrix, which indicates the accuracy of the current state estimation; Repeat the above process iteratively to obtain the predicted trajectory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the steps of the small target trajectory calculation method based on recognition and tracking as described in any one of claims 1 to 8.

Images