Aircraft target tracking method and optical measuring device

Abstract

The present application relates to the technical field of target tracking, and particularly relates to a kind of aircraft target tracking method and optical measuring equipment, aircraft target tracking method includes: in the process that aircraft target is in first flight, according to current target miss distance, aircraft target is tracked;In the process that aircraft target is in middle flight, based on current target miss distance, pitch angle parameter of optical measuring equipment and azimuth angle parameter of optical measuring equipment, extrapolation motion trajectory is fitted, and aircraft target is tracked according to extrapolation motion trajectory;In the process that aircraft target is in final flight, centroid tracking algorithm is used to process current acquired image, to obtain second bounding box parameter, first bounding box parameter and second bounding box parameter are optimized, to obtain target bounding box parameter, and aircraft target is tracked according to target bounding box parameter.The present application is at least beneficial to improve the precision of aircraft target tracking.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target tracking, and particularly relates to a flying object tracking method and a light measurement device. BACKGROUND

[0002] The light measurement device tracks the target automatically by extracting the target off-target amount to observe the flight trajectory of the flying object in the whole flight process, and obtain the real-time image and position information of the flying object. However, the whole flight process of the flying object includes the first flight process, the middle flight process and the last flight process, which are referred to as the first flight process, the middle flight process and the last flight process for short. The first flight process, the middle flight process and the last flight process have the same background characteristics and morphological characteristics. The target passing through the cloud in the first flight process, the target separation in the middle flight process and the small target in the last flight process will interfere with the automatic tracking of the target, reduce the accuracy of the target tracking, and even cause the target to be lost. SUMMARY

[0003] Therefore, the application aims to provide a flying object tracking method and a light measurement device, which at least improve the accuracy of the flying object tracking.

[0004] To achieve the above-mentioned purpose, the technical scheme of the application is as follows:

[0005] The application provides a flying object tracking method. The flying object tracking method is based on the light measurement device to track the flying object in the whole flight process. The flying object tracking method includes: acquiring an image with a flying object in real time, processing the current acquired image by using a basic tracking algorithm to obtain first bounding box parameters, and obtaining a target off-target amount based on the first bounding box parameters; tracking the flying object according to the target off-target amount when the flying object is in the first flight process; fitting an extrapolated motion trajectory based on the target off-target amount, a pitch angle parameter of the light measurement device and an azimuth angle parameter of the light measurement device when the flying object is in the middle flight process, and tracking the flying object according to the extrapolated motion trajectory; processing the current acquired image by using a centroid tracking algorithm to obtain second bounding box parameters when the flying object is in the last flight process, optimizing the first bounding box parameters and the second bounding box parameters to obtain target bounding box parameters, and tracking the flying object according to the target bounding box parameters.

[0006] Further, the basic tracking algorithm includes a YOLO-GD algorithm. The YOLO-GD algorithm is a YOLO-v7 target detection algorithm combined with one-dimensional gradient operation. The current acquired image is processed by using the YOLO-GD algorithm to obtain the first bounding box parameters.

[0007] Further, the YOLO-GD algorithm adopts an offline training YOLO-GD model and an online calling YOLO-GD model to process the image; the YOLO-GD model comprises a feature extraction backbone network, a feature fusion network and a head detection network; the feature extraction backbone network comprises a plurality of convolution layers conv, a plurality of normalization layers BN, a plurality of Mish activation functions, a plurality of L-relu activation functions and a fully connected layer Dense; the feature fusion network comprises an up-sampling module, a plurality of convolution layers conv, a plurality of fully connected layers Dense, a plurality of normalization layers BN and a plurality of L-relu activation functions; the head detection network comprises a plurality of global pooling layers, a plurality of convolution layers conv, a plurality of fully connected layers Dense, a plurality of normalization layers BN and a plurality of L-relu activation functions; the calling YOLO-GD model to process the image comprises: sending one-dimensional gradient information of the image and the image into the feature extraction backbone network.

[0008] Further, the first bounding box parameter comprises a first bounding box coordinate (x, y, w, h), wherein (x, y) is a relative upper left corner coordinate of the first bounding box, w is a width of the first bounding box, and h is a height of the first bounding box; and the target miss distance is obtained based on the first bounding box parameter, comprising: calculating the target miss distance based on the first bounding box coordinate (x, y, w, h) according to formula 1, formula 1 being as follows:

[0009]

[0010]

[0011] Wherein, ΔA and ΔE are the target miss distance.

[0012] Further, the extrapolated motion trajectory is obtained by fitting, comprising: obtaining a comprehensive angle value based on the target miss distance, the pitch angle parameter and the azimuth angle parameter according to formula 2, formula 2 being as follows:

[0013]

[0014] Wherein, ΔA and ΔE are the target miss distance, A zong and E zong are the comprehensive angle value, A e is the azimuth angle parameter, E e is the pitch angle parameter, and μ is the pixel size of the detector of the optical measuring device, and f is the current focal length of the optical measuring device; and the extrapolated motion trajectory is obtained by fitting using the Newton interpolation method according to the comprehensive angle value.

[0015] Further, the extrapolated motion trajectory is obtained by fitting using the Newton interpolation method according to the comprehensive angle value, comprising: fitting the extrapolated motion trajectory using an extrapolation formula of the Newton interpolation method, the extrapolation formula being as follows:

[0016] A zong (x) = A zong (x0)+a1(x-x0)+a2(x-x0)(x-x1)+...+a n (x-x0)(x-x1)...(xx n );

[0017] E zong (x) = E zong (x0)+b1(x-x0)+b2(x-x0)(x-x1)+...+b n (x-x0)(x-x1)...(xx n );

[0018] Among them, a0, a1...a n and b0,b1...b n All are first-order difference quotients;

[0019]

[0020] i=1, 2, 3...n, x0, x1...x n A is the acquisition time arranged in chronological order. zong (x) and E zong (x) is the extrapolated motion trajectory, A zong (x) is the predicted value in azimuth, E zong (x) is the predicted value at the pitch angle, A zong (x i ) is the acquisition time x i A zong , E zong (x i ) is the acquisition time x i E zong .

[0021] Furthermore, the second bounding box parameters include the second miss distance center coordinates (x c ,y c ), the second bounding box width W c , the second bounding box height H c and the second effective value V2; using the centroid tracking algorithm to process the currently acquired image includes: acquiring the pixel point (x i ,y j ) and the corresponding grayscale value I(x i ,y j ), according to formula 3, the center coordinate of the second miss distance (x c ,y c ), Formula 3 is as follows:

[0022]

[0023]

[0024] Among them, N is the width of the image, M is the height of the image, I(x i ,y j ) is the pixel point (x i ,y j ) grayscale value; the second bounding box width W c is the number of consecutive pixels whose grayscale values ​​are greater than the grayscale threshold in the horizontal direction, and the height of the second bounding box H c It is the number of consecutive pixels in the vertical direction whose grayscale values ​​are greater than the grayscale threshold. The second valid value V2 is true or false.

[0025] Furthermore, the first bounding box parameters include the first bounding box coordinates (x, y, w, h) and a first valid value V1, the first valid value V1 is true or false; the first miss distance center coordinates (x, y, w, h) are obtained according to the first bounding box coordinates (x, y, w, h). b ,y b ), first bounding box width W b and the first bounding box height H b , where x b =x,y b =y,W b =w,H b =h; optimizing the first bounding box parameter and the second bounding box parameter to obtain the target bounding box parameter includes: if V1 and V2 are both true, judging the relationship between the distance d between the first miss distance center and the second miss distance center and the threshold distance D, if d is less than D, using the first bounding box parameter as the target bounding box parameter, if d is greater than D, calculating the difference S1 of the one-dimensional gradient values ​​of the two frames before and after the input centroid tracking algorithm, calculating the difference S2 of the one-dimensional gradient values ​​of the two frames before and after the input base tracking algorithm, comparing S1 and S2, and selecting the bounding box parameter obtained by the algorithm corresponding to the smaller value as the target bounding box parameter; if one of V1 and V2 is true and the other is false, selecting the bounding box parameter corresponding to true as the target bounding box parameter; if both V1 and V2 are false, performing target tracking based on the external guidance data sent by the base center.

[0026] Furthermore, the difference S1 of the one-dimensional gradient values ​​of the two frames before and after the input to the centroid tracking algorithm is calculated according to Formula 4, and the difference S2 of the one-dimensional gradient values ​​of the two frames before and after the input to the basic tracking algorithm is calculated according to Formula 5;

[0027] Formula 4 is as follows:

[0028] Formula 5 is as follows:

[0029] wherein, is a first-order gradient value of a current frame image, is a first-order gradient value of a previous frame image.

[0030] The application creates another aspect to provide an optical measurement device, comprising: an optical system, a sensor, an encoder, an electronic processing unit, an image processing unit, a master control unit and a mechanical structure unit, the mechanical structure unit at least comprising a servo, a basic mechanical structure and a base, the optical measurement device being used to execute the aircraft target tracking method described above.

[0031] Compared with the prior art, the application can achieve the following beneficial effects: the aircraft target tracking method provided by the embodiments of the application is beneficial to improve the adaptability of the tracking mode under target state change and background transformation; in addition, on the basis of the basic tracking algorithm, the centroid tracking algorithm is combined, which is beneficial to improve the tracking precision and make the wave door more stable in the process of tracking the aircraft target; for the short-term target information caused by the aircraft target penetrating the cloud and the separation zone, which is not obvious or disappears, the target can be continuously tracked by extrapolating the motion trajectory; in the final flight process of the aircraft target, the automatic switching of the tracking mode can be realized, which is beneficial to prevent the operation of the operator and reduce the labor loss. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings, which form a part of the present application, are used to provide a further understanding of the present application, and the illustrative embodiments of the present application and their description serve the purpose of explaining the present application. The accompanying drawings should not be used to limit the present application unduly. In the drawings:

[0033] Figure 1 A full segment tracking strategy diagram of the aircraft target tracking method described in the embodiments of the present application;

[0034] Figure 2 A framework diagram of the YOLO-GD model described in the embodiments of the present application;

[0035] Figure 3 A position relationship diagram of the first boundary box and the image described in the embodiments of the present application;

[0036] Figure 4 A position diagram of the target point described in the embodiments of the present application.

[0037] Legend of the drawings: 10, image; 11, first boundary box. DETAILED DESCRIPTION

[0038] It is found through analysis that the current aircraft target tracking technology has the following problems: poor adaptability to the deformation and attitude change of the aircraft target, specifically, as the flight process of the aircraft target progresses, the attitude change of the aircraft target may cause the aircraft target in the image to deform at the observation angle of the photoelectric theodolite, which requires the basic tracking algorithm to process the non-rigid deformation of the image; occlusion and overlap easily mis-track the aircraft target, specifically, the aircraft target may be occluded and overlapped in the flight process, that is, the aircraft target may be occluded and overlapped, which may interfere with the basic tracking algorithm and cause mis-tracking of the aircraft target; the initialization of the basic tracking algorithm usually requires an accurate target position, and as time goes on, the basic tracking algorithm needs to update the target model to adapt to the appearance change of the target, and the initialization and model updating need to be performed multiple times in the whole process to adapt to the appearance change of the target, the operation of updating the target model of the basic tracking algorithm needs manual judgment, and the initialization operation and model updating need to be manually judged in advance; the data set of the basic tracking algorithm has poor coverage in actual application, that is, the data set may not completely cover all scenarios in actual application, resulting in a gap between the performance of the basic tracking algorithm in actual application and the expectation; the aircraft target is an infrared target, the infrared target contains less feature information, and has no texture, color, etc., only gray information and irregular shape information, which is not conducive to the training of the model parameters of the basic tracking algorithm.

[0039] To solve the above problems, an aircraft target tracking method is provided in the embodiments of the present application, which formulates different tracking methods for different stages in the whole flight process of the aircraft target, specifically, a deep learning algorithm YOLO-GD algorithm is used to detect the texture and shape of the aircraft target in the first stage, which effectively adapts to the identification of the aircraft target at different flight angles and prevents the aircraft target from being wrongly tracked in the tracking process; on the basis of the YOLO-GD algorithm, the encoder value of the optical measurement equipment is combined to fit the extrapolated motion trajectory of the aircraft target, wherein the encoder value is the pitch angle parameter and the azimuth angle parameter, after the extrapolated motion trajectory is obtained, the accuracy of the extrapolated motion trajectory is verified according to the trajectory information sent in advance by the base center, the aircraft target is tracked according to the verified extrapolated motion trajectory, and the target is prevented from being wrongly tracked in the middle stage; in the last flight process of the aircraft target, the tracking results obtained by the centroid tracking algorithm and the tracking results obtained by the YOLO-GD algorithm are judged, the optimal tracking strategy is automatically matched in real time according to the quality of each information source, and real-time automatic switching is realized. The embodiments of the present application ensure stable tracking without single-pole correction in the whole process after the aircraft target takes off, single-pole correction refers to the process of manual operation adjustment, and truly realizes stable and automatic tracking in the whole process after take-off and emergency handling of common situations, so as to realize more accurate, more robust and more real-time tracking effect.

[0040] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not constitute a limitation on the present application.

[0041] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0042] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited by "first", "second" and the like can explicitly or implicitly include one or more features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0043] In the description of the present application, it should be noted that unless otherwise specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0044] The present application will be described in detail below with reference to the drawings and embodiments.

[0045] The present application provides an aircraft target tracking method, which is based on an optical measurement device for tracking an aircraft target throughout the whole segment. The optical measurement device at least includes an optical system, a sensor, an encoder, an electronic processing unit, an image processing unit, a main control unit and a mechanical structure unit. The mechanical structure unit at least includes a servo, a basic mechanical structure and a base.

[0046] In some embodiments of the present application, the optical measuring device is an electro-optical theodolite, which is a high-precision optical measuring device combining optical and electronic technologies for measuring the azimuth and elevation angles of an object and widely used in engineering surveying, geographic information systems, astronomical observation and military reconnaissance.

[0047] When the aircraft target tracking method provided by the present application is performed by using the optical measuring device, the aircraft target is imaged on the target surface of the sensor through the optical system, then the optical signal is converted into an electrical signal by the electronic processing unit, the image is acquired by the acquisition card, and the image is processed by the algorithm in the image processing unit to determine the spatial position of the target, which is transmitted to the servo in the mechanical structure unit through the serial port, the servo controls the basic mechanical structure to control the optical measuring device to track the target and obtain the next frame of image, and the process is repeated to form a closed-loop automatic tracking.

[0048] It should be noted that the algorithm in the image processing unit at least includes a basic tracking algorithm, a centroid tracking algorithm and other auxiliary algorithms, and the algorithm in the image processing unit is used to implement the aircraft target tracking method provided by the present application.

[0049] Reference Figure 1 The aircraft target tracking method comprises the following steps: acquiring images with the aircraft target in real time, processing the currently acquired images by using a basic tracking algorithm to obtain first bounding box parameters, and obtaining a target miss distance based on the first bounding box parameters; tracking the aircraft target according to the target miss distance in the process of the first stage of flight of the aircraft target; fitting an extrapolated motion trajectory based on the target miss distance, an elevation angle parameter of the optical measuring device and an azimuth angle parameter of the optical measuring device in the process of the middle stage of flight of the aircraft target, and tracking the aircraft target according to the extrapolated motion trajectory; processing the currently acquired images by using a centroid tracking algorithm to obtain second bounding box parameters in the process of the last stage of flight of the aircraft target, optimizing the first bounding box parameters and the second bounding box parameters to obtain target bounding box parameters, and tracking the aircraft target according to the target bounding box parameters.

[0050] The aircraft target tracking method provided by the present application has different tracking methods for different stages in the whole flight process of the aircraft target, the first stage uses a basic tracking algorithm to track the aircraft target, the middle stage tracks the aircraft target according to an extrapolated motion trajectory obtained by fitting the extrapolated motion trajectory based on the basic tracking algorithm, the elevation angle parameter and the azimuth angle parameter, so as to prevent the target from being mistaken in the middle stage tracking process, and the last stage of flight of the aircraft target judges the tracking results obtained by the centroid tracking algorithm and the basic tracking algorithm, realizes automatic matching of the optimal tracking strategy according to the quality of each information source in real time, and automatically switches in real time.

[0051] It should be noted that the first bounding box parameter represents the parameter of the first bounding box for framing the aircraft target, and determining the first bounding box parameter can determine the first bounding box, and the target miss distance is the target position data calculated based on the first bounding box parameter for sending to the servo. The second bounding box parameter represents the parameter of the second bounding box for framing the aircraft target, and determining the second bounding box parameter can determine the second bounding box.

[0052] Further, the basic tracking algorithm includes: YOLO-GD (You Only Look Once version gradient) algorithm, the YOLO-GD algorithm is a YOLO-v7 (You Only Look Once version 7) target detection algorithm combined with one-dimensional gradient operation; the YOLO-GD algorithm is used to process the current acquired image to obtain the first bounding box parameter.

[0053] Further, the YOLO-GD algorithm adopts an offline training YOLO-GD model, and processes the image in an online calling manner of the YOLO-GD model; the YOLO-GD model includes a feature extraction backbone network, a feature fusion network and a head detection network; the feature extraction backbone network includes a plurality of convolution layers conv, a plurality of normalization layers BN, a plurality of Mish activation functions, a plurality of L-relu (leaky Rectified Linear Unit, leaky rectified linear unit) activation functions and a fully connected layer Dense, which is conducive to improving the depth of the feature extraction backbone network for extracting target features; the feature fusion network includes an up-sampling module (Upsampling), a plurality of convolution layers conv, a plurality of fully connected layers Dense, a plurality of normalization layers BN and a plurality of L-relu activation functions; the head detection network includes a plurality of global pooling layers (Global pool Layer), a plurality of convolution layers conv, a plurality of fully connected layers Dense, a plurality of normalization layers BN and a plurality of L-relu activation functions; processing the image by calling the YOLO-GD model includes: sending one-dimensional gradient information of the image and the image into the feature extraction backbone network together.

[0054] The YOLO-GD algorithm is a basic tracking algorithm, and the YOLO-GD algorithm is used as the basic tracking algorithm because: when the first-stage aircraft target emerges from the mountain, the image collected by the optical measurement device is an infrared image, in the infrared image, the mountain presents a high gray feature due to the sunlight during the day, and the traditional tracking method will have a situation of mistakenly grasping the target, since the first-stage optical measurement device is relatively close to the aircraft target, the pixel size of the aircraft target under the visible condition is greater than 32*32 pixels, and the aircraft target has more feature information such as attitude and shape, therefore, in order to improve the anti-interference performance of the algorithm, an improved YOLO-GD algorithm is provided by combining the YOLO-v7 target detection algorithm and gradient operation, and is used for quickly and accurately identifying the aircraft target in the image, wherein the YOLO-GD algorithm comprises one-dimensional gradient operation, and the information input into the YOLO-GD model is increased by one-dimensional gradient information, so that the YOLO-GD model can be better trained.

[0055] It can be understood that in the process of offline training the YOLO-GD model: the one-dimensional gradient operation is performed on the data set input into the YOLO-GD algorithm, and the one-dimensional gradient information obtained by the one-dimensional gradient operation is input into the YOLO-GD algorithm, so as to distinguish the background and the target in the subsequent process, and the data set of the YOLO-GD algorithm can better cover all scenes in actual application, and the one-dimensional gradient operation specifically comprises: calculating the first derivative of an image f(x 像 ,y 像 ), wherein x 像 , y 像 represent pixel points of the image, the first derivative of the image f(x 像 ,y 像 ) is a vector, which contains the maximum change rate of the image in each direction, for a two-dimensional image f(x 像 ,y 像 ), the first derivative of the image f(x 像 ,y 像 ) is composed of the change rate in the horizontal direction (i.e., the partial derivative with respect to x) and the change rate in the vertical direction (i.e., the partial derivative with respect to y). The first derivative of the image f(x 像 ,y 像 ) can be used to estimate the brightness change, and the point with the fastest brightness change is the edge pixel of the target, and the first derivative of the image f(x 像 ,y 像 ) is calculated by the following formula:

[0056]

[0057] wherein, is the image f(x 像 ,y像 ) with respect to the partial derivative of x, is the image f(x 像 ,y 像 ) with respect to the partial derivative of y.

[0058] In some embodiments of the present invention, reference Figure 2 The YOLO-GD model includes a feature extraction backbone network, a feature fusion network, and a head detection network. The feature extraction backbone network includes 19 convolutional layers (conv), 8 normalization layers (BN), 3 Mish activation functions, 3 L-relu activation functions, and a fully connected layer (Dense), which is conducive to improving the depth of target feature extraction by the feature extraction backbone network; the feature fusion network includes an upsampling module, 6 convolutional layers (conv), 2 fully connected layers (Dense), 2 normalization layers (BN), and 2 L-relu activation functions, which divides the image into three scales of 7×7 pixels, 14×14 pixels, and 28×28 pixels for output, which is conducive to strengthening the training of different feature dimensions and fully learning the shape and size of aircraft targets at different times; the head detection network includes multiple global pooling layers, multiple convolutional layers (conv), multiple fully connected layers (Dense), multiple normalization layers (BN), and multiple L-relu activation functions. The head detection network is used to predict target type and target location, output multiple candidate bounding box parameters, screen the candidate bounding box parameters, and then obtain the first bounding box parameters.

[0059] It should be noted that Figure 2 The lower left box shows Figure 2 The specific units included in the CBM module, CCC module, CBL module and DCC module of the YOLO-GD model. The "×3" marked above the modules in the backbone network and head detection network represents the number of modules multiplied by 3.

[0060] In addition, the candidate bounding box parameters typically include: the coordinates of the candidate bounding box, the category probability, and the target existence confidence. The category probability is a vector that represents the probability that the object contained in the candidate bounding box belongs to different categories.

[0061] The specific basic steps of screening a plurality of candidate bounding box parameters to obtain the first bounding box parameter can be as follows: the YOLO-GD algorithm can adjust the input image to 416x416 pixels in size, and then divide the entire image into 13x13 grids on average, and each grid corresponds to an image piece that can be responsible for predicting three candidate bounding boxes. For each candidate bounding box, the candidate bounding box parameters are predicted, specifically including the class probability in the candidate bounding box, the target existence confidence, and the candidate bounding box coordinates. A plurality of candidate bounding box parameters are compared, and the candidate bounding box parameters corresponding to the maximum target existence confidence are screened out. According to the class probability in the candidate bounding box parameters, the candidate bounding box is divided into a target or a background, and the candidate bounding box corresponding to the target is taken as the first bounding box, and the candidate bounding box parameter corresponding to the target is taken as the first bounding box parameter.

[0062] Further, the first bounding box parameter includes first bounding box coordinates (x, y, w, h), wherein, with reference to Figure 3 , (x, y) is the position coordinates of the first bounding box 11 relative to the upper left corner of the image 10, w is the width of the first bounding box 11, and h is the height of the first bounding box 11; obtaining the target miss distance based on the first bounding box parameter includes: calculating the target miss distance based on the first bounding box coordinates (x, y, w, h) according to formula 1, and formula 1 is as follows:

[0063]

[0064]

[0065] Wherein, ΔA and ΔE are the target miss distance, ΔA is the azimuth angle offset of the aircraft target center relative to the center of the field of view, and ΔE is the pitch angle offset of the aircraft target center relative to the center of the field of view.

[0066] Specifically, the YOLO-GD algorithm can be input with an image at a frequency of 50hz, and the first bounding box coordinates (x, y, w, h) can be obtained by performing bounding box prediction on each frame of image, and the target miss distance ΔA and ΔE can be obtained. The processing time of each frame of image is usually no more than 15ms.

[0067] For the mid-flight process of the aircraft target, the pitch angle parameter and the azimuth angle parameter are used to assist the basic tracking algorithm to track the target, so as to prevent the fairing cover of the aircraft target from separating and causing the basic tracking algorithm to grab the wrong target. It should be noted that when the aircraft target enters the cloud and the aircraft target separates, the target miss distance of the aircraft target will be lost, and in this case, the predicted value obtained by extrapolating the motion trajectory is used as a temporary miss distance for target tracking.

[0068] Furthermore, fitting the extrapolated motion trajectory includes: obtaining a comprehensive angle value according to Formula 2 based on the target miss distance, the pitch angle parameter, and the azimuth angle parameter, wherein the target miss distance is historical data, and the comprehensive angle value is calculated based on the historical target miss distance. Formula 2 is as follows:

[0069]

[0070] Among them, ΔA and ΔE are the target off-target amounts, A zong and E zong is the comprehensive angle value, A zong Indicates the comprehensive azimuth angle generated by the target point moving in the horizontal direction, E zong Indicates the comprehensive pitch angle generated by the target point moving in the vertical direction, A e is the azimuth parameter, E e is the pitch angle parameter, μ is the pixel size of the detector of the optical measurement device, and f is the current focal length of the optical measurement device. The extrapolated motion trajectory is obtained by fitting the comprehensive angle value using the Newton interpolation method.

[0071] Specifically, refer to Figure 4 ,like Figure 4 As shown in (a), point O is the optical center of the optical measurement device; point M is the center of the aircraft target, referred to as the target point. The distance between the target point M and the optical center O is R, the projection of the target point M on the XOZ plane is M', ∠MOM' is the pitch angle E generated by the vertical movement of the target point M, and the angle between the line connecting the projection point M' and the optical center O and the x-axis is the azimuth angle A generated by the horizontal movement of the target point. Figure 4 As shown in (b), in the image processing unit, the target miss distance is the azimuth offset and pitch angle offset of the aircraft target center relative to the center of the field of view. When the aircraft target deviates, the optical measurement device tracks the aircraft target and records the azimuth offset ΔA and pitch angle offset ΔE of the target point relative to the center of the camera (sensor) crosshairs, which is the target miss distance. The azimuth parameter A e Indicates the azimuth angle of the encoder. The encoder is the encoder of the optical measurement device. The pitch angle parameter E e Indicates the encoder pitch angle.

[0072] Furthermore, the extrapolated motion trajectory is obtained by fitting the comprehensive angle value using the Newton interpolation method, including: zong and E zong Corresponding to the acquisition time, the extrapolated motion trajectory is obtained by fitting the extrapolation formula of the Newton interpolation method. The extrapolation formula is as follows:

[0073] A zong (x) = A zong(x0)+a1(x-x0)+a2(x-x0)(x-x1)+...+a n (x-x0)(x-x1)...(xx n );

[0074] E zong (x) = E zong (x0)+b1(x-x0)+b2(x-x0)(x-x1)+...+b n (x-x0)(x-x1)...(xx n );

[0075] Among them, a0, a1...a n and b0,b1...b n are all first-order difference quotients;

[0076]

[0077] i=1, 2, 3...n-1, x0, x1...x n A is the acquisition time arranged in chronological order. zong (x) and E zong (x) is the extrapolated motion trajectory, A zong (x) is the predicted value in azimuth, E zong (x) is the predicted value at the pitch angle, A zong (x i ) is the acquisition time x i A zong , E zong (x i ) is the acquisition time x i E zong .

[0078] It should be noted that after obtaining the extrapolated motion trajectory, it is necessary to use the ballistic information sent in advance by the base center to verify the accuracy of the extrapolated motion trajectory to ensure that the extrapolated motion trajectory can more accurately simulate the actual motion trajectory. Among them, the ballistic information is usually the motion trajectory data of the aircraft target measured by the base center through radar or other means.

[0079] When the aircraft target is in mid-flight and the target is lost, an embodiment of the present invention uses an extrapolated motion trajectory to predict the spatial position of the aircraft target in the next frame, transmits the extrapolated motion trajectory to the servo, and tracks the aircraft target. When the target is not lost, the target miss distance is used to track the aircraft target.

[0080] The aircraft target is in the process of terminal flight, the attitude feature of the aircraft target itself is not obvious, and the aircraft target as a whole presents in the form of a round dot with a tail flame, therefore, a tracking strategy of automatically switching between a centroid tracking algorithm and a YOLO-GD algorithm is adopted to track the aircraft target.

[0081] Further, the second boundary box parameters obtained by the centroid tracking algorithm include a second center coordinate (x c ,y c ) of a second miss distance, a second boundary box width W c , a second boundary box height H c , and a second effective value V2; processing the current acquired image by using the centroid tracking algorithm includes: acquiring pixel point coordinates (x i ,y j ) and corresponding gray values I(x i ,y j ) of the image, considering that the gray value of a pixel point is usually in the range of 0 to 255, therefore, the gray value of each pixel point needs to be weighted, that is, the second center coordinate (x c ,y c ) of the second miss distance is obtained according to formula 3, and formula 3 is as follows:

[0082]

[0083]

[0084] wherein, N is the width of the image, M is the height of the image, I(x i ,y j ) is the gray value of the pixel point (x i ,y j ) in the image; the second boundary box width W c is the number of continuous pixel points in the horizontal direction whose gray value is greater than a gray threshold value, the second boundary box height H c is the number of continuous pixel points in the vertical direction whose gray value is greater than the gray threshold value, and the second effective value V2 is true or false, wherein the gray threshold value = (maximum gray value of the image - minimum gray value of the image) / 2 + gray mean value of the image.

[0085] Further, the first boundary box parameters include a first boundary box coordinate (x, y, w, h) and a first effective value V1, and the first effective value V1 is true or false; a first center coordinate (x b ,y b ) of a first miss distance, a first boundary box width W b , and a first boundary box height H b are obtained according to the first boundary box coordinate (x, y, w, h), wherein x b =x, y b =y, Wb = w, H b = h. It should be noted that the optical measurement device transmits the image into the image processing unit in real time through the sensor and the electronic processing unit, and the image transmitted into the image processing unit is processed in two ways, which are the YOLO-GD algorithm and the centroid tracking algorithm. The method for determining the second effective value V2 is: judging whether the number of pixel points greater than the gray threshold in the second boundary box is greater than 10, if yes, it is judged to be effective, V2 is true, if not, V2 is false, and 10 is the minimum value of the target size of the aircraft target in the field of view range.

[0086] The method for determining the first effective value V1 is: judging whether the number of pixel points greater than the gray threshold in the first boundary box is greater than 10, if yes, it is judged to be effective, V1 is true, if not, V1 is false.

[0087] Further, the first boundary box parameter and the second boundary box parameter are optimized to obtain the target boundary box parameter, which specifically includes:

[0088] According to V1 and V2, specifically, if V1 and V2 are both true, the relationship between the distance d between the first miss distance center and the second miss distance center and the threshold distance D is judged, wherein D = N / 2, N is the width of the image, if d is less than D, it means that the positions extracted by the two are close, and the first boundary box parameter is taken as the target boundary box parameter, if d is greater than D, it means that one of them misextracts the target, therefore, the difference S1 of the one-dimensional gradient values of the two frames of images input into the centroid tracking algorithm is calculated, the difference S2 of the one-dimensional gradient values of the two frames of images input into the basic tracking algorithm is calculated, the size of S1 and S2 is compared, the smaller the value represents the smaller the target shape change, and the boundary box parameter obtained by the algorithm corresponding to the smaller value is selected as the target boundary box parameter, which is subsequently sent to the servo system for guiding the movement of the optical measurement device, and then tracking the aircraft target;

[0089] If one of V1 and V2 is true and the other is false, the boundary box parameter corresponding to true is selected as the target boundary box parameter;

[0090] If V1 and V2 are both false, the target tracking is carried out based on the external guidance data issued by the base center.

[0091] It should be noted that after obtaining the target boundary box parameter, it is necessary to determine whether the current target boundary box parameter and the historical data can be fitted into a complete curve, if there is a jump point, the target boundary box parameter is not selected as the basis for tracking the aircraft target, and the target tracking is carried out based on the external guidance data issued by the base center.

[0092] Further, the difference S1 of the one-dimensional gradient values of the two images before and after the input centroid tracking algorithm is calculated according to formula 4, and the difference S2 of the one-dimensional gradient values of the two images before and after the input base tracking algorithm is calculated according to formula 5.

[0093] Formula 4 is as follows:

[0094] Formula 5 is as follows:

[0095] wherein, is the first-order gradient value of the current frame image, is the first-order gradient value of the previous frame image, N is the width of the image, and M is the height of the image.

[0096] In some embodiments of the present application, the base center sends a time stamp to the image processing unit at the time when the aircraft enters the first, middle and last segments, and the image processing unit switches the tracking strategy according to the corresponding time of the time stamp.

[0097] The present application further provides an optical measurement device for performing the aircraft target tracking method.

[0098] In some embodiments of the present application, the optical measurement device at least includes an optical system, a sensor, an encoder, an electronic processing unit, an image processing unit, a main control unit and a mechanical structure unit. The mechanical structure unit at least includes a servo, a basic mechanical structure and a base.

[0099] When the aircraft target tracking method provided by the present application is performed by the optical measurement device, the aircraft target is imaged on the target surface of the sensor through the optical system, then the optical signal is converted into an electrical signal by the electronic processing unit, the image is acquired by the acquisition card, and the image is processed by the algorithm in the image processing unit to determine the spatial position of the target. The spatial position is transmitted to the servo in the mechanical structure unit through the serial port, the servo controls the basic mechanical structure to control the optical measurement device to track the target and obtain the next frame of image, and the process is repeated to form a closed-loop automatic tracking.

[0100] The embodiment of the present application provides a full-segment aircraft target tracking method, which formulates an optimal tracking strategy in combination with the different backgrounds and target characteristics of the first, middle and last segments.

[0101] Specifically, the method uses an optoelectronic theodolite to configure an infrared detector (sensor), and formulates an optimal tracking algorithm strategy according to different backgrounds and target characteristics of the first segment, the middle segment and the tail segment. The first segment target shape is obvious, and the background is cloudy, therefore, the first segment uses a deep learning algorithm YOLO-GD algorithm improved on the basis of YOLO-v7 to detect the texture and shape of the target, effectively adapting to the recognition of the aircraft target at different angles and preventing the aircraft target from being mistaken in the tracking process; the middle segment aircraft target separates, and there are interference targets such as boosters and fairings, therefore, the middle segment obtains the azimuth value and the pitch value of the encoder on the basis of the YOLO-GD algorithm, and based on the azimuth value, the pitch value and the target miss distance obtained by the YOLO-GD algorithm, an extrapolated motion trajectory of the aircraft target is fitted, and the aircraft target is screened according to the coincidence degree of the current frame of the aircraft target in the trajectory and the extrapolated motion trajectory, to prevent mistaken grabbing; the target in the tail segment is far away, therefore, on the basis of the YOLO-GD algorithm, a pixel-level centroid tracking algorithm is combined to judge the two, and the tracking of the aircraft target is performed according to the judgment result. At the same time, multi-dimensional position judgment can be performed in combination with external guidance of the equipment in the whole process, that is, the optical measurement equipment is allowed to receive external guidance data sent by the base center to control the optical measurement equipment to realize tracking correction of the aircraft target. It should be noted that the optical measurement equipment receives the external guidance data sent by the base center through the network.

[0102] To sum up, the embodiment of the application ensures that the optical measurement equipment can automatically match the basic tracking algorithm in real time according to the time stamp and automatically switch in real time during the whole process after the aircraft target takes off, truly realizes stable automatic tracking and emergency processing of common situations during the whole process after taking off, and realizes more accurate, more robust and more real-time tracking effect.

[0103] It should be understood that the various forms of flow shown above can be reordered, added or deleted steps. For example, the steps described in the present disclosure can be executed in parallel, in sequence or in different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, which is not limited herein.

[0104] The above specific embodiments do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An aircraft target tracking method, characterized by, The aircraft target tracking method is based on full-segment tracking of an aircraft target by an optical measurement device, and the aircraft target tracking method comprises: real-time acquisition of an image having the aircraft target, processing of the currently acquired image by using a basic tracking algorithm to obtain first bounding box parameters, and obtaining of a target miss distance based on the first bounding box parameters; during a process in which the aircraft target is in a first flight segment, tracking of the aircraft target according to the target miss distance; during a process in which the aircraft target is in a middle flight segment, fitting of an extrapolated motion trajectory based on the target miss distance, a pitch angle parameter of the optical measurement device, and an azimuth angle parameter of the optical measurement device, and tracking of the aircraft target according to the extrapolated motion trajectory; during a process in which the aircraft target is in a last flight segment, processing of the currently acquired image by using a centroid tracking algorithm to obtain second bounding box parameters, optimization of the first bounding box parameters and the second bounding box parameters to obtain target bounding box parameters, and tracking of the aircraft target according to the target bounding box parameters.

2. The aircraft target tracking method of claim 1, wherein, The basic tracking algorithm comprises a YOLO-GD algorithm, which is a YOLO-v7 target detection algorithm combined with one-dimensional gradient operation. The YOLO-GD algorithm is used to process the currently acquired image to obtain the first bounding box parameters.

3. The aircraft target tracking method according to claim 2, wherein, The YOLO-GD algorithm processes the image in an offline training YOLO-GD model and online calling YOLO-GD model manner; The YOLO-GD model comprises a feature extraction backbone network, a feature fusion network, and a head detection network; The feature extraction backbone network comprises a plurality of convolution layers conv, a plurality of normalization layers BN, a plurality of Mish activation functions, a plurality of L-relu activation functions, and a fully connected layer Dense; The feature fusion network comprises an up-sampling module, a plurality of convolution layers conv, a plurality of fully connected layers Dense, a plurality of normalization layers BN, and a plurality of L-relu activation functions; The head detection network comprises a plurality of global pooling layers, a plurality of convolution layers conv, a plurality of fully connected layers Dense, a plurality of normalization layers BN, and a plurality of L-relu activation functions; Calling the YOLO-GD model to process the image comprises feeding one-dimensional gradient information of the image and the image into the feature extraction backbone network together.

4. The aircraft target tracking method according to claim 1, 2 or 3, characterized in that, The first bounding box parameters comprise first bounding box coordinates (x, y, w, h), wherein (x, y) is the position coordinates of the first bounding box relative to the top left corner of the image, w is the width of the first bounding box, and h is the height of the first bounding box. Obtaining the target miss distance based on the first bounding box parameters comprises calculating the target miss distance based on the first bounding box coordinates (x, y, w, h) according to Formula 1, and the Formula 1 is as follows: wherein ΔA and ΔE are the target miss distance.

5. The aircraft target tracking method according to claim 4, wherein, The fitting of the extrapolated motion trajectory comprises: obtaining a comprehensive angle value according to Formula 2 based on the target miss distance, the pitch angle parameter and the azimuth angle parameter, the Formula 2 being as follows: wherein, ΔA and ΔE are the target miss distance, A zong and E zong are the integrated angle value, A e is the azimuth angle parameter, E e is the elevation angle parameter, μ is the pixel size of the detector of the photometric device, and f is the current focal length of the photometric device. The extrapolated motion trajectory is fitted according to the comprehensive angle value by using a Newton interpolation method.

6. The aircraft target tracking method according to claim 5, wherein, The fitting of the extrapolated motion trajectory according to the comprehensive angle value by using the Newton interpolation method comprises: The extrapolated motion trajectory is fitted by using an extrapolation formula of the Newton interpolation method, the extrapolation formula being as follows: A zong (x) = A zong (x0) + a1(x - x0) + a2(x - x0)(x - x1) +... + a n (x - x0)(x - x1)...(x - xn) = 0 n ); E zong (x) = E zong (x0) + b1(x - x0) + b2(x - x0)(x - x1) +... + b n (x - x0)(x - x1)...(x - x n ); wherein a0, a1...a n and b0, b1...b n are first-order difference quotients; i = 1, 2, 3...n, xo, xi... xn n is the acquisition time in chronological order, A zong (x) and E zong (x) is the extrapolated motion trajectory, A zong (x) is the predicted value in azimuth angle, E zong (x) is the predicted value in elevation angle, A zong (x i ) is the acquisition time x i of A zong , E zong (x i ) is the acquisition time x i of E zong .

7. The aircraft target tracking method of claim 1, wherein, The second bounding box parameter includes a second off-target amount center coordinate (x c ,y c ), a second bounding box width W c , a second bounding box height H c , and a second effective value V2; The processing of the current acquired image by using the centroid tracking algorithm comprises: acquiring pixel points (x i ,y j ) and corresponding gray values I(x i ,y j ) of the image, and obtaining the second miss distance center coordinates (x c ,y c ) according to formula 3, wherein the formula 3 is as follows: where N is the width of the image, M is the height of the image, I(x i ,y j ) is the gray value of the pixel point (x i ,y j ) in the image. The second boundary box width W c The number of continuous pixel points with a gray value greater than a gray threshold in the transverse direction, the second boundary box height H c The number of continuous pixel points with a gray value greater than a gray threshold in the longitudinal direction, the second effective value V2 is true or false.

8. The aircraft target tracking method according to claim 7, wherein, The first bounding box parameter comprises a first bounding box coordinate (x, y, w, h) and a first valid value V1, the first valid value V1 being true or false; The first miss distance center coordinates (x, y, w, h) are obtained according to the first bounding box coordinates (x, y, w, h) b ,y b ), first bounding box width W b and the first bounding box height H b , where x b =x,y b =y,W b =w,H b =h; The optimization of the first bounding box parameter and the second bounding box parameter to obtain the target bounding box parameter comprises: If V1 and V2 are both true, the relationship between a distance d between the first miss distance center and the second miss distance center and a threshold distance D is judged, if d is less than D, the first bounding box parameter is taken as the target bounding box parameter, if d is greater than D, a difference S1 of one-dimensional gradient values of two frames of images input into the centroid tracking algorithm is calculated, a difference S2 of one-dimensional gradient values of two frames of images input into the basic tracking algorithm is calculated, S1 and S2 are compared, and the bounding box parameter corresponding to the smaller value is taken as the target bounding box parameter; If one of V1 and V2 is true and the other is false, the bounding box parameter corresponding to true is selected as the target bounding box parameter; If V1 and V2 are both false, target tracking is performed based on external guidance data issued by the base center.

9. The aircraft target tracking method according to claim 8, wherein, S1 of one-dimensional gradient values of two frames of images input into the centroid tracking algorithm is calculated according to Formula 4, and S2 of one-dimensional gradient values of two frames of images input into the basic tracking algorithm is calculated according to Formula 5; The formula 4 is as follows: The formula 5 is as follows: wherein, is a first order gradient value of the previous frame image, is a first order gradient value of the previous frame image.

10. A light measuring device characterized by comprising: Comprise: An optical system, a sensor, an encoder, an electronic processing unit, an image processing unit, a main control unit and a mechanical structure unit, the mechanical structure unit at least comprising a servo, a basic mechanical structure and a base, the optical measurement device being used to execute the aircraft target tracking method in any one of claims 1 to 9.

Images