Aircraft target tracking method and optical measurement equipment

Abstract

The invention relates to the technical field of target tracking, in particular to an aircraft target tracking method and optical measurement equipment, and the aircraft target tracking method comprises the steps: tracking an aircraft target according to the current target miss distance in the process that the aircraft target flies at the first stage; in the middle flight process of the aircraft target, fitting based on the current target miss distance, the pitch angle parameter of the optical measurement equipment and the azimuth angle parameter of the optical measurement equipment to obtain an extrapolation motion track, and tracking the aircraft target according to the extrapolation motion track; and in the process that the aircraft target flies at the end, processing the currently acquired image by adopting a centroid tracking algorithm to obtain a second bounding box parameter, optimizing the first bounding box parameter and the second bounding box parameter to obtain a target bounding box parameter, and tracking the aircraft target according to the target bounding box parameter. The method is at least beneficial to improving the precision of aircraft target tracking.

Description

Technical Field

[0001] The invention belongs to the technical field of target tracking, and in particular relates to an aircraft target tracking method and an optical measurement device. Background Art

[0002] The optical measurement equipment automatically tracks the target by extracting the target miss distance to observe the flight trajectory of the aircraft in the whole scene and obtain the real-time image and position information of the aircraft. However, the whole process of the aircraft flight includes the first flight process, the middle flight process and the final flight process, hereinafter referred to as the first, middle and final stages. The first, middle and final stages have different background characteristics and morphological characteristics. Among them, the first stage target penetrates the cloud, the middle stage target is separated and the final stage target is small, which will interfere with the automatic tracking of the target, reduce the accuracy of target tracking, and even cause the target to be lost. Summary of the invention

[0003] In view of this, the present invention aims to provide an aircraft target tracking method and an optical measurement device, which are at least helpful in improving the accuracy of aircraft target tracking.

[0004] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0005] The invention provides an aircraft target tracking method. The aircraft target tracking method tracks the aircraft target in the whole section based on an optical measurement device. The aircraft target tracking method comprises: acquiring an image with the aircraft target in real time, processing the currently acquired image by using a basic tracking algorithm to obtain a first bounding box parameter, and obtaining a target miss amount based on the first bounding box parameter; tracking the aircraft target according to the target miss amount when the aircraft target is in the first section of flight; obtaining an extrapolated motion trajectory based on the target miss amount, the pitch angle parameter of the optical measurement device and the azimuth angle parameter of the optical measurement device when the aircraft target is in the middle section of flight, and tracking the aircraft target according to the extrapolated motion trajectory; and processing the currently acquired image by using a centroid tracking algorithm to obtain a second bounding box parameter when the aircraft target is in the last section of flight, optimizing the first bounding box parameter and the second bounding box parameter to obtain a target bounding box parameter, and tracking the aircraft target according to the target bounding box parameter.

[0006] Furthermore, the basic tracking algorithm includes: a YOLO-GD algorithm, which is a YOLO-v7 target detection algorithm combined with a one-dimensional gradient operation; the YOLO-GD algorithm is used to process the currently acquired image to obtain a first bounding box parameter.

[0007] Furthermore, the YOLO-GD algorithm uses an offline training YOLO-GD model and calls the YOLO-GD model online to process the image; the YOLO-GD model includes a feature extraction backbone network, a feature fusion network and a head detection network; the feature extraction backbone network includes multiple convolutional layers conv, multiple normalization layers BN, multiple Mish activation functions, multiple L-relu activation functions and a fully connected layer Dense; the feature fusion network includes an upsampling module, multiple convolutional layers conv, multiple fully connected layers Dense, multiple normalization layers BN and multiple L-relu activation functions; the head detection network includes multiple global pooling layers, multiple convolutional layers conv, multiple fully connected layers Dense, multiple normalization layers BN and multiple L-relu activation functions; calling the YOLO-GD model to process the image includes: sending the one-dimensional gradient information of the image and the image to the feature extraction backbone network together.

[0008] Further, the first bounding box parameters include first bounding box coordinates (x, y, w, h), where (x, y) is the relative upper left corner coordinate of the first bounding box, w is the width of the first bounding box, and h is the height of the first bounding box; obtaining the target miss amount based on the first bounding box parameters includes: calculating the target miss amount according to Formula 1 based on the first bounding box coordinates (x, y, w, h), and Formula 1 is as follows:

[0009]

[0010]

[0011] Among them, ΔA and ΔE are the target off-target amounts.

[0012] Furthermore, fitting the extrapolated motion trajectory includes: obtaining a comprehensive angle value according to Formula 2 based on the target miss amount, the pitch angle parameter, and the azimuth angle parameter. Formula 2 is as follows:

[0013]

[0014] Among them, ΔA and ΔE are the target off-target amounts, A zong and E zong is the comprehensive angle value, A e is the azimuth parameter, E e is the pitch angle parameter, μ is the pixel size of the detector of the optical measurement device, and f is the current focal length of the optical measurement device. The extrapolated motion trajectory is obtained by fitting the comprehensive angle value using the Newton interpolation method.

[0015] Further, fitting the extrapolated motion trajectory using the Newton interpolation method according to the comprehensive angle value includes: fitting the extrapolated motion trajectory using the extrapolation formula of the Newton interpolation method, and the extrapolation formula is as follows:

[0016] A zong (x) = A zong (x 0 )+a 1 (xx 0 )+a 2 (xx 0 )(xx 1 )+...+a n (xx 0 )(xx 1 )...(xx n );

[0017] E zong (x) = E zong (x 0 )+b 1 (xx 0 )+b 2 (xx 0 )(xx 1 )+...+b n (xx 0 )(xx 1 )...(xx n );

[0018] Among them, a 0 ,a 1 ...a n and b 0 ,b 1 ...b n All are first-order difference quotients;

[0019]

[0020] i=1、2、3……n,x 0 ,x 1 ...x n A is the acquisition time in chronological order. zong (x) and E zong (x) is the extrapolated motion trajectory, A zong (x) is the predicted value in azimuth, E zong (x) is the predicted value at the pitch angle, A zong (x i ) is to obtain the time x i A zong , E zong (x i ) is to obtain the time x i E zong .

[0021] Furthermore, the second bounding box parameters include the second miss distance center coordinates (x c ,y c ), the second bounding box width W c , the second bounding box height H c and the second effective value V 2 ; Using the centroid tracking algorithm to process the currently acquired image includes: acquiring the pixel point (x i ,y j ) and the corresponding gray value I(x i ,y j ), and the second miss distance center coordinate (x c ,y c ), Formula 3 is as follows:

[0022]

[0023]

[0024] Where N is the width of the image, M is the height of the image, and I(x i ,y j ) is the pixel point (x i ,y j ) grayscale value; the second bounding box width W c is the number of continuous pixels whose grayscale values ​​are greater than the grayscale threshold in the horizontal direction, and the height of the second bounding box H c is the number of consecutive pixels whose grayscale values ​​are greater than the grayscale threshold in the vertical direction, and the second effective value V 2 Is true or false.

[0025] Furthermore, the first bounding box parameters include first bounding box coordinates (x, y, w, h) and a first valid value V 1 , the first effective value V 1 True or false; get the first miss distance center coordinate (x) according to the first bounding box coordinate (x, y, w, h) b ,y b ), the first bounding box width W b and the first bounding box height H b , where x b =x,y b =y,W b =w,H b = h; optimizing the first bounding box parameter and the second bounding box parameter to obtain the target bounding box parameter includes: if V 1 and V 2If both are true, determine the relationship between the distance d between the first miss distance center and the second miss distance center and the threshold distance D. If d is less than D, use the first bounding box parameter as the target bounding box parameter. If d is greater than D, calculate the difference S of the one-dimensional gradient values ​​of the two frames before and after the input centroid tracking algorithm. 1 , calculate the difference S of the one-dimensional gradient values ​​of the two frames before and after the input basic tracking algorithm 2 , compared with S 1 and S 2 The size of V is selected, and the bounding box parameters obtained by the algorithm corresponding to the smaller value are selected as the target bounding box parameters; if V 1 and V 2 If one of them is true and the other is false, the bounding box parameters corresponding to true are selected as the target bounding box parameters; if V 1 and V 2 If both are false, target tracking is performed based on external guidance data sent by the base center.

[0026] Further, the difference S of the one-dimensional gradient values ​​of the two frames before and after the input centroid tracking algorithm is calculated according to Formula 4: 1 , calculate the difference S of the one-dimensional gradient values ​​of the two frames before and after the input basic tracking algorithm according to formula 5 2 ;

[0027] Formula 4 is as follows:

[0028] Formula 5 is as follows:

[0029] in, is the first-order gradient value of the current frame image, is the first-order gradient value of the previous frame image.

[0030] On the other hand, the present invention provides an optical measurement device, including: an optical system, a sensor, an encoder, an electronic processing unit, an image processing unit, a main control unit and a mechanical structure unit, wherein the mechanical structure unit at least includes a servo, a basic mechanical structure and a base, and the optical measurement device is used to execute the above-mentioned aircraft target tracking method.

[0031] Compared with the prior art, the invention can achieve the following beneficial effects: the aircraft target tracking method provided by the embodiment of the invention is conducive to improving the adaptability of the tracking method under target state changes and background changes; in addition, combining the centroid tracking algorithm on the basis of the basic tracking algorithm is conducive to improving the tracking accuracy, and at the same time makes the wave gate more stable during the tracking of the aircraft target; for the short-term target information caused by the aircraft target penetrating the cloud and separating, it is possible to continue to track the target by extrapolating the motion trajectory; in the final flight process of the aircraft target, the automatic switching of the tracking mode can be realized, which is conducive to preventing the operator from operating incorrectly and reducing manpower loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings:

[0033] Figure 1 A schematic diagram of a full-segment tracking strategy of an aircraft target tracking method according to an embodiment of the present invention;

[0034] Figure 2 A framework diagram of the YOLO-GD model described in the embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the positional relationship between the first bounding box and the image according to an embodiment of the present invention;

[0036] Figure 4 A schematic diagram of the location of a target point described in an embodiment of the present invention.

[0037] Description of the accompanying drawings: 10, image; 11, first bounding box. DETAILED DESCRIPTION

[0038] After analysis, it is found that the current aircraft target tracking technology has the following problems: poor adaptability to the deformation and attitude changes of aircraft targets. Specifically, as the flight process of the aircraft target progresses, the attitude change of the aircraft target causes the aircraft target in the image to deform at the observation angle of the photoelectric theodolite, which requires the basic tracking algorithm to be able to handle the non-rigid deformation of the image; occlusion and overlap can easily lead to mistracking of aircraft targets. Specifically, the aircraft target may pass through the clouds during flight, that is, the aircraft target may be occluded and overlapped, which will interfere with the basic tracking algorithm and lead to mistracking of the aircraft target; the initialization of the basic tracking algorithm usually requires an accurate target position, which changes with time. As time goes by, the basic tracking algorithm needs to update the target model to adapt to the changes in the target's appearance. Initialization and model updating need to be performed multiple times in the whole process to adapt to the changes in the target's appearance. The operation of updating the target model by the basic tracking algorithm requires manual judgment, and the initialization operation and model updating require manual judgment in advance; the data set of the basic tracking algorithm has poor coverage in practical applications, that is, the data set may not be able to fully cover all scenes in practical applications, resulting in a gap between the performance of the basic tracking algorithm in practical applications and expectations; the aircraft target is an infrared target, which contains less feature information, and has no texture, color and other information, only grayscale information and irregular shape information, which is not conducive to the training of the basic tracking algorithm model parameters.

[0039] To solve the above problems, an embodiment of the present invention proposes a method for tracking an aircraft target. The method formulates different tracking methods for different stages in the whole flight process of the aircraft target. Specifically, the first stage adopts the deep learning algorithm YOLO-GD algorithm to detect the texture and shape of the aircraft target, effectively adapting to the identification of the aircraft target at different flight angles, and preventing the aircraft target from being caught incorrectly during the tracking process; the middle stage is based on the YOLO-GD algorithm, combined with the encoder value of the optical measurement device to fit the extrapolated motion trajectory of the aircraft target, wherein the encoder value is the pitch angle parameter and the azimuth angle parameter. After the extrapolated motion trajectory is fitted, the accuracy of the extrapolated motion trajectory is verified according to the ballistic information sent in advance by the base center, and the aircraft target is tracked according to the verified extrapolated motion trajectory, so as to prevent the target from being caught incorrectly during the middle tracking process; in the final flight process of the aircraft target, the tracking results obtained by the centroid tracking algorithm and the tracking results obtained by the YOLO-GD algorithm are judged, so as to realize the automatic matching of the optimal tracking strategy according to the quality of each information source in real time, and to automatically switch in real time. The embodiment of the present invention ensures stable tracking of the aircraft target throughout the entire process after takeoff, without single-rod correction of the servo. Single-rod correction refers to the process of manual adjustment, which truly achieves stable automatic tracking throughout the entire process after takeoff and emergency handling of common situations, and realizes more accurate, more robust and more real-time tracking effects.

[0040] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0041] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0042] In the description of the invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the invention, unless otherwise specified, the meaning of "multiple" is two or more.

[0043] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0044] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0045] The invention provides an aircraft target tracking method on one hand, which tracks the aircraft target in the whole section based on an optical measuring device, wherein the optical measuring device at least includes: an optical system, a sensor, an encoder, an electronic processing unit, an image processing unit, a main control unit and a mechanical structure unit. The mechanical structure unit at least includes a servo, a basic mechanical structure and a base.

[0046] In some embodiments of the present invention, the optical measuring device is a photoelectric theodolite, which is a high-precision optical measuring device that combines optical and electronic technologies and is used to measure the azimuth and elevation angles of an object. It is widely used in engineering surveying, geographic information systems, astronomical observations, military reconnaissance and other fields.

[0047] When the method for tracking an aircraft target provided by the present invention is executed using an optical measurement device, the aircraft target is imaged on the target surface of the sensor through an optical system, and then the optical signal is converted into an electrical signal through an electronic processing unit. The acquisition card acquires the image, and then the image is processed by the algorithm in the image processing unit to determine the spatial position of the target. The spatial position is transmitted to the servo in the mechanical structure unit through the serial port. The servo controls the basic mechanical structure and then controls the optical measurement device to track the target and obtain the next frame of image. This process is repeated to form a closed-loop automatic tracking.

[0048] It should be noted that the algorithms in the image processing unit at least include a basic tracking algorithm, a centroid tracking algorithm and other auxiliary algorithms, and the algorithms in the image processing unit are used to implement the aircraft target tracking method provided by the present invention.

[0049] refer to Figure 1 The aircraft target tracking method includes: acquiring an image with an aircraft target in real time, processing the currently acquired image by using a basic tracking algorithm to obtain a first bounding box parameter, and obtaining a target miss amount based on the first bounding box parameter; tracking the aircraft target according to the target miss amount when the aircraft target is in the first stage of flight; obtaining an extrapolated motion trajectory based on the target miss amount, a pitch angle parameter of an optical measurement device, and an azimuth angle parameter of the optical measurement device when the aircraft target is in the middle stage of flight, and tracking the aircraft target according to the extrapolated motion trajectory; and processing the currently acquired image by using a centroid tracking algorithm to obtain a second bounding box parameter when the aircraft target is in the last stage of flight, optimizing the first bounding box parameter and the second bounding box parameter to obtain a target bounding box parameter, and tracking the aircraft target according to the target bounding box parameter.

[0050] The aircraft target tracking method proposed in the embodiment of the present invention formulates different tracking methods for different stages in the whole flight process of the aircraft target. The basic tracking algorithm is used to track the aircraft target in the first stage; the aircraft target is tracked according to the extrapolated motion trajectory after fitting based on the basic tracking algorithm, combined with the pitch angle parameters and the azimuth angle parameters, to prevent the wrong target from being captured during the tracking process in the middle stage; in the final flight process of the aircraft target, the tracking results obtained by the centroid tracking algorithm and the tracking results obtained by the basic tracking algorithm are judged to realize automatic matching of the optimal tracking strategy according to the quality of each information source in real time, and automatic switching in real time.

[0051] It should be noted that the first bounding box parameters represent the parameters of the first bounding box used to select the aircraft target. The first bounding box can be determined by determining the first bounding box parameters, and the target miss distance is the target position data calculated based on the first bounding box parameters and sent to the servo. The second bounding box parameters represent the parameters of the second bounding box used to select the aircraft target. The second bounding box can be determined by determining the second bounding box parameters.

[0052] Furthermore, the basic tracking algorithm includes: a YOLO-GD (You Only Look Once version gradient) algorithm, the YOLO-GD algorithm is a YOLO-v7 (You Only Look Once version 7) target detection algorithm combined with a one-dimensional gradient operation; the YOLO-GD algorithm is used to process the currently acquired image to obtain a first bounding box parameter.

[0053] Furthermore, the YOLO-GD algorithm processes the image by training the YOLO-GD model offline and calling the YOLO-GD model online; the YOLO-GD model includes a feature extraction backbone network, a feature fusion network and a head detection network; the feature extraction backbone network includes multiple convolutional layers conv, multiple normalization layers BN, multiple Mish activation functions, multiple L-relu (leaky Rectified Linear Unit) activation functions and a fully connected layer Dense, which is conducive to improving the depth of target feature extraction by the feature extraction backbone network; the feature fusion network includes an upsampling module (Upsampling), multiple convolutional layers conv, multiple fully connected layers Dense, multiple normalization layers BN and multiple L-relu activation functions; the head detection network includes multiple global pooling layers (Global pool Layer), multiple convolutional layers conv, multiple fully connected layers Dense, multiple normalization layers BN and multiple L-relu activation functions; calling the YOLO-GD model to process the image includes: sending the one-dimensional gradient information of the image and the image to the feature extraction backbone network together.

[0054] As a basic tracking algorithm, the YOLO-GD algorithm is the core of the aircraft target tracking method. The reason for using the YOLO-GD algorithm as the basic tracking algorithm is that when the first stage aircraft target comes out of the mountain, the image collected by the optical measurement device is an infrared image. In the infrared image, the mountain presents high grayscale characteristics due to sunlight during the day. The traditional tracking method may miscapture the target. Since the first stage optical measurement device is close to the aircraft target, the pixel size of the aircraft target under visual conditions meets the requirement of being greater than 32×32 pixels. The aircraft target has more characteristic information such as posture and shape. Therefore, in order to improve the anti-interference ability of the algorithm, the embodiment of the present invention combines the YOLO-v7 target detection algorithm and gradient operation to propose an improved YOLO-GD algorithm for quickly and accurately identifying aircraft targets in images. The YOLO-GD algorithm includes a one-dimensional gradient operation, and the information input into the YOLO-GD model adds one-dimensional gradient information, which is conducive to better training the YOLO-GD model.

[0055] It can be understood that in the process of offline training of the YOLO-GD model: a one-dimensional gradient operation is performed on the data set input to the YOLO-GD algorithm, and the one-dimensional gradient information obtained by the one-dimensional gradient operation is sent to the YOLO-GD algorithm for subsequent distinction between background and target, which is conducive to ensuring that the data set of the YOLO-GD algorithm better covers all scenes in practical applications. The one-dimensional gradient operation specifically includes: calculating the image f(x 像 ,y 像 ), where x 像 ,y 像 Represents the pixel point of the image, image f(x 像 ,y 像 ) is a vector that contains the maximum rate of change of the image in all directions. For a two-dimensional image f(x 像 ,y 像 ), image f(x 像 ,y 像 The first-order derivative of ) consists of the following two parts: the rate of change in the horizontal direction (i.e., the partial derivative with respect to x) and the rate of change in the vertical direction (i.e., the partial derivative with respect to y). 像 ,y 像 ) can be used to estimate the brightness change. The point where the brightness changes fastest is the edge pixel of the target. 像 ,y 像 ) is the first derivative of It can be calculated by the following formula:

[0056]

[0057] in, is the image f(x 像 ,y像 ) with respect to x, is the image f(x 像 ,y 像 ) with respect to the partial derivative of y.

[0058] In some embodiments of the present invention, reference Figure 2 The YOLO-GD model includes a feature extraction backbone network, a feature fusion network and a head detection network. The feature extraction backbone network includes 19 convolutional layers conv, 8 normalization layers BN, 3 Mish activation functions, 3 L-relu activation functions and a fully connected layer Dense, which is conducive to improving the depth of target feature extraction by the feature extraction backbone network; the feature fusion network includes an upsampling module, 6 convolutional layers conv, 2 fully connected layers Dense, 2 normalization layers BN and 2 L-relu activation functions, and divides the image into three scale outputs of 7×7 pixels, 14×14 pixels and 28×28 pixels, which is conducive to strengthening the training of different feature dimensions and fully learning the shape and size of the aircraft target at different times; the head detection network includes multiple global pooling layers, multiple convolutional layers conv, multiple fully connected layers Dense, multiple normalization layers BN and multiple L-relu activation functions. The head detection network is used to predict the target type and target position, output multiple candidate bounding box parameters, screen the candidate bounding box parameters, and then obtain the first bounding box parameters.

[0059] It should be noted that Figure 2 The lower left box shows Figure 2 The specific units included in the CBM module, CCC module, CBL module and DCC module of the YOLO-GD model. The “×3” marked above the modules in the backbone network and head detection network represents the number of modules multiplied by 3.

[0060] In addition, the candidate bounding box parameters generally include: the coordinates of the candidate bounding box, the category probability, and the target existence confidence. The category probability is a vector indicating the probability that the object contained in the candidate bounding box belongs to different categories.

[0061] The specific basic steps of screening multiple candidate bounding box parameters to obtain the first bounding box parameters can be as follows: the YOLO-GD algorithm can adjust the input image to 416×416 pixels, and then divide the entire image into 13×13 grids on average. The image slice corresponding to each grid can be responsible for predicting 3 candidate bounding boxes. For each candidate bounding box, predict the candidate bounding box parameters, including the category probability, target existence confidence, and candidate bounding box coordinates in the candidate bounding box. Compare multiple candidate bounding box parameters, screen out the candidate bounding box parameters corresponding to the maximum target existence confidence, divide the candidate bounding box into target or background according to the category probability in the candidate bounding box parameters, and use the candidate bounding box corresponding to the target as the first bounding box. The candidate bounding box corresponding to the target is used as the first bounding box, and the candidate bounding box parameters corresponding to the target are used as the first bounding box parameters.

[0062] Further, the first bounding box parameters include first bounding box coordinates (x, y, w, h), where reference Figure 3 , (x, y) is the position coordinate of the first bounding box 11 relative to the upper left corner of the image 10, w is the width of the first bounding box 11, and h is the height of the first bounding box 11; obtaining the target miss amount based on the first bounding box parameters includes: calculating the target miss amount according to Formula 1 based on the first bounding box coordinates (x, y, w, h), and Formula 1 is as follows:

[0063]

[0064]

[0065] Among them, ΔA and ΔE are the target miss distances, ΔA is the azimuth offset of the aircraft target center relative to the center of the field of view, and ΔE is the pitch offset of the aircraft target center relative to the center of the field of view.

[0066] Specifically, images can be input to the YOLO-GD algorithm at a frequency of 50 Hz, and bounding box prediction can be performed on each frame of the image to obtain the first bounding box coordinates (x, y, w, h), and the target miss distances ΔA and ΔE can be obtained. The processing time for each frame of the image is usually no more than 15 ms.

[0067] For the mid-flight process of the aircraft target, the pitch angle parameters and azimuth angle parameters are used to assist the basic tracking algorithm to track the target, so as to prevent the fairing and other structures of the aircraft target from separating and causing the basic tracking algorithm to catch the wrong target. It should be noted that when the aircraft target enters the cloud and separates from the aircraft target, the target miss amount of the aircraft target will be lost. In this case, the predicted value obtained by extrapolating the motion trajectory is used as the temporary miss amount for target tracking.

[0068] Further, fitting the extrapolated motion trajectory includes: obtaining a comprehensive angle value according to Formula 2 based on the target miss amount, the pitch angle parameter, and the azimuth angle parameter, wherein the target miss amount is historical data, and the comprehensive angle value is calculated based on the historical target miss amount. Formula 2 is as follows:

[0069]

[0070] Among them, ΔA and ΔE are the target off-target amounts, A zong and E zong is the comprehensive angle value, A zong It represents the comprehensive azimuth angle generated by the target point moving in the horizontal direction, E zong It represents the comprehensive pitch angle caused by the vertical movement of the target point, A e is the azimuth parameter, E e is the pitch angle parameter, μ is the pixel size of the detector of the optical measurement device, and f is the current focal length of the optical measurement device. The extrapolated motion trajectory is obtained by fitting the comprehensive angle value using the Newton interpolation method.

[0071] Specifically, refer to Figure 4 ,like Figure 4 As shown in (a), point O is the optical center of the optical measuring device; point M is the center of the aircraft target, referred to as the target point, the distance between the target point M and the optical center O is R, the projection of the target point M on the XOZ plane is M', ∠MOM' is the pitch angle E generated by the vertical movement of the target point M, and the angle between the line connecting the projection point M' and the optical center O and the x-axis is the azimuth angle A generated by the horizontal movement of the target point. Figure 4 As shown in (b) in the figure, in the image processing unit, the target miss distance is the azimuth offset and pitch angle offset of the aircraft target center relative to the center of the field of view. When the aircraft target is offset, the optical measurement device tracks the aircraft target and records the azimuth offset ΔA and pitch angle offset ΔE of the target point relative to the center of the camera (sensor) crosshairs, which is the target miss distance. The azimuth parameter A e Indicates the azimuth angle of the encoder. The encoder is the encoder of the optical measurement device. The pitch angle parameter E e Indicates the encoder pitch angle.

[0072] Further, the extrapolated motion trajectory is obtained by fitting the comprehensive angle value using the Newton interpolation method, including: zong and E zong Corresponding to the acquisition time, the extrapolated motion trajectory is fitted using the extrapolation formula of the Newton interpolation method. The extrapolation formula is as follows:

[0073] A zong (x) = A zong (x 0 )+a 1(xx 0 )+a 2 (xx 0 )(xx 1 )+...+a n (xx 0 )(xx 1 )...(xx n );

[0074] E zong (x) = E zong (x 0 )+b 1 (xx 0 )+b 2 (xx 0 )(xx 1 )+...+b n (xx 0 )(xx 1 )...(xx n );

[0075] Among them, a 0 ,a 1 ...a n and b 0 ,b 1 ...b n are all first-order difference quotients;

[0076]

[0077] i=1、2、3……n-1,x 0 ,x 1 ...x n A is the acquisition time in chronological order. zong (x) and E zong (x) is the extrapolated motion trajectory, A zong (x) is the predicted value in azimuth, E zong (x) is the predicted value at the pitch angle, A zong (x i ) is to obtain the time x i A zong , E zong (x i ) is to obtain the time x i E zong .

[0078] It should be noted that after obtaining the extrapolated motion trajectory, it is necessary to use the ballistic information sent in advance by the base center to verify the accuracy of the extrapolated motion trajectory to ensure that the extrapolated motion trajectory can more accurately simulate the actual motion trajectory. Among them, the ballistic information is usually the motion trajectory data of the aircraft target measured by the base center through radar or other means.

[0079] When the aircraft target is in mid-flight and the target is lost, an embodiment of the present invention uses an extrapolated motion trajectory to predict the spatial position of the aircraft target in the next frame, transmits the extrapolated motion trajectory to the servo, and tracks the aircraft target. When the target is not lost, the target miss amount is used to track the aircraft target.

[0080] When the aircraft target is in the final stage of flight, the aircraft target's own posture characteristics are not obvious, and the aircraft target as a whole appears in the form of a dot with a tail flame. Therefore, a tracking strategy that automatically switches between the centroid tracking algorithm and the YOLO-GD algorithm is used to track the aircraft target.

[0081] Furthermore, the second bounding box parameters obtained by the centroid tracking algorithm include the second miss distance center coordinates (x c ,y c ), the second bounding box width W c , the second bounding box height H c and the second effective value V 2 ; Using the centroid tracking algorithm to process the currently acquired image includes: obtaining the pixel coordinates of the image (x i ,y j ) and the corresponding gray value I(x i ,y j ), considering that the gray value of a pixel is usually in the range of 0 to 255, it is necessary to weight the gray value of each pixel, that is, the second miss distance center coordinate (x c ,y c ), Formula 3 is as follows:

[0082]

[0083]

[0084] Where N is the width of the image, M is the height of the image, and I(x i ,y j ) is the pixel point (x i ,y j ) grayscale value; the second bounding box width W c is the number of continuous pixels whose grayscale values ​​are greater than the grayscale threshold in the horizontal direction, and the height of the second bounding box H cis the number of consecutive pixels whose grayscale values ​​are greater than the grayscale threshold in the vertical direction, and the second effective value V 2 True or false, where grayscale threshold = (maximum grayscale value of the image - minimum grayscale value of the image) / 2 + image grayscale mean.

[0085] Furthermore, the first bounding box parameters include first bounding box coordinates (x, y, w, h) and a first valid value V 1 , the first effective value V 1 True or false; get the first miss distance center coordinate (x) according to the first bounding box coordinate (x, y, w, h) b ,y b ), the first bounding box width W b and the first bounding box height H b , where x b =x,y b =y,W b =w,H b =h. It should be noted that the optical measurement device transmits the image to the image processing unit in real time through the sensor and the electronic processing unit. The image transmitted to the image processing unit is processed in two ways, namely the YOLO-GD algorithm and the centroid tracking algorithm. Determine the second effective value V 2 The method is: determine whether the number of pixels in the second boundary box that are greater than the grayscale threshold is greater than 10. If so, it is determined to be valid, V 2 is true, if not, then V 2 is false, and 10 is the minimum target size of the aircraft target within the field of view.

[0086] Determine the first effective value V 1 The method is: determine whether the number of pixels in the first boundary box that are greater than the grayscale threshold is greater than 10. If so, it is determined to be valid, V 1 is true, if not, then V 1 is false.

[0087] Further, the first bounding box parameters and the second bounding box parameters are optimized to obtain target bounding box parameters, specifically including:

[0088] According to V 1 and V 2 To make a judgment, specifically, if V 1 and V 2If both are true, the relationship between the distance d between the first miss distance center and the second miss distance center and the threshold distance D is determined, where D = N / 2, N is the width of the image. If d is less than D, it means that the positions extracted by the two are close. The first bounding box parameter is used as the target bounding box parameter. If d is greater than D, it means that one of them has mistakenly extracted the target. Therefore, the difference S of the one-dimensional gradient values ​​of the two frames before and after the input centroid tracking algorithm is calculated. 1 , calculate the difference S of the one-dimensional gradient values ​​of the two frames before and after the input basic tracking algorithm 2 , compared with S 1 and S 2 The smaller the value, the smaller the target shape change. The bounding box parameters obtained by the algorithm corresponding to the smaller value are selected as the target bounding box parameters. The target bounding box parameters are subsequently sent to the servo system to guide the movement of the optical measurement device, thereby tracking the aircraft target.

[0089] If V 1 and V 2 If one of them is true and the other is false, the bounding box parameters corresponding to true are selected as the target bounding box parameters;

[0090] If V 1 and V 2 If both are false, target tracking is performed based on external guidance data sent by the base center.

[0091] It should be noted that after obtaining the target bounding box parameters, it is necessary to determine whether the current target bounding box parameters and historical data can fit into a complete curve. If there is a jump point, the target bounding box parameters will not be selected as the basis for aircraft target tracking, and target tracking needs to be based on external guidance data sent by the base center.

[0092] Further, the difference S of the one-dimensional gradient values ​​of the two frames before and after the input centroid tracking algorithm is calculated according to Formula 4: 1 , calculate the difference S of the one-dimensional gradient values ​​of the two frames before and after the input basic tracking algorithm according to formula 5 2 ;

[0093] Formula 4 is as follows:

[0094] Formula 5 is as follows:

[0095] in, is the first-order gradient value of the current frame image, is the first-order gradient value of the previous frame image, N is the width of the image, and M is the height of the image.

[0096] In some embodiments of the present invention, the base center sends a timestamp to the image processing unit when the aircraft enters the first section, the middle section, and the last section, and the image processing unit switches the tracking strategy according to the corresponding moment of the timestamp.

[0097] Another aspect of the present invention provides an optical measurement device, which is used to execute the above-mentioned aircraft target tracking method.

[0098] In some embodiments of the present invention, the optical measurement device comprises at least: an optical system, a sensor, an encoder, an electronic processing unit, an image processing unit, a main control unit and a mechanical structure unit. The mechanical structure unit comprises at least a servo, a basic mechanical structure and a base.

[0099] When the method for tracking an aircraft target provided by the present invention is executed using an optical measurement device, the aircraft target is imaged on the target surface of the sensor through an optical system, and then the optical signal is converted into an electrical signal through an electronic processing unit. The acquisition card acquires the image, and then the image is processed by the algorithm in the image processing unit to determine the spatial position of the target. The spatial position is transmitted to the servo in the mechanical structure unit through the serial port. The servo controls the basic mechanical structure and then controls the optical measurement device to track the target and obtain the next frame of image. This process is repeated to form a closed-loop automatic tracking.

[0100] The embodiment of the present invention proposes a full-stage aircraft target tracking method, which formulates an optimal tracking strategy by combining different backgrounds and target characteristics of the first stage, middle stage, and last stage.

[0101] Specifically, the method uses a photoelectric theodolite configured with an infrared detector (sensor) and formulates an optimal tracking algorithm strategy based on different background and target characteristics in the first, middle and last sections. The target shape in the first section is obvious and the background is cloudy. Therefore, the YOLO-GD algorithm, a deep learning algorithm improved on the basis of YOLO-v7, is used to detect the texture and shape of the target in the first section, which effectively adapts to the recognition of aircraft targets at different angles and prevents the wrong aircraft targets from being captured during tracking. The aircraft target is separated in the middle section, and there are interfering targets such as boosters and fairings. Therefore, on the basis of the YOLO-GD algorithm, the azimuth and pitch values ​​of the encoder are obtained in the middle section. Based on the azimuth value, pitch value and the target miss amount obtained by the YOLO-GD algorithm, the extrapolated motion trajectory of the aircraft target is fitted, and the aircraft target is screened according to the overlap between the current frame of the aircraft target in the trajectory information and the extrapolated motion trajectory to prevent miscapture. The target in the tail section is far away. Therefore, on the basis of the YOLO-GD algorithm, the pixel-level centroid tracking algorithm is combined to judge the two and track the aircraft target according to the judgment result. At the same time, the entire process can be combined with the external guidance of the equipment to perform multi-dimensional position judgment. That is to say, during the entire process, the optical measurement equipment is allowed to receive the external guidance data sent by the base center and control the optical measurement equipment to achieve tracking and correction of the aircraft target. It should be noted that the optical measurement equipment receives the external guidance data sent by the base center through the network.

[0102] In summary, the embodiments of the present invention ensure that during the entire process of the aircraft target after takeoff, the optical measurement equipment can automatically match the basic tracking algorithm according to the timestamp in real time and automatically switch in real time, truly achieving stable automatic tracking throughout the entire process after takeoff and emergency handling of common situations, and realizing more accurate, more robust, and more real-time tracking effects.

[0103] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0104] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for tracking an aircraft target, characterized in that: The aircraft target tracking method performs full-segment tracking of an aircraft target based on an optical measurement device, and the aircraft target tracking method comprises: Acquire an image with the aircraft target in real time, process the currently acquired image using a basic tracking algorithm to obtain first bounding box parameters, and obtain a target miss distance based on the first bounding box parameters; When the aircraft target is in the first stage of flight, tracking the aircraft target according to the target miss distance; When the aircraft target is in mid-flight, an extrapolated motion trajectory is obtained by fitting based on the target miss amount, the pitch angle parameter of the optical measurement device, and the azimuth angle parameter of the optical measurement device, and the aircraft target is tracked according to the extrapolated motion trajectory; When the aircraft target is in the final stage of flight, a centroid tracking algorithm is used to process the currently acquired image to obtain second bounding box parameters, the first bounding box parameters and the second bounding box parameters are optimized to obtain target bounding box parameters, and the aircraft target is tracked according to the target bounding box parameters.

2. The method for tracking an aircraft target according to claim 1, characterized in that: The basic tracking algorithm includes: a YOLO-GD algorithm, wherein the YOLO-GD algorithm is a YOLO-v7 target detection algorithm combined with a one-dimensional gradient operation; The YOLO-GD algorithm is used to process the currently acquired image to obtain the first bounding box parameters.

3. The aircraft target tracking method according to claim 2, characterized in that: The YOLO-GD algorithm processes images by training the YOLO-GD model offline and calling the YOLO-GD model online; The YOLO-GD model includes a feature extraction backbone network, a feature fusion network and a head detection network; The feature extraction backbone network includes multiple convolutional layers conv, multiple normalization layers BN, multiple Mish activation functions, multiple L-relu activation functions and a fully connected layer Dense; The feature fusion network includes an upsampling module, multiple convolutional layers conv, multiple fully connected layers Dense, multiple normalization layers BN and multiple L-relu activation functions; The head detection network includes multiple global pooling layers, multiple convolutional layers conv, multiple fully connected layers Dense, multiple normalization layers BN and multiple L-relu activation functions; Calling the YOLO-GD model to process the image includes: sending the one-dimensional gradient information of the image and the image to the feature extraction backbone network.

4. The method for tracking an aircraft target according to claim 1, 2 or 3, characterized in that: The first bounding box parameters include first bounding box coordinates (x, y, w, h), where (x, y) are the position coordinates of the first bounding box relative to the upper left corner of the image, w is the width of the first bounding box, and h is the height of the first bounding box; Obtaining the target miss amount based on the first bounding box parameters includes: calculating the target miss amount according to Formula 1 based on the first bounding box coordinates (x, y, w, h), and the Formula 1 is as follows: Wherein, ΔA and ΔE are the target off-target amounts.

5. The method for tracking an aircraft target according to claim 4, characterized in that: Fitting the extrapolated motion trajectory includes: obtaining a comprehensive angle value according to Formula 2 based on the target miss amount, the pitch angle parameter, and the azimuth angle parameter, and the Formula 2 is as follows: Among them, ΔA and ΔE are the target off-target amounts, A zong and E zong is the comprehensive angle value, A e is the azimuth parameter, E e is the pitch angle parameter, μ is the pixel size of the detector of the optical measurement device, and f is the current focal length of the optical measurement device; The extrapolated motion trajectory is obtained by fitting the comprehensive angle value using the Newton interpolation method.

6. The method for tracking an aircraft target according to claim 5, characterized in that: The fitting of the extrapolated motion trajectory using the Newton interpolation method according to the comprehensive angle value comprises: The extrapolated motion trajectory is obtained by fitting the extrapolation formula of the Newton interpolation method, and the extrapolation formula is as follows: A zong (x)=A zong (x0)+a1(x-x0)+a2(x-x0)(x-x1)+...+a n (x-x0)(x-x1)...(xx n ); Yes zong (x)=E zong (x0)+b1(x-x0)+b2(x-x0)(x-x1)+...+b n (x-x0)(x-x1)...(xx n ); Among them, a0, a1...a n and b0,b1...b n All are first-order difference quotients; i=1, 2, 3...n, x0, x1...x n A is the acquisition time in chronological order. zong (x) and E zong (x) is the extrapolated motion trajectory, A zong (x) is the predicted value in azimuth, E zong (x) is the predicted value at the pitch angle, A zong (x i ) is to obtain the time x i A zong , E zong (x i ) is to obtain the time x i E zong .

7. The aircraft target tracking method according to claim 1, characterized in that: The second bounding box parameters include the second miss distance center coordinates (x c ,y c ), the second bounding box width W c , the second bounding box height H c and the second effective value V2; The process of processing the currently acquired image by using the centroid tracking algorithm includes: acquiring pixel points (x i ,y j ) and the corresponding gray value I(x i ,y j ), the second miss distance center coordinate (x c ,y c ), the formula 3 is as follows: Where N is the width of the image, M is the height of the image, I(x i ,y j ) is the pixel point (x i ,y j )’s grayscale value; The second bounding box width W c is the number of continuous pixels whose grayscale values ​​are greater than the grayscale threshold in the horizontal direction, and the height of the second bounding box H c It is the number of continuous pixel points whose grayscale values ​​are greater than the grayscale threshold in the vertical direction. The second valid value V2 is true or false.

8. The method for tracking an aircraft target according to claim 7, characterized in that: The first bounding box parameter includes a first bounding box coordinate (x, y, w, h) and a first valid value V1, where the first valid value V1 is true or false; The first miss distance center coordinate (x) is obtained according to the first bounding box coordinate (x, y, w, h) b ,y b ), the first bounding box width W b and the first bounding box height H b , where x b =x,y b =y,W b =w,H b =h; Optimizing the first bounding box parameters and the second bounding box parameters to obtain the target bounding box parameters includes: If V1 and V2 are both true, determine the relationship between the distance d between the first miss distance center and the second miss distance center and the threshold distance D. If d is less than D, use the first bounding box parameter as the target bounding box parameter. If d is greater than D, calculate the difference S1 of the one-dimensional gradient values ​​of the two frames before and after the centroid tracking algorithm, calculate the difference S2 of the one-dimensional gradient values ​​of the two frames before and after the basic tracking algorithm, compare S1 and S2, and select the bounding box parameter obtained by the algorithm corresponding to the smaller value as the target bounding box parameter. If one of V1 and V2 is true and the other is false, select the bounding box parameters corresponding to true as the target bounding box parameters; If both V1 and V2 are false, target tracking is performed based on external guidance data sent by the base center.

9. The method for tracking an aircraft target according to claim 8, characterized in that: The difference S1 of the one-dimensional gradient values ​​of the two frames before and after the input to the centroid tracking algorithm is calculated according to Formula 4, and the difference S2 of the one-dimensional gradient values ​​of the two frames before and after the input to the basic tracking algorithm is calculated according to Formula 5; The formula 4 is as follows: The formula 5 is as follows: in, is the first-order gradient value of the current frame image, is the first-order gradient value of the previous frame image.

10. An optical measurement device, characterized in that: include: An optical system, a sensor, an encoder, an electronic processing unit, an image processing unit, a main control unit and a mechanical structure unit, wherein the mechanical structure unit at least includes a servo, a basic mechanical structure and a base, and the optical measurement equipment is used to execute the aircraft target tracking method described in any one of claims 1 to 9.

Images