Aero-engine blade tracking and counting method

Abstract

The invention relates to the technical field of aero-engine detection, in particular to an aero-engine blade tracking and counting method, which comprises the steps of aero-engine blade tracking and aero-engine blade counting: step 1, adopting a blade tracking algorithm based on a TBD framework to realize aero-engine blade tracking, comprising: step 1, target detection; 2, motion modeling and state estimation; 3, association matching; step 2, counting the aero-engine blades based on the trajectory information calculated in the step 1: step 1, identifying newly appearing aero-engine blades; 2, judging the rotating direction of the newly-appearing aero-engine blade and recording the starting direction of the newly-appearing aero-engine blade; and 3, updating the current count value according to the movement direction of the aero-engine blade. According to the invention, the problem of missing detection in the manual blade counting process can be solved, the problem of repeated detection of aero-engine blades by workers due to missing detection is solved, and the working efficiency of the workers is improved.

Description

Technical Field

[0001] The invention relates to the technical field of aero-engine detection, and in particular to a method for tracking and counting blades of an aero-engine. Background Art

[0002] In the traditional manual blade inspection process, the number of blades is often counted manually, with each blade passing the inspection being counted manually, and the number of blades inspected being counted in the end. Since the blades in the same compartment have basically the same shape and appearance, the manual counting of blades may result in omissions or over-counting due to visual fatigue or lack of concentration, resulting in repeated work for workers. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a method for tracking and counting blades of an aircraft engine. Compared with the manual counting method, the method adopts the target tracking method in computer vision to track the blades to obtain the motion trajectory of the blades and calculate the number of blades, getting rid of the dependence on the human eye. At the same time, it uses powerful deep learning technology to obtain an accuracy comparable to that of the human eye.

[0004] The technical problem to be solved by the present invention is achieved by adopting the following technical solutions:

[0005] A method for tracking and counting blades of an aircraft engine includes tracking blades of an aircraft engine and counting blades of an aircraft engine. The specific steps are as follows:

[0006] Step 1: Implementing aeroengine blade tracking using a blade tracking algorithm based on a TBD framework, including:

[0007] Step 1: Target detection: Send the video frame to be tested into the target detection network to locate the edge of the aircraft engine blade;

[0008] Step 2: Motion modeling and state estimation: by analyzing the motion state of the currently tracked object in real time, predict the position of the object in the next frame;

[0009] Step 3: Association matching, matching the target detection of the new frame with the predicted value to achieve tracking;

[0010] Step 2: Counting the blades of the aircraft engine based on the trajectory information calculated in step 1, including:

[0011] Step 1: Identify the newly appeared aircraft engine blades and obtain the latest successfully matched track ID number of the aircraft engine blade tracking algorithm for blade edge tracking. If it is different from the ID number recorded last time, the blade is regarded as a newly appeared blade.

[0012] Step 2: Determine the rotation direction of the newly appeared aircraft engine blade and record the starting direction, obtain the starting point and ending point of the latest successfully matched trajectory of the aircraft engine blade tracking algorithm for blade edge tracking, subtract the horizontal coordinates of the ending point from the starting point and use this value as the direction value. If the value is greater than 0, the rotation direction is from left to right, and if the value is less than 0, the rotation direction is from right to left. At the same time, the calculated direction value of the trajectory ID number 1 is regarded as the starting direction;

[0013] Step 3: Update the current count value according to the movement direction of the aircraft engine blades. If the direction is opposite to the starting direction, the count value is reduced by 1; if the direction is the same as the starting direction, the count value is increased by 1.

[0014] As a further improvement of the present invention, the target detection in step 1 is specifically as follows:

[0015] The target detection model for the edge of aviation engine blades obtained by training the YOLOv8n network is used as the target detector of the blade tracking algorithm based on the TBD framework, and the BiFormer attention module is embedded in the head structure of the YOLOv8n network.

[0016] As a further improvement of the present invention, for the embedded Biformer attention module, its main implementation process is as follows:

[0017] Step 1: Divide the input two-dimensional image into S×S blocks, and easily obtain the feature vector X of each area T The number is X T Through the formula (1), we can perform linear transformation to get the QK V vectors, which are as follows:

[0018]

[0019] Step 2: Establish connections through directed graphs to find other blocks that need to work together with the current block, as follows:

[0020] Calculate the average value of each area according to Q and K r , K r , multiply the two by formula (2) to obtain the correlation matrix A r , according to the correlation matrix, the first k regions with the highest correlation are reserved for subsequent use. The process is shown in formula (3). r For S 2 *k-sized matrix, whose i-th row stores the k index values ​​with the highest correlation with the i-th region. Formula (2) and formula (3) are as follows:

[0021] A r =Q r (K r )T (2)

[0022] I r =Topindex(A r ) (3);

[0023] Step 3: Aggregate K and V vectors according to the index value. The expression is shown in formula (4). Finally, perform traditional self-attention operation between pixel blocks and introduce a local enhancement algorithm based on convolution to improve the feature extraction capability, that is, formula (5). Formula (4) and formula (5) are as follows:

[0024] K g =gather(K,I r ),V g =gather(V,I r ) (4)

[0025] O=Attention(Q,K g ,V g )+LCE(V) (5).

[0026] As a further improvement of the present invention, the motion modeling and state estimation in step 1 are specifically as follows:

[0027] The Kalman filter is used to model the movement of the blade edge, and the subsequent movement is predicted and updated. The prediction process is to send the target position state vector into the state equation for calculation and estimate the target position in the next frame. The update process is to obtain the current observation value based on the measurement equation to update the target position.

[0028] As a further improvement of the present invention, the position state vector of the target is optimized and improved, and formula (6) is as follows:

[0029] X=[x,y,w,h,x',y',w',h',x”,y”] T (6)

[0030] Where x, y, w, and h are the horizontal coordinate, vertical coordinate, width, and height of the center point of the target frame, respectively; x', y', w', and h' are the first-order derivatives of the above variables reflecting the corresponding speed; and x' and y' are the second-order derivatives corresponding to the acceleration of the target in the horizontal and vertical directions;

[0031] According to the basic displacement formula of physics s = s0 + vt + 1 / 2at 2 The velocity formula v = v0 + at can be used to deduce that the state transfer matrix A of the Kalman filter should be modified to equation (7), where Δt is the length of the interval between tracking adjacent frames. Equation (7) is as follows:

[0032]

[0033] As a further improvement of the present invention, the association matching in step 1 is specifically as follows:

[0034] Based on the motion change method, the IOU distance calculation is performed according to the positional relationship between the prediction box generated by the Kalman filter in motion modeling and state estimation and the detection box predicted by the target detection network, and the Hungarian algorithm is used for association matching calculation.

[0035] The beneficial effects of the present invention are:

[0036] The present invention can solve the problem of missed detection in the process of manual blade counting, solve the problem of workers' repeated detection of aircraft engine blades due to missed detection, improve the work efficiency of workers, and at the same time reduce the work content required by workers when drilling aircraft engines, so as to better help workers focus on the work of finding damage to aircraft engine blades. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The present invention is further described below in conjunction with the accompanying drawings and embodiments:

[0038] Figure 1 This is a framework flow chart of the TBD architecture tracking method in the present invention. DETAILED DESCRIPTION

[0039] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further described below with reference to the accompanying drawings and embodiments.

[0040] A method for tracking and counting blades of an aircraft engine includes tracking blades of an aircraft engine and counting blades of an aircraft engine. The specific steps are as follows:

[0041] Step 1: Use the blade tracking algorithm based on the TBD framework to implement aeroengine blade tracking, such as Figure 1 As shown, including:

[0042] The first step: target detection, sending the video frame to be tested into the target detection network to locate the edge of the aircraft engine blade.

[0043] The technical solution of the present invention is to use the target detection model for the edge of aircraft engine blades obtained by YOLOv8n network training as the target detector of the blade tracking algorithm in the present invention, adjust the head structure of the YOLOv8n network, and embed the BiFormer attention module, so that the network can obtain more powerful detection performance for scenes with large changes in target size such as the edge of aircraft engine blades.

[0044] The main implementation process of the Biformer attention module embedded in the target detection network in aircraft engine blade tracking is as follows:

[0045] (1) Divide the input two-dimensional image into S×S blocks, and easily obtain the feature vector X of each area T The number is X T Through the formula (1), we can perform linear transformation to get the QK V vectors, which are as follows:

[0046]

[0047] (2) Establish connections through directed graphs to find other blocks that need to work together with the current block, as follows:

[0048] Calculate the average value of each area according to Q and K r , K r , multiply the two by formula (2) to obtain the correlation matrix A r , according to the correlation matrix, the first k regions with the highest correlation are reserved for subsequent use. The process is shown in formula (3). r For S 2 *k-sized matrix, whose i-th row stores the k index values ​​with the highest correlation with the i-th region. Formula (2) and formula (3) are as follows:

[0049] A r =Q r (K r ) T (2)

[0050] I r =Topindex(A r ) (3);

[0051] (3) Due to the continuity of GPU memory, it is necessary to aggregate K and V vectors according to the index value. The expression is shown in formula (4). Finally, the traditional self-attention operation is performed between pixel blocks, and the convolution-based local enhancement algorithm is additionally introduced to improve the feature extraction capability, that is, formula (5). Formula (4) and formula (5) are as follows:

[0052] K g =gather(K,I r ),V g =gather(V,I r ) (4)

[0053] O=Attention(Q,K g ,V g )+LCE(V) (5).

[0054] The second step: motion modeling and state estimation, by real-time analysis of the motion state of the currently tracked object, predicting the position of the object in the next frame.

[0055] With regard to motion modeling and state estimation in aircraft engine blade tracking, the technical solution of the present invention is to use a Kalman filter to model the motion of the blade edge and predict and update the subsequent motion; the prediction process is to send the position state vector of the target into the state equation for calculation and estimate the next frame position of the target, and the update process is to obtain the current observation value based on the measurement equation to update the target position.

[0056] The present invention optimizes and improves the state vector of the target position in motion modeling and state estimation in aircraft engine blade tracking. The specific scheme is as follows: since the shooting angle is located at the side of the engine, the blade rotation process presents the characteristics of first acceleration and then deceleration on the two-dimensional image, so acceleration data is introduced to improve the prediction effect of the Kalman algorithm; in view of the large change in the aspect ratio of the blade edge detection frame, the length and width change speed variable is added to the state vector to enhance the tracking effect.

[0057] The modified state vector is shown in formula (6), which is as follows:

[0058] X=[x,y,w,h,x',y',w',h',x”,y”] T (6)

[0059] Where x, y, w, and h are the horizontal coordinate, vertical coordinate, width, and height of the center point of the target frame, respectively; x', y', w', and h' are the first-order derivatives of the above variables reflecting the corresponding speed; and x' and y' are the second-order derivatives corresponding to the acceleration of the target in the horizontal and vertical directions;

[0060] According to the basic displacement formula of physics s = s0 + vt + 1 / 2at 2 The velocity formula v = v0 + at can be used to deduce that the state transfer matrix A of the Kalman filter should be modified to equation (7), where Δt is the length of the interval between tracking adjacent frames. Equation (7) is as follows:

[0061]

[0062] Step 3: Association matching, matching the target detection in the new frame with the predicted value to achieve tracking.

[0063] With regard to association matching in aircraft engine blade tracking, the technical solution of the present invention is to perform IOU (intersection over union) distance calculation based on the motion change method according to the positional relationship between the prediction box generated by the Kalman filter in motion modeling and state estimation and the detection box predicted and generated by the target detection network, and use the Hungarian algorithm to perform association matching calculation.

[0064] In summary, for aircraft engine blade tracking, the overall process of the present invention is as follows:

[0065] (1) The input image is sent to the improved YOLOv8n model to obtain the target detection results of the leaf edge type, and the detection box is classified into two sets of high score and low score by setting the threshold (typical value 0.6);

[0066] (2) By calculating the IOU distance and the Hungarian algorithm, the high-scoring detection frame is matched with the existing trajectory set. If the match is successful, a sequentially increasing ID number will be assigned to the trajectory, which will be sent to the Kalman filter for recursive iteration to update the trajectory information and predict the position of the next frame. If the match fails, the trajectory set will be temporarily retained and the detection frames with matching values ​​below the threshold will be discarded. The detection frames above the threshold will be considered as a new trajectory and incorporated into the trajectory set;

[0067] (3) Using the same method, the trajectory set that failed to match last time is matched again with the low-scoring detection frame. The detection frame that failed to match will be directly deleted, and the trajectory that failed to match will be sent to the cache queue. The Kalman filter is responsible for updating and predicting the trajectory that successfully matched;

[0068] (4) The cache queue length is adjustable (typical value is 30 frames). If the trajectory cannot be matched within the specified number of frames, it is considered that the target has left the camera range and its record will be deleted.

[0069] Step 2: Counting the blades of the aircraft engine based on the trajectory information calculated in step 1, including:

[0070] Step 1: Identify the newly appeared aircraft engine blades and obtain the latest successfully matched trajectory ID number of the aircraft engine blade tracking algorithm for blade edge tracking. If it is different from the ID number recorded last time, the blade is regarded as a newly appeared blade.

[0071] Step 2: Determine the rotation direction of the newly appeared aircraft engine blade and record the starting direction, obtain the starting point and ending point of the latest successfully matched trajectory of the aircraft engine blade tracking algorithm for blade edge tracking, subtract the horizontal coordinates of the ending point from the starting point and use this value as the direction value. If the value is greater than 0, the rotation direction is from left to right. If the value is less than 0, the rotation direction is from right to left. At the same time, the calculated direction value of the trajectory ID number 1 is regarded as the starting direction.

[0072] Step 3: Update the current count value according to the movement direction of the aircraft engine blades. If the direction is opposite to the starting direction, the count value is reduced by 1; if the direction is the same as the starting direction, the count value is increased by 1.

[0073] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for tracking and counting blades of an aircraft engine, characterized in that: Including aircraft engine blade tracking and aircraft engine blade counting, the specific steps are as follows: Step 1: Implementing aeroengine blade tracking using a blade tracking algorithm based on a TBD framework, including: Step 1: Target detection: Send the video frame to be tested into the target detection network to locate the edge of the aircraft engine blade; Step 2: Motion modeling and state estimation: by analyzing the motion state of the currently tracked object in real time, predict the position of the object in the next frame; Step 3: Association matching, matching the target detection of the new frame with the predicted value to achieve tracking; Step 2: Counting the blades of the aircraft engine based on the trajectory information calculated in step 1, including: Step 1: Identify the newly appeared aircraft engine blades and obtain the latest successfully matched track ID number of the aircraft engine blade tracking algorithm for blade edge tracking. If it is different from the ID number recorded last time, the blade is regarded as a newly appeared blade. Step 2: Determine the rotation direction of the newly appeared aircraft engine blade and record the starting direction, obtain the starting point and ending point of the latest successfully matched trajectory of the aircraft engine blade tracking algorithm for blade edge tracking, subtract the horizontal coordinates of the ending point from the starting point and use this value as the direction value. If the value is greater than 0, the rotation direction is from left to right, and if the value is less than 0, the rotation direction is from right to left. At the same time, the calculated direction value of the trajectory ID number 1 is regarded as the starting direction; Step 3: Update the current count value according to the movement direction of the aircraft engine blades. If the direction is opposite to the starting direction, the count value is reduced by 1; if the direction is the same as the starting direction, the count value is increased by 1.

2. The method for tracking and counting blades of an aircraft engine according to claim 1, characterized in that: The target detection in step 1 is as follows: The target detection model for the edge of aviation engine blades obtained by training the YOLOv8n network is used as the target detector of the blade tracking algorithm based on the TBD framework, and the BiFormer attention module is embedded in the head structure of the YOLOv8n network.

3. The method for tracking and counting blades of an aircraft engine according to claim 2, characterized in that: For the embedded Biformer attention module, the main implementation process is as follows: Step 1: Divide the input two-dimensional image into S×S blocks, and obtain the feature vector X of each area T The number is X T Through the linear transformation of formula (1), we can get Q K V vector, formula (1) is as follows: Step 2: Establish connections through directed graphs to find other blocks that need to work together with the current block, as follows: Calculate the average value of each area according to Q and K r , K r , multiply the two by formula (2) to obtain the correlation matrix A r , according to the correlation matrix, the first k regions with the highest correlation are reserved for subsequent use. The process is shown in formula (3). r For S 2 *k-sized matrix, whose i-th row stores the k index values ​​with the highest correlation with the i-th region. Formula (2) and formula (3) are as follows: A r =Q r (K r ) T (2) I r =Top index(A r ) (3); Step 3: Aggregate K and V vectors according to the index value. The expression is shown in formula (4). Finally, perform traditional self-attention operation between pixel blocks and introduce a local enhancement algorithm based on convolution to improve the feature extraction capability, that is, formula (5). Formula (4) and formula (5) are as follows: K g =gather(K,I r ),V g =gather(V,I r ) (4) O=Attention(Q,K g ,V g )+LCE(V) (5)。 4. The method for tracking and counting blades of an aircraft engine according to claim 1, characterized in that: The motion modeling and state estimation in step 1 are as follows: The Kalman filter is used to model the movement of the blade edge, and the subsequent movement is predicted and updated. The prediction process is to send the target position state vector into the state equation for calculation and estimate the target position in the next frame. The update process is to obtain the current observation value based on the measurement equation to update the target position.

5. The method for tracking and counting blades of an aircraft engine according to claim 4, characterized in that: The position state vector of the target is optimized and improved, and formula (6) is as follows: X=[x,y,w,h,x',y',w',h',x”,y”] T (6) Where x, y, w, and h are the horizontal coordinate, vertical coordinate, width, and height of the center point of the target frame, respectively; x', y', w', and h' are the first-order derivatives of the above variables reflecting the corresponding speed; and x' and y' are the second-order derivatives corresponding to the acceleration of the target in the horizontal and vertical directions; According to the basic displacement formula of physics s = s0 + vt + 1 / 2at 2 The velocity formula v = v0 + at can be used to deduce that the state transfer matrix A of the Kalman filter should be modified to equation (7), where Δt is the length of the interval between tracking adjacent frames. Equation (7) is as follows:

6. The method for tracking and counting blades of an aircraft engine according to claim 1, characterized in that: The specific association matching in step 1 is as follows: Based on the motion change method, the IOU distance calculation is performed according to the positional relationship between the prediction box generated by the Kalman filter in motion modeling and state estimation and the detection box predicted by the target detection network, and the Hungarian algorithm is used for association matching calculation.

Images