An intelligent detection and determination method for repeated damage in aerial borescope videos

Abstract

The present invention discloses an intelligent detection and determination method for repeated damage in an aerial boresight video. First, target detection is performed on the internal damage, and the position of the target detection frame of the current frame and the depth features of the image blocks of each detection frame are read; then the detection frame is filtered according to the confidence level; the detection frame is maximum suppressed to eliminate the phenomenon that the same damage has multiple detection frames; the position of the target in the current frame is predicted using a Kalman filter tracker; then the cost matrix of the trajectory information and the detection information is calculated based on the Mahalanobis distance of the appearance information, and cascade matching and IOU matching are performed successively; finally, the Kalman filter tracker parameters and feature sets are updated, and the target disappearance and the appearance of new targets are judged at the same time. In this process, in order to solve the problem of misjudgment of repeated damage in the boresight video, the present invention proposes an intelligent detection and determination method for repeated damage in the aerial boresight video based on the fusion of the twin convolutional neural network and the optimal stacking block.

Description

Technical Field

[0001] The invention relates to an intelligent detection and determination method of repeated damage in an aerial boresight video based on a twin network and an optimal stacking block, and belongs to the field of computer vision. Background Art

[0002] Traditional target tracking methods have made great progress after years of development and can achieve better tracking effects to a certain extent. For example, target tracking is performed through feature space and compressed sampling, a filter is used to alleviate the impact of occlusion during the tracking process, and multi-target tracking is performed through the bee colony optimization algorithm. Although the above methods can achieve target tracking effects to a certain extent, the designed tracker cannot fully learn the characteristics of the target content in the image, so its tracking effect is very unstable, and it is difficult to complete real-time labeling and determine whether it is repeated damage.

[0003] In recent years, target detection and tracking technology has developed rapidly, and tracking-detection technology has become the mainstream technology in the field of multi-target tracking. However, it is mainly focused on the detection and tracking of pedestrians and vehicles, and there is little research on aircraft engine damage. As the core power in the aviation field, it is of great significance to effectively track the damage of the engine. The YOLO algorithm is an efficient recognition algorithm with excellent performance in target detection and recognition applications. It can be used as a target detector in the detection stage of target tracking to provide the initial position of the tracked aircraft engine damage target.

[0004] In the actual application of aviation engine boresight technology, the internal structure of the engine is complex, and the shooting angle of the boresight varies greatly. There will be problems such as obstruction of the detection target, shooting angle, and lighting changes. The existing technology is not suitable for actual boresight technology and cannot meet the needs of real-time detection and judgment of whether it is repeated damage. Summary of the invention

[0005] In order to solve the above problems, the present invention provides an intelligent detection and judgment method for repeated damage in aerial borescope videos based on twin networks and optimal stacking blocks. The method has high accuracy and fast processing speed, reduces the repetition rate of damage in detection, and is more suitable for objects with large angle changes.

[0006] The present invention firstly performs target detection on internal (e.g., internal engine) damage through an aviation boresight video (e.g., internal aircraft engine boresight video), and reads the position of the target detection frame of the current frame and the depth features of each detection frame image block;

[0007] Then, the detection frames are filtered according to the confidence level; the detection frames are maximum suppressed to eliminate the phenomenon of multiple detection frames for the same damage; the Kalman filter tracker is used to predict the position of the target in the current frame;

[0008] Then, the cost matrix of trajectory information and detection information is calculated based on the Mahalanobis distance of appearance information, and cascade matching and IOU matching are performed in sequence;

[0009] Finally, the Kalman filter tracker parameters and feature set are updated, and the disappearance of the target and the appearance of the new target are judged.

[0010] In this process, in order to solve the problem of repeated damage interference in online statistical damage, an intelligent detection and judgment method for repeated damage in aerial borescope videos is proposed based on the fusion technology of twin convolutional neural networks and optimal stacking blocks. The specific steps include:

[0011] Step 1: In the collected boresight video of the aircraft engine, for the original frame of a given video, create a corresponding trajectory frame for the result detection frame detected in the first frame;

[0012] Step 2: Perform IOU matching on the target detection frame of the frame and the trajectory prediction frame of the previous frame one by one, and calculate their cost matrix;

[0013] Step 3: Use all the cost matrices obtained in step 2 as the input of the Hungarian algorithm, and obtain the linear matching result through the Hungarian algorithm;

[0014] Step 4: Repeat steps 2 and 3 until a confirmed track appears in the matching result or the video ends;

[0015] Step 5: Use Kalman filtering to predict the prediction boxes corresponding to the confirmed trajectory and the unconfirmed trajectory of the matching result; then perform cascade matching on the prediction box of the confirmed trajectory and the detection box of the detection result of the original video frame, and calculate the corresponding cost matrix from the matching result;

[0016] Step 6: All the cost matrices obtained in step 5 are used as inputs of the Hungarian algorithm to obtain linear matching results through the Hungarian algorithm;

[0017] Step 7: Send the matching result obtained in step 6 to the model integrating the twin convolutional neural network and the optimal stacking block to perform similarity judgment, and detect and judge the obtained damage;

[0018] Step 8. Repeat steps 5 to 7 until the video ends.

[0019] The above detection box is the target detection box.

[0020] The beneficial effects of the present invention are:

[0021] This technical solution focuses on objects with large changes in shooting angles and those that are occluded. It can accurately propagate their semantic features, add tracking algorithms based on target detection, deduplicate duplicate damage, and improve the accuracy of damage detection.

[0022] The technical reasons for the above achievements of this technical solution are:

[0023] 1) Use existing detectors such as YOLOV5 as target detection, and modify its weight values ​​and network to solve the problem of the same damage recognition being affected by large changes in shooting angles.

[0024] 2) The method to solve the association between detection results and tracking prediction results is to use the Hungarian algorithm, which considers the association of motion information and the association of target appearance information.

[0025] 3) Introducing cascade matching, when a target is obscured for a long time, the uncertainty of the Kalman filter prediction will be greatly increased, and the observability in the state space will be greatly reduced.

[0026] If two trackers compete for the matching right of the same detection result, the track with longer occlusion time will have greater uncertainty in tracking prediction position because the position information has not been updated for a long time, that is, the covariance will be larger. The inverse of the covariance is used in the Mahalanobis distance calculation, so the Mahalanobis distance will be smaller, making the detection result more likely to be associated with the track with longer occlusion time. This undesirable effect often destroys the continuity of tracking. The final stage of matching also performs IOU-based matching on uncertain and outdated tracks that have not been successfully matched.

[0027] 4) The addition of twin network and optimal stacking fast fusion effectively improves the problem of repeated judgment of the same damage caused by occlusion and lighting of the engine damage target. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0029] Figure 1 It is a flow chart of the whole invention.

[0030] Figure 2 It is a flow chart of fusion twin network and optimal stacking blocks. DETAILED DESCRIPTION

[0031] The present invention discloses an intelligent detection and determination method for repeated damage in an aerial boresight video. First, target detection is performed on the internal damage, and the position of the target detection frame of the current frame and the depth features of the image blocks of each detection frame are read; then the detection frame is filtered according to the confidence level; the detection frame is maximum suppressed to eliminate the phenomenon that the same damage has multiple detection frames; the position of the target in the current frame is predicted using a Kalman filter tracker; then the cost matrix of the trajectory information and the detection information is calculated based on the Mahalanobis distance of the appearance information, and cascade matching and IOU matching are performed successively; finally, the Kalman filter tracker parameters and feature sets are updated, and the target disappearance and the appearance of new targets are judged at the same time. In this process, in order to solve the problem of misjudgment of repeated damage in the boresight video, the present invention proposes an intelligent detection and determination method for repeated damage in the aerial boresight video based on the fusion of the twin convolutional neural network and the optimal stacking block.

[0032] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0033] The specific implementation of the present invention is as follows:

[0034] Step 1: In the collected borescope video of the aircraft engine, for the original frame of the given video, create the corresponding track for the detection frame of the first frame. Specifically, step 1 includes the following sub-steps:

[0035] Step 1.1, use the target detector to detect the video frame and obtain the detection frame;

[0036] Step 1.2, mark the corresponding damage in all target frames and extract features; the features include appearance features and motion features;

[0037] Step 1.3: Initialize the motion variable of the Kalman filter and predict its corresponding prediction box through the Kalman filter.

[0038] Step 2: Perform IOU matching between the detection frame and the trajectory frame of the previous frame one by one, and calculate their cost matrix.

[0039] Step 3: All cost matrices obtained in step 2 are used as inputs of the Hungarian algorithm to obtain the linear matching result through the Hungarian algorithm. Specifically, step 3 includes the following sub-steps:

[0040] Step 3.1: For each row of the cost matrix, subtract the minimum element of that row;

[0041] Step 3.2: Subtract the minimum element of each column from the elements of each column of the generated matrix;

[0042] Step 3.3, connect all zeros in the new matrix with the least number of row lines and column lines, and check whether the current allocation is optimal; if the row lines and column lines do not connect all elements of the matrix, proceed to step 3.4, otherwise proceed to step 3.5;

[0043] Step 3.4, find the smallest element among the elements that are not connected between the row and column lines, subtract the smallest element from the remaining elements, and add the smallest element to the element corresponding to the intersection of the row and column lines;

[0044] Step 3.5, find the 0 element corresponding to each row and column, and find the optimal allocation based on the 0 element;

[0045] Step 3.6: To incorporate motion information, use the square of the Mahalanobis distance between the predicted Kalman filter state and the newly arrived measurement, i.e., the first metric is d (1) express:

[0046]

[0047] Among them, d (1) (i, j) represents the motion matching degree between the j-th detection box and the i-th track, x i is the predicted observation of the trajectory at the current moment, S i is the covariance matrix of the observation space at the current moment predicted by the Kalman filter, expressed as d j Indicates the current trajectory status of the detected target;

[0048] Step 3.7. Using the indicator from step 3.6, exclude associations with low likelihood by setting the threshold of the Mahalanobis distance to the 95% confidence interval calculated from the inverse chi-square distribution, and use an indicator to indicate this decision:

[0049]

[0050] Among them, the corresponding Mahalanobis threshold is t (1) When d (1) (i, j) is greater than t (1) ,but The value is set to 0, otherwise it is set to 1. When it is 0, it means that the Mahalanobis distance between the i-th trajectory box and the j-th detection box does not match, and when it is 1, it means that they match;

[0051] Step 3.8: Use the second metric to measure the minimum cosine distance between the i-th trajectory box and the j-th detection box, using d (2) express:

[0052]

[0053] Among them, R iIt is a container that stores the feature vector of the trajectory box of the matching result. j is the feature vector of the jth detection box, Represents the k feature vectors of the i-th trajectory box;

[0054] Step 3.9: Introduce a binary variable to indicate whether this metric can accept an association:

[0055]

[0056] Among them, t (2) is a threshold, if d (2) (i, j) is greater than t (2) ,but The value of is set to 0, otherwise it is 1; When it is 0, it means that the cosine distance between the i-th trajectory box and the j-th detection box does not match, and when it is 1, it means they match;

[0057] Step 3.10. To formulate the association question, combine the two measures using a weighted sum:

[0058] h (i，j) =λd (1) (i, j)+(1-λ)d (2) (i, j)

[0059] Among them, d (1) (i, j) has been defined in step 3.6, d (2) (i, j) has been defined in step 3.8, λ represents the weight, ranging from 0 to 1;

[0060] Step 3.11: If a correlation is within the gated region of both metrics, it is called an acceptable correlation:

[0061]

[0062] There are three results of linear matching:

[0063] The first is trajectory mismatch, in which case the mismatched trajectory is directly deleted;

[0064] The second is that the prediction result and the detection result do not match. In this case, such a detection result is initialized as a new trajectory;

[0065] The third is that the detection frame of the next frame is successfully paired with the prediction frame of the previous frame, that is, the tracking of two adjacent frames is successful. At this time, the detection result corresponding to the next frame is updated through Kalman filtering to update its corresponding trajectory variable.

[0066] Step 4: Repeat steps 2 and 3 until a confirmed track appears in the matching result or all frames are processed.

[0067] Step 5: Use Kalman filtering to predict the boxes corresponding to the confirmed state trajectory and the unconfirmed state trajectory, and perform cascade matching on the confirmed state trajectory box and the detection box. Specifically, step 5 includes the following sub-steps:

[0068] Step 5.1, Kalman filter prediction, predicting the value at the next moment based on the estimated value at the previous moment, called a priori estimation, and this step will also predict the error at the next moment, called a priori error;

[0069] Among them, there are three results of cascade matching:

[0070] The first one is trajectory matching, which updates the corresponding trajectory variables through Kalman filtering.

[0071] The second type is trajectory mismatch. The previously uncertain trajectory is combined with the mismatched trajectory and the mismatched detection frame for IOU matching one by one, and then the cost matrix is ​​calculated based on the IOU matching results.

[0072] The third method is to detect mismatches. The previously uncertain trajectory is combined with the mismatched trajectory and the mismatched detection frame for IOU matching one by one, and then the cost matrix is ​​calculated based on the results of IOU matching.

[0073] Step 6: All the cost matrices obtained in step 5 are used as the input of the Hungarian algorithm to obtain the linear matching result through the Hungarian algorithm. Specifically, step 6 includes the following sub-steps:

[0074] Step 6.1: If the track is mismatched, delete the mismatched track directly, otherwise go to step 6.2;

[0075] Step 6.2: If the detection result is mismatched, initialize such detection result as a new trajectory, otherwise go to step 6.3;

[0076] Step 6.3: If the detection frame and the predicted frame are paired successfully, that is, the previous frame and the next frame are tracked successfully, the corresponding detection result is updated through the Kalman filter to update the corresponding trajectory variable. Specifically:

[0077] Step 6.3.1, Kalman filter update, first calculate the Kalman gain, then use the prior estimate in the previous step to calculate the posterior estimate, and update the prior error to obtain the posterior error;

[0078] Step 6.3.2: At this time, feature extraction is performed on the damage and identification is performed again; for the case where the same damage appears in the previous and next frames and is assigned different ID values, deduplication is performed.

[0079] Step 7: Send the matching results obtained in step 6 to the model integrating the twin convolutional neural network and the optimal stacking block for similarity judgment, and detect and judge the obtained damage. Specifically, step 7 includes the following sub-steps:

[0080] Step 7.1, for two adjacent frame images, respectively divide them into a plurality of stacked blocks, and the stacked blocks are used as sub-images;

[0081] Step 7.2, each sub-image is processed independently for preliminary damage similarity verification decision;

[0082] Step 7.3: For the two groups of sub-images, output similarity scores by judging damage similarity;

[0083] If the score is not greater than the threshold, they are not the same lesion; if the score is less than the threshold, the irrelevant sub-image with low discriminability is discarded, and the best verification accuracy of the sub-image with high discriminability is selected;

[0084] Step 7.4: Use the twin convolutional neural network and the best stacked block fusion to train the pre-trained model to extract the features of the best selected sub-image, and then fuse them into a single feature vector;

[0085] Step 7.5, passing the feature set consisting of the feature vector to the classifier to obtain the final damage similarity verification decision;

[0086] Step 7.6: Obtain damage similarity verification results and compare them to reduce the misjudgment rate of the same damage in the aerial borescope video.

[0087] Furthermore, step 7.4 includes the following sub-steps:

[0088] Step 7.4.1, using a supervised end-to-end deep Siamese convolutional neural network based on feature similarity; in particular, the Siamese network architecture based on feature similarity, this method is different from other methods, it uses a Siamese network to verify the damage similarity from the entire image through a single feature extraction, and then applies a single similarity metric to merge and calculate the similarity between feature vectors, and then determine whether two damages are the same damage;

[0089] Step 7.4.2: Use the twin convolutional neural network and the optimal stacking block for training to obtain the best training model to enhance the judgment of damage similarity.

[0090] Furthermore, step 7.5 includes the following sub-steps:

[0091] The results of the classifier are further processed to improve the performance of damage similarity judgment:

[0092] Remove irrelevant parts from the sub-image blocks and keep only the common sub-images with strong correlation as the relational evidence of damage similarity to minimize the distance between damage sub-image pairs (same damage) or maximize the distance between damage sub-image pairs (non-same damage);

[0093] If the original image is stacked and divided into m blocks, i.e. m sub-images, they and the image to be detected form m image pairs;

[0094] Recall the similarity scores of the previous m image pairs; suppose there are N containers C = (c1, c2, c3, ..., c n ), each container has a parent and subset

[0095] Given P a ∈F p =(P a，1 , P a，2 , P a，3 , ..., P a，n ) and P b ∈S c =(P b，1 , P b，2 , P b，3 , ..., P b，n ), where P a ∈F p and P b ∈S c Respectively indicate that they belong to F p and S c The m sub-image pairs in the pair, P a and P b Represents each sub-image;

[0096] For each container C, the sub-image P a and P b The sequence correlation is C = (c1 (P a -P b ), c2(P a -P b ), c3(P a -P b ), ..., c n (P a -P b ));

[0097] Then, based on the test set T = {e1(c1(P a -P b )), e2(c2(P a -P b )), e3(c3(P a -Pb )), ..., e n (c n (P a -P b ))}, where e n represents the similarity accuracy evaluation, c n Represents a container, P a and P b represents a sub-image;

[0098] Use the accuracy e obtained in the previous step n To determine the most similar appearance components between the two lesions;

[0099] After obtaining the accuracy score, the lesion components are located and selected according to the specified similarity threshold, represented by sub-images that share the most similar cues and ignore the dissimilar parts.

[0100] Step 8: Repeat steps 5, 6 and 7 until the video frame ends.

[0101] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An intelligent detection and determination method for repeated damage in aerial borescope videos, characterized in that: The following steps are involved: Step 1: In the collected aerial boresight video, for the original frame of a given video, create a corresponding trajectory frame from the detection frame of the first frame; Step 2: Perform IOU matching between the detection frame and the trajectory frame of the previous frame one by one, and calculate their cost matrix; Step 3: Use all the cost matrices obtained in step 2 as the input of the Hungarian algorithm, and obtain the linear matching result through the Hungarian algorithm; Step 4: loop step 2 and step 3 until the matching result shows a confirmed track or all frames are processed; Step 5: Predict the confirmed state trajectory prediction frame and the unconfirmed state trajectory prediction frame of the matching result through Kalman filtering; Then, the confirmed trajectory prediction box and the detection box of the original video frame detection result are cascade matched, and the corresponding cost matrix is ​​calculated from the matching results; Step 6: Use all the cost matrices obtained in step 5 as the input of the Hungarian algorithm, and obtain the linear matching result through the Hungarian algorithm; Step 7: Send the matching results obtained in step 6 to the model integrating the twin convolutional neural network and the optimal stacking block to perform similarity judgment, and detect and judge the obtained damage; Step 8: Loop through steps 5 to 7 until all frames are processed; In step seven: Step 7.1, for two adjacent frame images, respectively divide them into a plurality of stacked blocks, and the stacked blocks are used as sub-images; Step 7.2, each sub-image is processed independently for preliminary damage similarity verification decision; Step 7.3: For the two groups of sub-images, output similarity scores by judging damage similarity; If the score is not greater than the threshold, they are not the same lesion; if the score is less than the threshold, the irrelevant sub-image with low discriminability is discarded, and the best verification accuracy of the sub-image with high discriminability is selected; Step 7.4: Use the twin convolutional neural network and the best stacked block fusion to train the pre-trained model to extract the features of the best selected sub-image, and then fuse them into a single feature vector; Step 7.5, passing the feature set consisting of the feature vector to the classifier to obtain the final damage similarity verification decision; Step 7.6: Obtain damage similarity verification results and compare them to reduce the misjudgment rate of the same damage in the aerial borescope video.

2. The intelligent detection and determination method of repeated damage in aerial borescope video according to claim 1 is characterized in that: In the step 1: Step 1.1, use the target detector to detect the video frame and obtain the detection frame; Step 1.2, mark the corresponding damage in all target frames and extract features; the features include appearance features and motion features; Step 1.3: Initialize the motion variable of the Kalman filter and predict its corresponding prediction box through the Kalman filter.

3. The intelligent detection and determination method of repeated damage in aerial borescope video according to claim 1 is characterized in that: In the step three: Step 3.1: For each row of the cost matrix, subtract the minimum element of that row; Step 3.2: Subtract the minimum element of each column from the elements of each column of the generated matrix; Step 3.3, connect all zeros in the new matrix with the least number of row lines and column lines, and check whether the current allocation is optimal; if the row lines and column lines do not connect all elements of the matrix, proceed to step 3.4, otherwise proceed to step 3.5; Step 3.4, find the smallest element among the elements where the row and column lines are not connected, subtract the smallest element from the remaining elements, and add the smallest element to the element at the intersection of the row and column lines; Step 3.5, find the 0 element corresponding to each row and column, and find the optimal allocation based on the 0 element; Step 3.6: To incorporate motion information, use the square of the Mahalanobis distance between the predicted Kalman filter state and the newly arrived measurement, i.e., the first metric is d (1) express: Among them, d (1) (i, j) represents the motion matching degree between the j-th detection box and the i-th track, x i is the predicted observation of the trajectory at the current moment, S i is the covariance matrix of the observation space at the current moment predicted by the Kalman filter, expressed as d j Indicates the current trajectory status of the detected target; Step 3.

7. Using the indicator from step 3.6, exclude associations with low likelihood by setting the threshold of the Mahalanobis distance to the 95% confidence interval calculated from the inverse chi-square distribution, and use an indicator to indicate this decision: Among them, the corresponding Mahalanobis threshold is t (1) If d (1) (i, j) is greater than t (1) ,but The value of is set to 0, otherwise it is 1; When it is 0, it means that the Mahalanobis distance between the i-th trajectory box and the j-th detection box does not match, and when it is 1, it means that they match; Step 3.8: Use the second metric to measure the minimum cosine distance between the i-th trajectory box and the j-th detection box, using d (2) express: d (2) (i,j)=min{1-r j T r i (k) |r i (k) ∈R i } Among them, R i It is a container that stores the feature vector of the trajectory box of the matching result; r j is the feature vector of the jth detection box, r i (k) Represents the k feature vectors of the i-th trajectory box; Step 3.9: Introduce a binary variable to indicate whether this metric can accept an association: Among them, t (2) is a threshold, if d (2) (i, j) is greater than t (2) ,but The value of is set to 0, otherwise it is 1; When it is 0, it means that the cosine distance between the i-th trajectory box and the j-th detection box does not match, and when it is 1, it means they match; Step 3.

10. To formulate the association question, combine the two measures using a weighted sum: h (i，j) =λd (1) (i,j)+(1-λ)d (2) (i,j) Among them, d (1) (i, j) has been defined in step 3.6, d (2) (i, j) has been defined in step 3.8, λ represents the weight, ranging from 0 to 1; Step 3.11: If a correlation is within the gated region of both metrics, it is called an acceptable correlation: There are three results of linear matching: The first is trajectory mismatch, in which case the mismatched trajectory is directly deleted; The second is that the prediction result and the detection result do not match. In this case, such a detection result is initialized as a new trajectory; The third is that the detection frame of the next frame is successfully paired with the prediction frame of the previous frame, that is, the tracking of two adjacent frames is successful. At this time, the detection result corresponding to the next frame is updated through Kalman filtering to update its corresponding trajectory variable.

4. The intelligent detection and determination method of repeated damage in aerial borescope video according to claim 1 is characterized in that: In the step 5: Step 5.1, Kalman filter prediction, predicting the value at the next moment based on the estimated value at the previous moment, called a priori estimation, and this step will also predict the error at the next moment, called a priori error; Among them, there are three results of cascade matching: The first one is trajectory matching, which updates the corresponding trajectory variables through Kalman filtering. The second type is trajectory mismatch. The previously uncertain trajectory is combined with the mismatched trajectory and the mismatched detection frame for IOU matching one by one, and then the cost matrix is ​​calculated based on the IOU matching results. The third method is to detect mismatches. The previously uncertain trajectory is combined with the mismatched trajectory and the mismatched detection frame for IOU matching one by one, and then the cost matrix is ​​calculated based on the results of IOU matching.

5. The intelligent detection and determination method of repeated damage in aerial borescope video according to claim 1 is characterized in that: In step six: Step 6.1: If the track is mismatched, delete the mismatched track directly, otherwise go to step 6.2; Step 6.2: If the detection result is mismatched, initialize such detection result as a new trajectory, otherwise go to step 6.3; Step 6.3: If the detection frame and the predicted frame are paired successfully, that is, the previous frame and the next frame are tracked successfully, the corresponding detection result is updated through the Kalman filter to update the corresponding trajectory variable.

6. The intelligent detection and determination method of repeated damage in aerial borescope video according to claim 5 is characterized in that: In step 6.3: Step 6.3.1, Kalman filter update, first calculate the Kalman gain, then use the prior estimate in the previous step to calculate the posterior estimate, and update the prior error to obtain the posterior error; Step 6.3.2: At this time, feature extraction is performed on the damage and identification is performed again; for the case where the same damage appears in the previous and next frames and is assigned different ID values, deduplication is performed.

7. The intelligent detection and determination method of repeated damage in aerial borescope video according to claim 1 is characterized by: The step 7.4 comprises: Step 7.4.1, using a supervised end-to-end deep Siamese convolutional neural network based on feature similarity; The convolutional neural network is a Siamese network architecture based on feature similarity. It uses a Siamese network to verify damage similarity through a single feature extraction from the entire image, and then applies a single similarity metric to merge and calculate the similarity between feature vectors to determine whether two damages are the same damage; Step 7.4.2: Use the twin convolutional neural network and the optimal stacking block for training to obtain the best training model to enhance the judgment of damage similarity.

8. The intelligent detection and determination method of repeated damage in aerial borescope video according to claim 1 is characterized by: The step 7.5 comprises: The results of the classifier are further processed to improve the performance of damage similarity judgment: Remove irrelevant parts from the sub-image blocks and keep only the common sub-images with strong correlation as the relational evidence of damage similarity to minimize the distance between the damage sub-image pairs, i.e. the same damage, or maximize the distance between the damage sub-image pairs, i.e. non-identical damage; If the original image is stacked and divided into m blocks, i.e. m sub-images, they and the image to be detected form m image pairs; Recall the similarity scores of the previous m image pairs; suppose there are N containers C = (c1, c2, c3, ..., c n ), each container has a parent and subset Given and Where P a ∈F p and P b ∈S c Respectively indicate that they belong to F p and S c The m sub-image pairs in the pair, P a and P b Represents each sub-image; For each container C, the sub-image P a and P b The sequence correlation is C = (c1 (P a -P b ), c2(P a -P b ), c3(P a -P b ), ..., c n (P a -P b )); Then, based on the test set T = {e1(c1(P a -P b )), e2(c2(P a -P b )), e3(c3(P a -P b )), ..., e n (c n (P a -P b ))}, where e n represents the similarity accuracy evaluation, c n Represents a container, P a and P b represents a sub-image; Use the accuracy e obtained in the previous step n To determine the most similar appearance components between the two lesions; After obtaining the accuracy score, the lesion components are located and selected according to the specified similarity threshold, represented by sub-images that share the most similar cues and ignore the dissimilar parts.

9. The intelligent detection and determination method of repeated damage in aerial borescope video according to claim 1 is characterized by: The aviation borescope video is a borescope video inside an aircraft engine.

Images