Aero-engine blade online counting method based on borescope video stream

Through parallel tasks based on hole-detection video stream and inter-frame differential and cosine similarity algorithm, the problem of low error and efficiency in aircraft engine blade detection and counting is solved, real-time blade rotation direction judgment and online counting are realized, and detection efficiency is improved and accuracy.

CN119090804APending Publication Date: 2024-12-06NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411016898.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art has problems of error and low efficiency in the detection and counting of aircraft engine blades, especially the manual counting method is cumbersome and difficult to apply to various types of engines, resulting in repeated detection and errors.

Method used

The method based on hole-detection video stream is adopted to judge the blade rotation direction in real time through parallel tasks and update the number of blades. The inter-frame difference and cosine similarity algorithm are used to realize online counting to ensure detection efficiency and accuracy.

Benefits of technology

It improves the efficiency and accuracy of engine blade detection, reduces repeated detection, concentrates workers on damage detection, and reduces the possibility of errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090804A_ABST
    Figure CN119090804A_ABST
Patent Text Reader

Abstract

The invention discloses an aero-engine blade online counting method based on a borescope video stream, and aims to solve the problem that part of blades are missed or overrecorded due to various blades under manual counting of the blades. The aero-engine blade online counting method based on the borescope video stream comprises the steps of collecting borescope video stream images and feeding the borescope video stream images to two parallel tasks, including blade rotation direction judgment and blade counting algorithm implementation. In the first task, three continuous frames of images are obtained from a video stream, the area of a moving target in the three frames of images is calculated, and finally the rotation direction of the blade is judged; in the second task, the image of the input video frame is preprocessed, the image of the template frame is obtained, then the online counting algorithm is initialized, and finally the blade online counting algorithm is operated; and after the execution of the two parallel tasks is finished, updating the leaf count value. The method can be embedded in the aero-engine blade damage detection process, the number of the currently detected blades is recorded, detection personnel concentrate on the damage on the blades, and repeated detection or missing detection is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an online counting method for aircraft engine blades based on borescope video stream, belonging to the field of video interpretation processing methods. Background Art

[0002] Most of the engine accidents in flight are related to the failure of engine blades. Once the blade breaks, the damage to the engine and even the aircraft is fatal. Therefore, safe flight requires frequent maintenance, repair and overhaul of the aircraft. The operator observes and records the slight changes or damage of the internal structure through the borescope. In this process, the inspected blades need to be marked to improve the inspection efficiency and avoid repeated inspections. Although there are some electric rotating devices that can count the engine blades, these devices are difficult to apply to every engine. Therefore, the manual counting method cannot be replaced, but checking and counting blades is a boring and tedious task, which is easy to make mistakes.

[0003] The invention discloses an online counting method for aircraft engine blades based on borescope video stream, which can judge the rotation direction of blades and update the number of blades in real time, is conducive to focusing workers' attention on damage detection, improves detection efficiency and avoids repeated detection. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention provides an online counting method for aircraft engine blades based on borescope video stream, which can judge the rotation direction of engine blades in real time and update the number of blades currently inspected, thereby improving blade detection efficiency and avoiding repeated detection.

[0005] An online counting method for aircraft engine blades based on borescope video stream, the method comprising:

[0006] The borescope video stream is used as input and the image information is fed into two parallel tasks. The first task is to determine the rotation direction of the blade.

[0007] The second task is to calculate the correlation between the template frame and the subsequent video stream, and use the correlation strength value to obtain the number of leaves through an online counting algorithm;

[0008] Furthermore, the borescope video stream is used as the input video stream features as follows:

[0009] When performing borescope inspection, ensure that the borescope camera position does not change in shooting angle and position during the operation of the blade counting algorithm;

[0010] Furthermore, the image information is fed into two parallel tasks. In the first task, the method for determining the rotation direction of the blade is as follows:

[0011] Set three consecutive frames of images in the video stream to f k-1 (x,y),f k (x,y),f k+1 (x, y), perform inter-frame difference operation on the previous frame image and the middle frame to obtain the motion change image g1(x, y), as shown in the following formula:

[0012]

[0013] The middle frame image and the next frame image are subjected to inter-frame difference operation to obtain the motion change image g2(x, y). The motion change images g1(x, y) and g2(x, y) are subjected to AND operation to obtain the motion target area D(x, y) of the middle frame image, as shown in the following formula:

[0014]

[0015] Set a queue with a length of 10 to store the calculated horizontal coordinates of the center point of the minimum circumscribed rectangle of the largest contour in the moving target area. Add the horizontal coordinates of the center point of the minimum circumscribed rectangle at this time to the queue every 5 frames. When the length of the queue is 10, calculate the difference between adjacent elements, start traversing from the head of the queue, and count the number of differences less than 0 and greater than 0. If the number greater than 0 is greater than the number less than 0, the rotation direction is considered to be from right to left; if the number greater than 0 is less than the number less than 0, the rotation direction is considered to be from left to right.

[0016] Furthermore, the image information is fed into two parallel tasks. In the second task, the leaf online counting algorithm is implemented as follows:

[0017] First, the input video frame is preprocessed to reduce the computational complexity of subsequent similarity operations. Each frame of the input video is subjected to dimensionality reduction processing, and the original three-channel image format is converted into a single-channel image of size 112×112×1. This image is flattened to obtain a 12,544-length image vector that is convenient for similarity calculation. The template image vector is denoted as A, and the i-th value of this vector is denoted as A i , the comparison graph vector is denoted as B, and the i-th value of this vector is denoted as B i , calculate the cosine similarity S(A, B) between the template frame and all consecutive frames, as shown in the following formula:

[0018]

[0019] Second, the acquisition of the template frame. When the camera captures the image in a static state, that is, the camera position remains unchanged and the blades do not rotate, the current frame is intercepted as the template frame for calculating the cosine similarity with other frames of the video;

[0020] Third, the online counting algorithm is initialized. After intercepting the template frame, the leaves are rotated for a number of times (typical value is 2-3), and the direction of rotation at this time is regarded as the initial direction. The cosine similarity between each frame and the template frame is obtained to obtain a set of one-dimensional signals with a length of 200-300 frames. The peak and valley values ​​of this signal are obtained by using the AMPD algorithm to calculate the constraints of the leaf online counting algorithm, including the maximum value of the peak, the minimum value of the valley, the minimum peak-to-valley difference (set to 0.1 times the difference between the maximum peak and the minimum valley), and the initial value of the adaptive dynamic threshold (set to half the sum of the maximum peak and the minimum valley);

[0021] Fourth, the blade online counting algorithm is implemented. The constraint conditions calculated in the initialization of the blade counting algorithm in the third point are used to determine whether the current frame is a peak value or a valley value. Half of the sum of the peak value and valley value each time is used as the binarization condition of the current signal. The signal after cosine similarity calculation is binarized. The falling edge of the binarized signal is used as the counting condition. Whenever a falling edge comes, the count value is increased or decreased according to the rotation direction.

[0022] Furthermore, the online counting algorithm is initialized, and the peak and valley values ​​of the signal are obtained by using the AMPD algorithm to calculate the initialization conditions. The implementation method is as follows:

[0023] Using a multi-scale sliding window, find the local maximum value and calculate the peak value. By flipping the waveform data, calculate the valley value. Using these peak and valley values, calculate the initialization conditions of the online counting algorithm (maximum value of the peak value, minimum value of the valley value, minimum peak-to-valley interval, and initial value of the adaptive dynamic threshold).

[0024] Furthermore, the leaf online counting algorithm is implemented, and the constraint conditions obtained by initializing the leaf counting algorithm are used to determine whether the current frame is a peak value or a valley value. The implementation method is as follows:

[0025] Only when the local maximum value is greater than the current adaptive dynamic threshold and the difference between the current peak value and the valley value is greater than the minimum peak-valley difference, the value at this time is regarded as the current peak value;

[0026] Only when the local minimum value is smaller than the current adaptive dynamic threshold and the difference between the current peak value and the valley value is greater than the minimum peak-valley difference, the value at this time is regarded as the valley value; BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flowchart for an online counting method of aircraft engine blades based on borescope video stream DETAILED DESCRIPTION

[0028] In order to further describe the technical features and effects of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0029] like Figure 1 As shown, the present invention discloses an online counting method for aircraft engine blades based on borescope video stream, the method comprising:

[0030] First, after collecting the borescope video stream, each frame will be converted into an image and fed into two parallel tasks. Task one is to determine the rotation direction of the blades, and task two is the blade counting algorithm.

[0031] At the same time, when conducting borescope inspection, ensure that the shooting angle and position of the borescope camera do not change during the operation of the blade counting algorithm;

[0032] Task 1, determining the blade rotation direction, includes the following steps:

[0033] Step 1: Get three consecutive frames of images in the video stream and set the three consecutive frames of images in the video stream as f k-1 (x,y),f k (x,y),f k+1 (x, y), perform inter-frame difference operation on the previous frame image and the middle frame to obtain the motion change image g1(x, y), as shown in the following formula:

[0034]

[0035] Step 2: Calculate the moving target area. Perform inter-frame difference operation between the middle frame image and the next frame image to obtain the moving change image g2(x, y). Perform AND operation on the moving change images g1(x, y) and g2(x, y) to obtain the moving target area D(x, y) of the middle frame image, as shown in the following formula:

[0036]

[0037] Step 3, determine the rotation direction of the blade, set a queue with a length of 10 to store the calculated horizontal coordinates of the center point of the minimum circumscribed rectangle of the largest contour in the moving target area, and add the horizontal coordinates of the center point of the minimum circumscribed rectangle at this time to the queue every 5 frames. When the length of the queue is 10, calculate the difference between adjacent elements, and traverse from the head of the queue. Count the number of differences less than 0 and greater than 0. If the number greater than 0 is greater than the number less than 0, the rotation direction is considered to be from right to left; if the number greater than 0 is less than the number less than 0, the rotation direction is considered to be from left to right.

[0038] Task 2, leaf counting algorithm, includes the following steps:

[0039] Step 1: Preprocess the input video frame to reduce the computational complexity of subsequent similarity operations. Perform dimensionality reduction on each frame of the input video, convert the original three-channel image format into a single-channel image of size 112×112×1, and flatten the image to obtain a 12,544-length image vector that is convenient for similarity calculation. The template image vector is denoted as A, and the i-th value of the vector is denoted as A i , the comparison graph vector is denoted as B, and the i-th value of this vector is denoted as B i , calculate the cosine similarity S(A, B) between the template frame and all consecutive frames, as shown in the following formula:

[0040]

[0041] Acquisition of template frame: when the camera captures the image in a static state, that is, the camera position remains unchanged and the blades do not rotate, the current frame is intercepted as the template frame for calculating the cosine similarity with other frames of the video;

[0042] Step 2, online counting algorithm initialization, after intercepting the template frame, rotate a number of leaves (typical value is 2-3), regard the direction of rotation at this time as the initial direction, obtain the cosine similarity between each frame and the template frame, and obtain a set of one-dimensional signals with a length of 200-300 frames. Use the AMPD algorithm to obtain the peak and valley values ​​of this signal to calculate the constraints of the leaf online counting algorithm, including the maximum value of the peak, the minimum value of the valley, the minimum peak-to-valley difference (set to 0.1 times the difference between the maximum peak and the minimum valley), and the initial value of the adaptive dynamic threshold (set to half the sum of the maximum peak and the minimum valley);

[0043] The AMPD algorithm is implemented by using a multi-scale sliding window to find the local maximum value, calculate the peak value, and calculate the valley value by flipping the waveform data. Using these peak and valley values, the initialization conditions of the online counting algorithm (maximum value of the peak value, minimum value of the valley value, minimum peak-to-valley interval, and initial value of the adaptive dynamic threshold) are calculated.

[0044] Step 3: Implement the online blade counting algorithm. Use the constraint conditions calculated in the initialization of the blade counting algorithm in the third point to determine whether the current frame is a peak value or a valley value. Take half of the sum of the peak value and valley value each time as the binarization condition of the current signal. Binarize the signal after cosine similarity calculation. Take the falling edge of the binarized signal as the counting condition. Whenever a falling edge comes, the count value is increased or decreased according to the rotation direction.

[0045] The leaf online counting algorithm is implemented, and the constraint conditions obtained by initializing the leaf counting algorithm are used to determine whether the current frame is a peak value or a valley value. The implementation method is as follows:

[0046] Only when the local maximum value is greater than the current adaptive dynamic threshold and the difference between the current peak value and the valley value is greater than the minimum peak-valley difference, the value at this time is regarded as the current peak value;

[0047] Only when the local minimum value is smaller than the current adaptive dynamic threshold and the difference between the current peak value and the valley value is greater than the minimum peak-valley difference, the value at this time is regarded as the valley value;

[0048] Finally, when the two parallel tasks are finished, the number of blades is updated according to the calculation results of the blade counting algorithm and the blade rotation direction results. When the blade counting algorithm detects the arrival of the falling edge, it determines whether the current blade rotation direction is consistent with the starting direction of the blade. If they are consistent, the count value is increased by one, otherwise the count value is reduced by one.

[0049] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An online counting method for aircraft engine blades based on borescope video stream, used for counting the number of blades during borescope damage detection of aircraft engine blades, characterized in that: The method includes: The borescope video stream is used as input and the image information is sent to two parallel tasks. The first task is to determine the rotation direction of the blades. The second task is to correlate the template frame with the subsequent video stream and use the correlation strength value to obtain the number of blades through an online counting algorithm.

2. The method for online counting of aircraft engine blades based on borescope video stream according to claim 1, characterized in that: The borescope video stream is used as the input video stream, which is characterized in that when performing borescope detection, it is ensured that the shooting angle and position of the borescope camera do not change during the operation of the blade counting algorithm.

3. The method for online counting of aircraft engine blades based on borescope video stream according to claim 1, characterized in that: The image information is sent to two parallel tasks. In the first task, the specific implementation method is as follows: Set three consecutive frames of images in the video stream to f k-1 (x,y),f k (x,y),f k+1 (x, y), the previous frame image and the middle frame are subjected to inter-frame difference operation to obtain the binary image g1(x, y) of motion change, T0 is the threshold set for the difference operation binary processing, and the implementation method is shown in the following formula: The middle frame image and the next frame image are subjected to inter-frame difference operation to obtain the motion change image g2(x, y). The motion change images g1(x, y) and g2(x, y) are subjected to AND operation to obtain the motion target area D(x, y) of the middle frame image, as shown in the following formula: Set a queue with a length of 10 to store the calculated horizontal coordinates of the center point of the minimum enclosing rectangle of the largest contour in the moving target area. Add the horizontal coordinates of the center point of the minimum enclosing rectangle at this time to the queue every 5 frames. When the length of the queue is 10, calculate the difference between adjacent elements, and traverse from the head of the queue. Count the number of differences less than 0 and greater than 0. If the number greater than 0 is greater than the number less than 0, the rotation direction is considered to be from right to left; if the number greater than 0 is less than the number less than 0, the rotation direction is considered to be from left to right.

4. The method for online counting of aircraft engine blades based on borescope video stream according to claim 1, characterized in that: The image information is sent to two parallel tasks. In the second task, the specific implementation method is as follows: First, the input video frame is preprocessed to reduce the computational complexity of subsequent similarity operations. Each frame of the input video is subjected to dimensionality reduction processing, and the original three-channel image format is converted into a single-channel image of size 112×112×1. This image is flattened to obtain a 12,544-length image vector that is convenient for similarity calculation. The template image vector is denoted as A, and the i-th value of this vector is denoted as A i , the comparison graph vector is denoted as B, and the i-th value of this vector is denoted as B i , calculate the cosine similarity S between the template frame and all consecutive frames according to formula (3); Second, the acquisition of the template frame. When the camera captures the image in a static state, that is, the camera position remains unchanged and the blades do not rotate, the current frame is intercepted as the template frame for calculating the cosine similarity with other frames of the video; Third, the online counting algorithm is initialized. After intercepting the template frame, the leaves are rotated for a number of times (typically 2 to 3). The direction of rotation at this time is regarded as the initial direction. The cosine similarity between each frame and the template frame is obtained to obtain a set of one-dimensional signals with a length of 200 to 300 frames. The peak and valley values ​​of this signal are obtained using the AMPD algorithm to calculate the constraints of the leaf online counting algorithm, including the maximum value of the peak, the minimum value of the valley, the minimum peak-to-valley difference (set to 0.1 times the difference between the maximum peak and the minimum valley), and the initial value of the adaptive dynamic threshold (set to half the sum of the maximum peak and the minimum valley); Fourth, the blade online counting algorithm is implemented, and the constraints calculated in the initialization of the blade counting algorithm in the third point are used to determine whether the current frame is a peak or a valley. Half of the sum of the peak and valley values ​​each time is used as the binarization condition of the current signal, and the signal after cosine similarity calculation is binarized. The falling edge of the binarized signal is used as the counting condition. Whenever a falling edge comes, the count value is increased or decreased according to the rotation direction.

5. The method of claim 4 wherein the image information is fed into two parallel tasks, wherein in the second task, the online counting algorithm is initialized, and the peak and valley values ​​of the signal are obtained by using the AMPD algorithm to calculate the initialization conditions, wherein: The AMPD algorithm is an automatic multi-scale peak-finding algorithm that uses a multi-scale sliding window to find the local maximum and calculate the peak value. It calculates the valley value by flipping the waveform data, and uses these peak and valley values ​​to calculate the initialization conditions of the online counting algorithm (maximum peak value, minimum valley value, minimum peak-to-valley interval, and initial value of the adaptive dynamic threshold).

6. As claimed in claim 4, the image information is sent to two parallel tasks, in the second task, the leaf online counting algorithm is implemented, and the constraint conditions obtained by initializing the leaf counting algorithm are used to determine whether the current frame is a peak value or a valley value, characterized in that: Only when the local maximum value is greater than the current adaptive dynamic threshold and the difference between the current peak value and the valley value is greater than the minimum peak-valley difference, the value at this time is regarded as the current peak value; Only when the local minimum value is smaller than the current adaptive dynamic threshold and the difference between the current peak value and the valley value is greater than the minimum peak-valley difference, the value at this time is regarded as the valley value.