An engine blade detection system and method based on image feature similarity matching

Abstract

The present invention provides an engine blade detection system and method based on image feature similarity matching, which belongs to the field of engine detection technology. The detection system includes a borescope video acquisition unit for acquiring video frame images of rotating engine blades; a blade zero position positioning unit for marking a zero point in the blade; an image extraction unit for extracting the video frame images frame by frame into pictures, numbering each picture, and marking the zero point picture with zero position; a counting unit for determining the number of blades based on the similarity of the pictures, and the total number of similarity peaks is the number of blades captured by the video frame image. The present invention only needs to use the borescope video frame image and the similarity of the coordinate-based video frame image of the blade rotation process to accurately count the number of detected blades, and can also combine with the defect judgment unit to perform defect judgment, killing two birds with one stone, thereby greatly reducing the error rate and missed detection rate of manual counting.

Description

Technical Field

[0001] The present invention belongs to the technical field of engine detection, and in particular relates to an engine blade detection system and method based on image feature similarity matching. Background Art

[0002] In modern industry, especially in the aviation and automotive industries, engine performance and reliability are crucial. As core components, the structural integrity and functionality of engine blades are directly related to engine efficiency and safe operation. Therefore, accurate and efficient inspection of engine blades is a key step in ensuring engine quality.

[0003] During the blade inspection process, in order to ensure that all blades are detected and to prevent duplicate inspections, the blades being inspected need to be counted. When the number of blades is the same as the total number of blades in the current engine model, it can be determined that a complete inspection has been carried out. However, in the prior art, blade counting is mostly done manually or with auxiliary sensors. Due to the similar external features of engine blades, when the number of blades is large, problems such as missed detection and calculation errors are prone to occur, and the inspection time is long. With the rapid development of image processing technology, image-based blade detection technology has gradually increased. Therefore, if it is possible to synchronously count blades based on image information directly during the blade image acquisition process, the counting accuracy and detection efficiency can be greatly improved. The counting results can also be combined to review blade defect detection, thereby improving the practicality of blade detection. Summary of the Invention

[0004] The purpose of the present invention is to provide an engine blade detection system and method based on image feature similarity matching, which uses borescope video frame images to accurately count the number of detected blades to reduce the error rate and missed detection rate of blade detection.

[0005] To achieve the above objectives, in a first aspect, the present invention provides an engine blade detection system based on image feature similarity matching, comprising:

[0006] A borescope video acquisition unit, used to acquire video frame images of rotating engine blades;

[0007] A blade zero position positioning unit, used for marking a zero point in the blade;

[0008] An image extraction unit, configured to extract the video frame images frame by frame into pictures, number each of the pictures, and mark the zero position of the picture;

[0009] The counting unit is used to determine the number of leaves according to the similarity of the pictures, and the total number of similarity peaks is the number of leaves collected by the video frame image.

[0010] Furthermore, the counting unit includes a feature extraction module and a feature similarity calculation module; the feature extraction module is used to extract feature information of the image, and the feature information is a pixel value carrying coordinate information; the feature similarity calculation module calculates the similarity of the image based on the feature information.

[0011] Furthermore, the feature similarity calculation module first determines a template image for similarity determination from the feature information of the image, and then calculates the similarity between the current frame image and the template image according to the following formula:

[0012]

[0013] P(i, j) represents the pixel value of the feature image of the template image at the coordinate (i, j), and C(i, j) represents the pixel value of the feature image of the current frame image at the coordinate (i, j). Represents the average value of all pixel values ​​in the template image. Represents the average value of all pixel values ​​in the current frame; rows represents the row size of the image, and cols represents the column size of the image.

[0014] Furthermore, it also includes a blade shifting device and a data output unit, wherein the blade shifting device is used to shift the engine blades so that the borescope video acquisition unit can acquire video frame images of the engine blades; the zero mark is a physical mark or an electronic mark;

[0015] The data output unit is configured to output information about the engine blades, wherein the information about the blades includes the number of identified blades, a picture corresponding to each blade, and a graph showing similarity and the number of image frames;

[0016] The data output unit also constructs an information index database of the engine blades according to the processing result of the counting unit.

[0017] Furthermore, the engine blade detection system also includes a defect judgment unit for judging whether the blade has defects based on the image.

[0018] The present invention also provides an engine blade detection method based on image feature similarity matching, comprising the following steps:

[0019] S1, collecting a video frame image of a rotating engine blade; and marking a zero point in the blade;

[0020] S2, extracting the video frame images frame by frame to obtain pictures, numbering each of the pictures in sequence, and marking the zero position of the picture;

[0021] S3, calculating the similarity of the pictures, where the total number of similarity peaks is the total number of leaves captured by the video frame images.

[0022] Furthermore, in step S1, the rotation of the engine blades is driven by their own power, or the blades are moved by an external blade moving device; the rotation direction of the blades is fixed; the acquisition frequency of the borescope video acquisition unit is consistent with the frequency at which the blades appear in the borescope camera.

[0023] Furthermore, in step S3, feature information of the image is first extracted, where the feature information is pixel values ​​carrying coordinate information. Then, a template image for similarity determination is determined from the feature information of the image, and the similarity between the current frame image and the template image is calculated according to the following formula:

[0024]

[0025] Among them, P(i, j) represents the pixel value of the feature image of the template image at the coordinate (i, j), and C(i, j) represents the pixel value of the feature image of the current frame image at the coordinate (i, j). Represents the average value of all pixel values ​​in the template image. Represents the average value of all pixel values ​​in the current frame; rows represents the row size of the image, and cols represents the column size of the image.

[0026] Furthermore, the first extracted picture is used as the template picture, and the similarity between each picture and the template picture is calculated in turn. Then, a scatter plot is drawn with the number of frames as the horizontal axis and the similarity as the vertical axis. The similarity curve is fitted by the fitting algorithm. The number of peaks in the similarity curve is the total number of leaves.

[0027] Furthermore, it also includes: judging whether the blade has defects based on the extracted image; when defects exist, first preliminarily judging the corresponding collection position of the blade based on the zero mark and the similarity curve diagram, and then rotating or moving the blade to the corresponding collection position for repeated collection and judgment.

[0028] In general, the above technical solutions conceived by the present invention have the following technical advantages compared with the existing technology:

[0029] 1) The engine blade detection system and method based on image feature similarity matching provided by this invention uses only borescope video frames and accurately counts the number of detected blades based on the similarity of video frames during blade rotation. This solves the counting errors and low counting efficiency problems of existing technologies, significantly reducing the error rate and missed detection rate of manual counting. Furthermore, the zero mark facilitates rapid blade location and retrieval.

[0030] 2) The present invention can output a curve graph of similarity and image frame number, which can not only obtain the number of blades based on the number of peaks, but also intuitively judge the rotation of the blades during the acquisition process, facilitate the rapid positioning of the blade acquisition position corresponding to the corresponding image, and facilitate rapid review and tracking.

[0031] 3) The engine blade detection method based on image feature similarity matching provided by the present invention has important application value and broad market prospects in the field of engine detection. It can be widely used in the engine manufacturing and maintenance fields of aviation, automobile, energy and other industries, and can improve the efficiency and quality of detection work, effectively improve the reliability and safety of the engine, and reduce maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of an engine blade detection method based on image feature similarity matching provided by the present invention;

[0033] Figure 2 It is a similarity curve graph showing the number of leaves recognized as a function of the number of image frames;

[0034] Figure 3 Schematic diagram of the structure of multi-scale features. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0036] See also Figure 1 and 2 The present invention provides an engine blade detection system based on image feature similarity matching, comprising:

[0037] A borescope video acquisition unit, used to acquire video frame images of rotating engine blades;

[0038] A blade zero position positioning unit, used for marking a zero point in the blade;

[0039] An image extraction unit, configured to extract the video frame images frame by frame into pictures, number each of the pictures, and mark the zero position of the picture;

[0040] a counting unit, configured to determine the number of leaves according to the similarity of the pictures, wherein the total number of similarity peaks is the number of leaves captured by the video frame image;

[0041] A data output unit is used to output the information of the blade.

[0042] The borescope video acquisition unit remains in a fixed position during the acquisition process, with only the engine blades rotating. Once the blades have completed one turn, the acquisition is complete. The borescope video acquisition unit continuously acquires images during the blade rotation process to obtain video frame images. Due to the structural characteristics of the engine blades, each blade will inevitably appear in the same position when it rotates into the acquisition field of view. Therefore, the number of blades can be quickly determined using image similarity. The image extraction unit extracts the video frame images frame by frame as images, and numbers each image in sequence (image frame number) so that the image similarity can be plotted sequentially.

[0043] The blade zero-position positioning unit provides an absolute zero position for blade counting. This is linked to the blades via an auxiliary toggle mechanism, which sets a mark at the engine blade transmission interface as the zero position. Before each test, the toggle mechanism returns the blade mechanism to the zero position, and the blade captured in the image at this point is the zero-position blade. Each zero-position blade is zero-marked, and combined with a similarity graph, the number of blades captured between the zero and intermediate positions can be quickly determined, facilitating rapid positioning and backtracking.

[0044] The zero mark is a physical mark (such as a scale or a marking point) or an electronic mark (such as a sensor signal or an electronic tag).

[0045] Specifically, the counting unit includes a feature extraction module and a feature similarity calculation module. The feature extraction module is used to extract the image's feature information, which is the pixel value that carries the coordinate information. Since the borescope video acquisition unit is fixed, its acquisition center point can be used as the center of the coordinate axis to establish a coordinate system, thereby obtaining the coordinate information of each pixel in each image. As the current leaf gradually rotates out of the acquisition field of view and the next leaf gradually enters the field of view, the similarity is the same if each leaf is in the same position in the field of view.

[0046] In particular, the feature extraction network of the feature extraction module obtains a low-dimensional feature image by performing deep learning operations on the image to reduce the amount of similarity calculation.

[0047] Next, the feature similarity calculation module calculates the similarity of the images based on the feature information. The feature similarity calculation module first determines a template image for similarity determination from the feature information of the image and then calculates the similarity between the current frame image and the template image according to the following formula:

[0048]

[0049] P(i, j) represents the pixel value of the feature image of the template image at the coordinate (i, j), and C(i, j) represents the pixel value of the feature image of the current frame image at the coordinate (i, j). Represents the average value of all pixel values ​​in the template image, Represents the average value of all pixel values ​​in the current frame; rows represents the row size of the image, and cols represents the column size of the image.

[0050] Through the image feature extraction method based on neural networks (ResNet, etc.), a picture is selected in advance as a template image and a low-dimensional feature map is extracted using the neural network. Feature extraction is performed on each frame of the video to obtain the corresponding feature map. By calculating the similarity of the two feature maps, the similarity of each frame image is counted to obtain a scatter plot, and a curve is fitted. The number of peaks is the number of leaves.

[0051] Usually the first image captured can be used as a template image. When the previous leaf and the next leaf enter the same capture field of view, the similarity between the current frame image and the template image decreases first and then increases. When the similarity is the maximum, it means that the second leaf has been captured. Figure 2 , a curve graph of similarity and image frame number can be obtained. Based on this graph, the number of blades can be obtained based on the number of peaks, and the rotation speed of the blades during the acquisition process can be intuitively judged. When the acquisition frequency of the borescope video acquisition unit remains unchanged, if the blade rotation speed is slow, the wavelength of the obtained similarity waveform is larger. When the blade rotation speed is accelerated, the wavelength of the waveform decreases. Therefore, its rotation speed can be preliminarily estimated. Furthermore, based on the similarity change law, the approximate rotation position of the blade corresponding to a certain picture can also be judged. When performing defect detection, when a blade corresponding to a certain picture is found to have defects (especially when the defects are more obvious only in this orientation), the acquisition position can be preliminarily judged based on this similarity graph, and the blade can be quickly rotated to an approximately the same position, which is convenient for rapid review and tracking.

[0052] Specifically, the engine blade inspection system also includes a blade shifting device for shifting the engine blades so that the borescope video acquisition unit can capture video frames of the engine blades. The blade shifting device facilitates mechanical control of blade rotation, primarily to address situations where the engine fails to rotate automatically or rotates too quickly.

[0053] Preferably, the engine blade inspection system also includes a defect detection unit for determining whether defects exist based on the image. For example, defect identification can be performed using machine learning or deep learning methods, such as convolutional neural networks (CNNs) for image classification, or deep learning-based object detection algorithms such as Faster R-CNN and YOLO.

[0054] The blade information output by the data output unit includes the number of identified blades, the picture corresponding to each blade, a curve chart of similarity and image frame number, and the defect judgment result; in particular, the data output unit can also construct an information index database of engine blades based on the processing results of the counting unit.

[0055] See also Figure 1 The present invention also provides an engine blade detection method based on image feature similarity matching, comprising the following steps:

[0056] S1, collecting video frame images of rotating engine blades;

[0057] S2, extracting the video frame images frame by frame to obtain pictures, and numbering each of the pictures in sequence;

[0058] S3, calculating the similarity of the pictures, where the total number of pictures with the same similarity is the total number of leaves captured by the video frame image;

[0059] S4, outputting the information of the blades. When the total number of identified blades is the same as the number of engine blades, it indicates that the detection and counting of all blades are completed.

[0060] Specifically, in combination with the detection system, the engine blade detection method includes the following steps:

[0061] S1, collecting a video frame image of a rotating engine blade through a borescope video acquisition unit, and marking a zero point on the blade;

[0062] S2, extracting the video frame images frame by frame to obtain pictures through an image extraction unit, and numbering each of the pictures in sequence; and marking the zero position of the picture at the zero position;

[0063] S3, first selecting a template image from the image by a counting unit, the template image being derived from the first leaf captured in the video frame image; then calculating the similarity between the current frame image and the template image; if the similarities are the same, it indicates that the current frame image is derived from another leaf; and then counting the total number of images having the same similarity to the template image to obtain the number of leaves captured in the video frame image;

[0064] S4, the data output unit outputs the information of the blades. When the number of identified blades is the same as the number of engine blades, it indicates that the detection and counting of all blades are completed.

[0065] In step S1, the rotation of the engine blade is driven by its own power, or the blade is moved by an external blade moving device; the detection method of the present invention is preferably applied to the situation where the rotation direction of the blade is fixed, and when backtracking is required, it can continue to rotate in the same direction until the next time it reaches the target blade.

[0066] Preferably, the step length of each movement of the blade moving device is adjusted to be consistent with the number of blades appearing in the borescope camera. For example, the borescope video acquisition unit captures images, calculates the frequency of blade acquisition, and adjusts the step length of the moving device to be consistent with the frequency of blade appearance. This ensures that a video frame image is captured each time the blade rotates to the same position.

[0067] In step S3, the feature extraction module first extracts the feature information of the image, which is the pixel value carrying the coordinate information. Then, the feature similarity calculation module determines a template image for similarity determination from the feature information of the image, and then calculates the similarity between the current frame image and the template image according to the following formula:

[0068]

[0069] P(i, j) represents the pixel value of the feature image of the template image at the coordinate (i, j), and C(i, j) represents the pixel value of the feature image of the current frame image at the coordinate (i, j). Represents the average value of all pixel values ​​in the template image. Represents the average value of all pixel values ​​in the current frame; rows represents the row size of the image, and cols represents the column size of the image.

[0070] In particular, the first extracted image is used as the template image, and the similarity between each image and the template image is calculated in turn. Then, a scatter plot is drawn with the number of image frames as the horizontal axis and the similarity as the vertical axis. The similarity curve is fitted by the fitting algorithm, and the number of peaks in the similarity curve is the number of leaves.

[0071] In some preferred embodiments, the feature extraction module performs deep learning on the image to generate a low-dimensional feature image, reducing the computational complexity of similarity. A deep learning feature extraction network, such as ResNet, is used to extract a feature map from the template image and save it for future use. The subsequent detection process calculates the similarity between the feature map of the current image and the feature map of the template image frame by frame. A similarity peak is inevitable when the leaves rotate to the same position.

[0072] The present invention also uses a defect determination unit to determine whether the extracted images contain defects. In practical applications, the blades are typically slowly moved, and during the acquisition process, appropriate image processing techniques are used to determine whether the captured images contain defects. Once all blades have been acquired, the corresponding images are identified in the similarity curve according to their numbers. The zero mark and similarity variation patterns are used to preliminarily determine the corresponding blade position. The blades are then quickly moved to the corresponding position, and the images are re-acquired and rechecked.

[0073] In particular, multi-scale analysis can be used to calculate similarity. This approach captures features at different levels of the image at different scales, from coarse to fine, and assigns different weights when calculating feature similarity. This mitigates the impact of obvious defects on similarity while also avoiding the low similarity resulting from calculating a single feature map for defective images. By calculating similarity between feature images at different scales and the feature maps at the corresponding scales of the standard template image, a more comprehensive similarity measurement is achieved for features of different sizes.

[0074] like Figure 3 As shown in the figure, when the defect judgment unit detects a defect, it calculates similarity through multi-scale partitioning. For example, the feature values ​​are calculated based on the three-layer feature map. The similarities R1, R2, and R3 are calculated for the three feature scales (128, 128), (56, 56), and (28, 28), respectively, with weights of 0.7, 0.2, and 0.1, respectively. The total similarity is R = 0.7 × R1 + 0.2 × R2 + 0.1 × R3. When the scale is (128, 128), the rows and cols in the above similarity formula are 128. In the figure, W and H represent the row and column dimensions.

[0075] The present invention is particularly suitable for inspecting blades inside engines. By using a blade-moving device in conjunction with a borescope camera to detect whether there are any problems with the blades, it can comprehensively and accurately detect the internal conditions of the object to be inspected. Combined with a counting unit, it can quickly and accurately determine whether the acquisition is complete. Linking the similarity curve graph with the defect judgment unit facilitates the review and tracking of defective blades. The detection method of the present invention not only improves detection accuracy but can also be performed without disassembling the engine, significantly reducing the time and cost of the inspection process. It also reduces the complexity and error rate of manual operations, further improving the reliability of the inspection.

[0076] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An engine blade detection system based on image feature similarity matching, characterized in that: include: A borescope video acquisition unit, used to acquire video frame images of rotating engine blades; A blade zero position positioning unit, used for marking a zero point in the blade; An image extraction unit, configured to extract the video frame images frame by frame into pictures, number each of the pictures, and mark the zero position of the picture; a counting unit, configured to determine the number of leaves according to the similarity of the pictures, wherein the total number of similarity peaks is the number of leaves captured by the video frame image; Specifically, the first extracted picture is used as the template picture, and the similarity between each picture and the template picture is calculated in turn. Then, a scatter plot is drawn with the number of frames as the horizontal axis and the similarity as the vertical axis. The similarity curve is fitted by a fitting algorithm. The number of peaks in the similarity curve is the total number of leaves.

2. The engine blade detection system based on image feature similarity matching according to claim 1, characterized in that: The counting unit includes a feature extraction module and a feature similarity calculation module; the feature extraction module is used to extract feature information of the image, where the feature information is a pixel value carrying coordinate information; the feature similarity calculation module calculates the similarity of the images based on the feature information.

3. The engine blade detection system based on image feature similarity matching according to claim 2, characterized in that: The feature similarity calculation module first determines a template image for similarity determination from the feature information of the image, and then calculates the similarity between the current frame image and the template image according to the following formula: P(i, j) represents the pixel value of the feature image of the template image at the coordinate (i, j), and C(i, j) represents the pixel value of the feature image of the current frame image at the coordinate (i, j). Represents the average value of all pixel values ​​in the template image. Represents the average value of all pixel values ​​in the current frame; rows represents the row size of the image, and cols represents the column size of the image.

4. The engine blade detection system based on image feature similarity matching according to claim 1, characterized in that: It also includes a blade shifting device and a data output unit, wherein the blade shifting device is used to shift the engine blades so that the borescope video acquisition unit can acquire video frame images of the engine blades; the zero mark is a physical mark or an electronic mark; The data output unit is configured to output information about the engine blades, wherein the information about the blades includes the number of identified blades, a picture corresponding to each blade, and a graph showing similarity and the number of image frames; The data output unit also constructs an information index database of the engine blades according to the processing result of the counting unit.

5. The engine blade detection system based on image feature similarity matching according to claim 1, characterized in that: The engine blade detection system further includes a defect judgment unit configured to judge whether the blade has defects based on the image.

6. An engine blade detection method based on image feature similarity matching, characterized in that: The steps include: S1, collects a video frame image of a rotating engine blade and marks a zero point in the blade; S2, extracting the video frame images frame by frame to obtain pictures, numbering each of the pictures in sequence, and marking the zero position of the picture; S3, calculating the similarity of the pictures, and the total number of similarity peaks is the total number of leaves captured by the video frame image; specifically: taking the first extracted picture as the template picture, calculating the similarity of each picture with the template picture in turn, and then drawing a scatter plot with the number of frames as the horizontal axis and the similarity as the vertical axis, fitting a similarity curve graph through a fitting algorithm, and the number of peaks in the similarity curve graph is the total number of the leaves.

7. The engine blade detection method based on image feature similarity matching according to claim 6, characterized in that: In step S1, the rotation of the engine blades is driven by their own power, or by an external blade shifting device; the rotation direction of the blades is fixed; The acquisition frequency of the borescope video acquisition unit is consistent with the frequency at which the blades appear in the borescope camera.

8. The engine blade detection method based on image feature similarity matching according to claim 6, characterized in that: In step S3, the feature information of the image is first extracted. The feature information is the pixel value carrying the coordinate information. Then, a template image for similarity determination is determined from the feature information of the image. The similarity between the current frame image and the template image is calculated according to the following formula: Among them, P(i,j) represents the pixel value of the feature image of the template image at the coordinate (i, j), and C(i,j) represents the pixel value of the feature image of the current frame image at the coordinate (i, j). Represents the average value of all pixel values ​​in the template image. Represents the average value of all pixel values ​​in the current frame; rows represents the row size of the image, and cols represents the column size of the image.

9. The engine blade detection method based on image feature similarity matching according to claim 8, characterized in that: Also includes: Determine whether the blade has defects based on the extracted image. If defects exist, first preliminarily determine the corresponding acquisition position of the blade based on the zero mark and the similarity curve graph, then rotate or move the blade to the corresponding acquisition position for repeated acquisition and judgment.

Images