Visual counting method for aero-engine rotor blades

Abstract

The invention provides a visual counting method for aero-engine rotor blades. According to the method, automatic counting of the aero-engine rotor blades can be efficiently and accurately achieved. According to the specific scheme, firstly, an industrial endoscope is used for collecting image data of the blade through an endoscopic hole of the aero-engine; and meanwhile, the rotation of the blades is sensed by utilizing the offset of the feature points, if the blades are in a rotating state, the blades are detected in real time through a target detection algorithm, meanwhile, the blade states are managed and counted by adopting a finite-state machine, and the counting of the blades is realized by defining a transfer rule between different states. The method is suitable for counting the blades during visual surface defect detection of the aero-engine rotor blade endoscope, and has the advantages of rapidness and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of automated detection, and particularly to an automatic counting method for aero-engine rotor blades based on visual information and a finite state machine. This method is applied to the precise counting and position tracking of aero-engine rotor blades, and is suitable for the automated detection, fault diagnosis, and maintenance system of aero-engines through industrial endoscopes, which can effectively improve the accuracy and real-time performance of blade counting. Background Art

[0002] With the continuous development of the aviation industry, endoscope inspection has become an indispensable and important means in aero-engine maintenance. Through the endoscope, technicians can enter the interior of the engine, observe and detect the health status of key components such as rotor blades in real time, so as to achieve early fault diagnosis and preventive maintenance.

[0003] Endoscope inspection not only reduces the time and cost of engine disassembly, but also avoids potential risks brought by traditional methods. However, with the continuous development of aero-engines towards high efficiency and high thrust-to-weight ratio, their structural complexity has increased significantly. With the increase in the number of blades and the increasingly strict detection standards, the limitations of manual operation have become more prominent. In addition, due to the lack of automated means, operators must rely on the video sequence collected by the endoscope to manually count the aero-engine blades. This not only increases the workload, but is also easily affected by factors such as eye fatigue and distraction, resulting in problems such as insufficient maintenance detection accuracy, low efficiency, and missed blade detection, which have an adverse impact on subsequent maintenance decisions. In practical applications, the accurate counting of engine rotor blades remains an important challenge, especially in complex detection environments, and how to accurately count has become a technical problem. Summary of the Invention

[0004] The present invention proposes a visual counting method for aero-engine rotor blades, which can achieve precise counting of aero-engine rotor blades.

[0005] The technical solution adopted by the present invention to achieve the above object is: a visual counting method for aero-engine rotor blades, comprising the following steps:

[0006] Step 1: Read the video stream through an industrial video endoscope and initialize the region of interest for the first frame image;

[0007] Step 2: Extract feature points for each frame image, use the feature point matching algorithm to track the displacement of feature points between two consecutive frames, and sense the rotation amplitude and direction of the blade;

[0008] Step 3: For the image with the rotation degree of the blade greater than the threshold, perform image enhancement through histogram equalization, and then detect the blade edge;

[0009] Step 4: Extract the position feature points of the blade prediction box from the detection results output in Step 3, then check the counting conditions, trigger the counting, and update the count value.

[0010] In Step 1, read the video stream through an industrial video endoscope and initialize the region of interest for the first frame image, specifically as follows:

[0011] Read the video stream through an industrial video endoscope to obtain consecutive images of the blade;

[0012] Initialize the region of interest (ROI) for the first frame image so that the adjacent edges of two adjacent blades have a constraining effect on two adjacent counting ROIs;

[0013] Initialize the rotation direction of the blade according to the first frame image: if it rotates horizontally, initialize the rotation factor D = 0; if it rotates vertically, initialize the direction factor D = 1.

[0014] In Step 2, extract feature points from each frame image, use the feature point matching algorithm to track the displacement of feature points between two consecutive frames, and sense the rotation amplitude and direction of the blade. The specific steps are as follows:

[0015] For each frame image read, use the FAST algorithm to extract feature points;

[0016] Perform quadtree equalization on these feature points to improve the distribution uniformity of the feature points;

[0017] Use the feature point matching algorithm to track the displacement of feature points in two consecutive frame images;

[0018] Judge the rotation amplitude and angle of the blade according to the mean values of the displacement and direction of the feature points.

[0019] The judgment of the rotation amplitude and angle of the blade according to the mean values of the displacement and direction of the feature points includes the following steps:

[0020] 1) Calculate the blade rotation offset:

[0021] For each pair of feature points Calculate the deviation of its spatial position, that is, the displacement on the image coordinates: the coordinates of the matching point in the current frame are the coordinates of the matching point in the previous frame are Then the position deviation

[0022] According to the calculated position deviation ΔP of the feature points i , obtain the average position deviation of the feature points i and j represent the feature point indices, and N represents the total number of pairs of feature points;

[0023] For each pair of feature points, calculate the displacement vector v i =(v xi ,v yi ), and its deviation ΔP i and the displacement direction are calculated through θ i =atan(v yi ,v xi ); the rotation direction where θ avg is the average rotation angle of all pairs of feature points;

[0024] 2) Rotation direction perception:

[0025] If ΔP avg is greater than the set threshold, the rotation direction of the blade is judged through the rotation factor D and the cosine value cosθ avg and the sine value sinθ avg of the average angle:

[0026] When the rotation factor D = 0, if cosθ avg > 0, it is judged that the blade is right-handed; if cosθ avg < 0, it is judged to be left-handed;

[0027] When the rotation factor D = 1, if sinθ avg > 0, it is judged to be upward rotation; if sinθ avg < 0, it is judged to be downward rotation.

[0028] The step 3, for an image with a blade rotation degree greater than the threshold, performs progressive image enhancement through histogram equalization, and then detects the blade edge, specifically as follows:

[0029] 3.1) Histogram equalization: Process the input image I(x,y) to obtain the grayscale histogram H(k) of the image; based on the grayscale histogram H(k), calculate the cumulative distribution function C(k) of the image, and perform normalization processing on the cumulative distribution function C(k) to obtain the normalized cumulative distribution function C′(k); through the normalized cumulative distribution function C′(k), map each pixel value of the input image to a new grayscale value to generate the enhanced image I′(x,y);

[0030] 3.2) Image feature extraction:

[0031] Adjust the size of the enhanced image I′(x,y) to make it conform to the input of GEResNet, and perform feature extraction to output the feature map F:

[0032]

[0033] where, It represents extracting features through GEResNet. x and y respectively represent the position indices of pixels in the input image I′, and x' and y' respectively represent the position indices of pixels in the output feature map;

[0034] The GEResNet replaces the second 3x3 convolution in the residual structure of the original Resnet34 from stage 1 to stage 4 with a GS-ECA module. Among them, the GS-ECA module is composed of Ghost convolution and ECA attention mechanism and performs the following steps:

[0035] Ghost convolution is expressed as: F ghost =[Conv(F in ), A·Conv(F in )]; where F ghost is the redundant feature map, Conv is the convolution operation, A is the linear transformation matrix, and F in is the input feature map;

[0036] The ECA attention mechanism obtains the channel average value through global average pooling and generates an adaptive channel attention coefficient F att =α c ·F in ; where α c is the channel attention coefficient;

[0037] Finally, the GS-ECA module fuses the two feature maps through element-wise multiplication to obtain the final output feature map F out :

[0038] F out =F ghost ⊙F att ;

[0039] 3.3) Hybrid encoder processing: The feature map F(x', y') extracted by GEResNet is processed by the hybrid encoder to generate the final hybrid feature map Z to enhance the local and global information of the features;

[0040] 3.4) Decoder processing: The input feature map Z is processed by the decoder to generate the position of the target bounding box and the class prediction;

[0041] 3.5) Query selector processing: After obtaining the candidate box positions and the predicted classes output by the decoder, the candidate boxes are screened, and the redundant boxes are removed through the non-maximum suppression algorithm, and finally the optimal bounding box and the predicted class are output.

[0042] Step 4: Extract the position feature points of the blade prediction box from the detection results output in Step 3, then check the counting conditions, trigger the counting, and update the count value. The specific steps are as follows:

[0043] Step 4-1: Position feature point extraction: After obtaining the optimal bounding box and the predicted category, take the center point of each blade bounding box as the edge position feature point;

[0044] Step 4-2: Blade counter update: By taking the blade position feature points as the input of the spatio-temporal correlation counting model, the counter is updated through the spatio-temporal correlation counting model. Specifically as follows:

[0045] The spatio-temporal correlation counting model is represented as a quadruple M = (S, Σ, δ, s 0 ), where S is a finite state set, Σ is an event set, δ is a state transition function, and s 0 is the initial state, and s 0 ∈S;

[0046] The blade edge position feature points in the i-th frame are defined as the state set, and the blade edge position feature points in the (i + 1)-th frame are defined as the event set; the spatio-temporal correlation counting model makes effective state transitions according to the states satisfied by the blade edge position feature points in the i-th frame and the events corresponding to the blade edge position feature points in the (i + 1)-th frame; during a specific state transition process, the blade counting action is triggered to update the counter.

[0047] In Step 4-2, when using the spatio-temporal correlation model for aero-engine blade counting, the distributions of the blade edge position feature points in the i-th frame in R0, R1, and R2 respectively represent three states S 0 , S 1 and S 2 ; in the (i + 1)-th frame, the distributions of R0, R1, and R2 respectively represent three events E 0 , E 1 and E 2 ; A 1 and A 2 are the forward counting action and the reverse counting action that need to be triggered during the specified state transition process; when the event E 2 triggers the state transition of state S 2 , the action A 2 is executed; when the event E 1 triggers the state transition of state S 1 , the action A 1 is executed.

[0048] An aero-engine rotor blade vision counting system includes:

[0049] An image initialization module for initializing the region of interest (ROI) of the first frame of the video stream read by an industrial video endoscope.

[0050] A blade rotation amplitude and direction perception module for extracting feature points from each frame of the image, using the feature point matching algorithm to track the displacement of feature points between two consecutive frames, and perceiving the rotation amplitude and direction of the blade.

[0051] A blade edge detection module for images where the rotation degree of the blade is greater than the threshold, performing image enhancement through histogram equalization, and then detecting the blade edge.

[0052] A counting module for extracting the position feature points of the blade prediction box from the output detection results, then checking the counting conditions, triggering the count, and updating the count value.

[0053] The present invention has the following beneficial effects and advantages:

[0054] 1. The method of the present invention proposes a blade rotation perception method. Through the feature point extraction and matching algorithm, it can perceive the rotation direction and rotation amount of the blade in real time, and accurately judge the rotation state of the blade by combining the rotation offset and angle analysis.

[0055] 2. The method of the present invention proposes a spatio-temporal correlation blade counting model. By capturing the spatio-temporal position information of the position feature points of the blade edge, and using the current frame position feature points as events to trigger state transitions, the counting action is triggered during specific state transitions to achieve accurate blade counting. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flowchart of a visual counting method for the rotor blades of an aeroengine according to the present invention;

[0057] Figure 2 It is the architecture diagram of GEResNet;

[0058] Figure 3 It is a schematic diagram of ROI selection when the blade rotates in different directions; (a) Horizontal rotation ROI area; (b) Vertical rotation ROI area;

[0059] Figure 4 It is the state transition diagram of the spatio-temporal correlation state machine model. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following describes the specific implementation methods of the present invention in detail with reference to the accompanying drawings. Many specific details are set forth in the following description to facilitate a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the spirit of the invention. Therefore, the present invention is not limited by the specific implementations disclosed below.

[0061] Unless otherwise defined, all technical and scientific terms used in the methods of the present invention have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in the description of the invention in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0062] The method of the present invention is implemented based on an industrial endoscope hardware system.

[0063] As Figure 1 shown, the present invention includes the following steps:

[0064] Step 1, video stream reading and initialization;

[0065] Step 2, blade rotation perception;

[0066] Step 3, image enhancement and blade edge detection;

[0067] Step 4, counter update.

[0068] The specific implementation process is as follows.

[0069] Step 1, the operator inserts the industrial endoscope probe into the endoscope hole of the aero-engine, collects the image of the aero-engine rotor blade through the endoscope probe, and initializes the region of interest (ROI) and the rotation direction of the blade. The specific steps are as follows:

[0070] Step 1-1, read the video stream through the industrial video endoscope to obtain consecutive frame images of the blade;

[0071] Step 1-2, initialize the region of interest (ROI) to ensure that the edges of two adjacent blades have a constraint effect on two adjacent counting ROIs, and initialize the rotation direction of the blade according to the image data. If it rotates horizontally, initialize the rotation factor D = 0; if it rotates vertically, initialize the direction factor D = 1.

[0072] Step 1-3, check whether the image is successfully loaded. If the image reading fails, the system will immediately terminate the counting process; if the image reading is successful, proceed to the subsequent processing steps;

[0073] Step 2: Extract feature points for each frame of the image, use the feature point matching algorithm to track the displacement of feature points between consecutive frames, and sense the rotation amplitude and direction. The specific steps are as follows:

[0074] Step 2-1: Use the FAST algorithm to extract feature points for each frame of the image. For each pixel point P(x, y) in the image, considering its surrounding circular neighborhood N(P) (centered at P with a radius of r), the judgment condition of FAST is: If the difference between the brightness of this pixel and the brightness of the surrounding pixels is greater than the set threshold δ, then this point is considered a feature point. The formula is as follows:

[0075]

[0076] where I(P) is the brightness of pixel P, N(P) is the neighborhood centered at P, I(i) is the brightness of neighborhood pixel i, and δ is the threshold.

[0077] Step 2-2: Use the quadtree algorithm to equalize and match the feature points. The quadtree is a spatial partitioning method that can divide the image into multiple sub-regions, making the number of feature points in each region approximately uniform. Let the image region A 0 initially be the entire image region. The partitioning condition of the quadtree is: If the number of feature points in a region exceeds the set threshold N max , then divide this region; if the number of feature points in this region is lower than the threshold N min , then stop partitioning. The quadtree partitioning formula is as follows:

[0078] Split(A) = True if points? in A > Nmax (1.2)

[0079] Merge(A) = True if points? in A < Nmin (1.3)

[0080] where N max and N min are the maximum threshold and minimum threshold of the number of feature points respectively.

[0081] Let the set of feature points in the current frame be the set of feature points in the previous frame be the descriptor be D(P), and the matching algorithm uses the Euclidean distance to measure the similarity between two feature point descriptors. The matching metric formula is as follows:

[0082]

[0083] where are the descriptors of the feature points in the current frame and the previous frame respectively .

[0084] Use the K-Nearest Neighbor (KNN) algorithm to find the most matching pairs of feature points:

[0085]

[0086] where ε is the matching threshold.

[0087] Step 2-4: Calculate the rotational offset of the blade. After the matching is completed, for each pair of feature points calculate the deviation of their spatial positions (i.e., the displacement on the image coordinates). Let the coordinates of the matching point in the current frame be and the coordinates of the matching point in the previous frame be then the position deviation ΔP i is:

[0088]

[0089] Based on the calculated position deviation ΔP of the feature points i , the average position deviation ΔP of the feature points can be obtained avg , ΔP avg represents the degree of rotation of the blade, as shown in the following formula:

[0090]

[0091] where: N represents the total number of pairs of feature points;

[0092] Generally, the rotation direction of the blade can be determined by the trend of the displacement vector of the feature points. Assume that for each pair of matching feature points, the displacement vector v i =(v xi , v yi ) is calculated, and its deviation ΔP i and the displacement direction (angle) can be calculated by the following formula:

[0093] θ i = atan(v yi , v xi )(1.8)

[0094] where atan is the arctangent function, which returns the angle between the vector (v xi , v yi ) and the x-axis. According to this angular information, if the displacement direction of most feature points is positive (clockwise), then the blade rotation direction is judged to be clockwise; if it is negative (counterclockwise), then the blade rotation direction is judged to be counterclockwise. The decision formula for the rotation direction is:

[0095]

[0096] where θ avgIt is the average rotation angle of all pairs of feature points.

[0097] Step 2-5: Calculate the rotation direction of the blade. By ΔP avg and θ avg the rotation direction and the magnitude of the rotation amount of the blade can be sensed. If ΔP avg is greater than the set threshold, the rotation direction of the blade is judged by the rotation factor D and the cosine value (cosθ avg ) and sine value (sinθ avg ) of the average angle. When the rotation factor D = 0, if cosθ avg > 0, it is judged that the blade rotates clockwise; if cosθ avg < 0, it is judged to rotate counterclockwise. When the rotation factor D = 1, if sinθ avg > 0, it is judged to rotate upward; if sinθ avg < 0, it is judged to rotate downward.

[0098] Step 3: Perform histogram equalization on the image to enhance the contrast, and then detect the blade edge through the object detection model. The specific steps are as follows:

[0099] Step 3-1: Apply histogram equalization technology to enhance the input image obtained in Step 1 to improve the contrast and recognition of the image, thereby improving the subsequent image analysis effect. The specific steps are as follows:

[0100] First, process the input image I(x, y) by calculating the frequency distribution of each gray level k ∈ [0, L-1] in the image, that is, the gray histogram H(k) of the image, which is defined as:

[0101]

[0102] where M and N are the height and width of the image, and δ(·) is the indicator function, representing the number of pixels with a gray value of k.

[0103] Second, based on the calculated gray histogram H(k), calculate the cumulative distribution function (CDF) of the image, expressed as:

[0104]

[0105] where C(k) represents the number of pixels with a gray value less than or equal to k.

[0106] Further, normalize the cumulative distribution function C(k) and map its value to the range of [0, L-1] to obtain the normalized cumulative distribution function C′(k), and its calculation formula is:

[0107]

[0108] Among them, C min is the minimum value in the cumulative distribution function C(k), and M·N is the total number of pixels in the image.

[0109] Through the normalized cumulative distribution function C′(k), each pixel value I(x, y) of the input image is mapped to a new gray value to generate the equalized image I′(x, y), and its calculation formula is:

[0110]

[0111] Among them, represents the floor operation to ensure that the gray value of the output image is an integer.

[0112] After completing the image enhancement, the result I′(x, y) of the image enhancement is output. This image has higher contrast and clarity, which is convenient for subsequent steps such as feature extraction and object detection.

[0113] Step 3-2, Image feature extraction. Resize the enhanced image I′(x, y) to make it conform to the input of GEResNet ( Figure 2 ), and perform feature extraction to output the feature map F. The process can be expressed as:

[0114]

[0115] Among them, represents extracting features through GEResNet. x and y respectively represent the position indexes of pixels in the input image I′, and x' and y' respectively represent the position indexes of pixels in the output feature map.

[0116] Specifically, the present invention proposes an innovative module based on Ghost convolution and ECA attention mechanism, named GS-ECA. This module aims to enhance the ability of image feature representation through efficient feature extraction and attention mechanism enhancement. In order to build a powerful feature extraction network, we cleverly integrate the GS-ECA module into the classic 34-layer residual network ResNet34, thus obtaining a new GEResNet feature extraction network ( Figure 2 ). Use GS-ECA to replace the second 3x3 convolution in the residual structure from stage 1 to stage 4 of the original Resnet34. Among them, the image Ghost convolution generates a small number of intrinsic feature maps and uses linear transformation to expand and generate redundant feature maps, thereby reducing the computational complexity. The process can be expressed as:

[0117] F ghost =[Conv(F in ),A·Conv(F in )](1.15)

[0118] Among them, Conv is the convolution operation, A is the linear transformation matrix, and F in is the input feature map.

[0119] The ECA attention mechanism obtains the channel average value through global average pooling and generates an adaptive channel attention coefficient through one-dimensional convolution:

[0120] F att = α c ·F in (1.16)

[0121] Among them, α c is the channel attention coefficient.

[0122] Finally, the GS-ECA module fuses the two feature maps through element-wise multiplication to obtain the final output feature map:

[0123] F out = F ghost ⊙ F att (1.17)

[0124] This method effectively enhances the ability to extract leaf features, thereby improving the detection accuracy and efficiency in complex situations such as leaf surface reflection and attitude changes.

[0125] Step 3-3, Hybrid Encoder Processing. The input feature map is processed by the hybrid encoder to enhance the local and global information of the features. The hybrid encoder combines the convolution operation and the self-attention mechanism to adapt to different scales, attitudes, and background changes, and improves the robustness of the network in complex scenarios. Specifically, the hybrid encoder includes the following main steps:

[0126] First, the feature map F(x', y') extracted by GEResNet is mapped to a new space through three linear transformations of W Q (query matrix), W K (key matrix), and W V (value matrix) (corresponding to Query, Key, and Value respectively). Through these matrices, a linear transformation is performed on the feature map F(x', y') (abbreviation: F) to obtain the corresponding Q, K, V:

[0127] Q = F · W Q , K = F · W K , V = F · W V (1.18)

[0128] Among them, the shapes of Q, K, and V are [d, H×W], d is the dimension of the attention space, and H and W are the height and width of the input feature map respectively.

[0129] Based on the similarity between the Query and the Key, the dot product is usually used to calculate the attention score, and the calculated attention weights are used to weight the Value. When calculating the features at each position using the self-attention mechanism, the global context information of the entire input feature map is considered. The calculation formula of the self-attention mechanism is:

[0130]

[0131] where Q, K, and V are the query, key, and value respectively, and d k is the dimension of the key. This operation enables the features at each position to be weighted and adjusted according to the global information.

[0132] Then, the convolutional features and the attention features are multiplied element-wise to fuse the local and global information, generating the final hybrid feature map where C′ is the number of channels of the feature map after the convolutional operation. Specifically, it is expressed as:

[0133] Z = F ⊙ Attention(F)(1.20)

[0134] where ⊙ represents the element-wise multiplication operation.

[0135] This hybrid encoder improves the network's adaptability to different scales, poses, and background changes by fusing local convolutional features and global attention features, and enhances the robustness to problems such as specular reflection on the blade surface and pose changes.

[0136] Step 3 - 4: In the decoder stage, the input feature map Z is further processed by the decoder. The decoder interacts with the query vector through the self-attention mechanism to generate the bounding box position and class prediction of the target. The specific process is as follows:

[0137] First, the query vector interacts with the hybrid feature map Z through the self-attention mechanism to generate the updated query vector Q i :

[0138] Q i = Attention(Q i , Z)(1.21)

[0139] where Q i is the query vector, which is updated to a new query vector after the attention operation.

[0140] Then, for each query vector Q i , the decoder performs position regression and class classification through a fully connected layer:

[0141]

[0142]

[0143] Among them, is the predicted bounding box, is the predicted category. FC bbox represents the fully connected layer for bounding box regression, and FC cls represents the fully connected layer for classification, and Softmax represents the normalized exponential activation function.

[0144] Steps 3-5, After obtaining the candidate box positions and category predictions output by the decoder, the query selector processing step is performed for screening and optimization. First, the candidate boxes are screened according to the scores and category information of the predicted boxes. The score S i of the candidate box is calculated as follows:

[0145]

[0146] Among them, is the category probability, b gt is the ground truth box, is the intersection over union (IoU), which measures the overlapping degree between the predicted box and the ground truth box.

[0147] Next, redundant boxes are removed through the non-maximum suppression (NMS) algorithm, and finally the optimal bounding boxes and category predictions are output. The process of NMS is as follows:

[0148]

[0149] Through the above steps, the system finally obtains the optimal detection results.

[0150] Step 4, Extract the position feature points of the leaf prediction boxes from the detection results output in Step 3, then check the counting conditions, trigger the counting, and update the count value. The specific steps are as follows:

[0151] Step 4-1, After obtaining the optimal candidate box positions and category predictions, take the center point of each leaf prediction box output finally as the position feature point for subsequent leaf positioning and counting tasks. For each prediction box its center point coordinates (x c , y c ) are calculated by the following formula:

[0152]

[0153] Step 4-2. In the aero-engine blade counting task, accurately correlating the change of the feature points at the blade edge position over time is the core of achieving accurate counting. The Finite-State Machine (FSM) model can accurately depict the change process between states. The present invention proposes a spatio-temporal correlation counting model ( Figure 4 ), which updates the counter through the spatio-temporal correlation counting model by taking the blade position feature points as the input of the spatio-temporal correlation counting model. The spatio-temporal correlation counting model can be represented as a quadruple, as shown in the following formula:

[0154] M = (S, Σ, δ, s 0 ) (1.27)

[0155] In the formula: S is a finite state set; Σ is an event set; δ is a state transition function; S 0 is the initial state, s 0 ∈S.

[0156] The feature points at the blade edge position in the i-th frame in Formula 1.27 are defined as a set of states, and the feature points at the blade edge position in the (i + 1)-th frame are defined as an event set. Since there is a certain interval in the spatial distribution of the aero-engine blade edge position feature points in the blade rotation direction, the Region of Interest (ROI) is selected according to the rotation mode of the blade ( Figure 3 ), and three Regions of Interest are specifically defined (see Figure 3 ): R0, R1, and R2. R0 represents the region in the image except for the R1 and R2 regions, that is, the region in the non-counting state and does not participate in the state transition. The division of the ROI ensures that at the same moment, there is at most one feature point at the blade edge position in the union region composed of R1 and R2. When the aero-engine blade rotates horizontally, the R1 and R2 regions are distributed vertically; when the blade rotates vertically, the R1 and R2 regions are distributed horizontally. This distribution method ensures that the distribution directions of R1 and R2 are consistent with the rotation direction of the aero-engine blade. The spatio-temporal correlation counting model realizes the counting and direction recognition of the aero-engine blade by tracking the migration of the feature points at the blade edge position between R1 and R2. The model performs effective state transitions according to the state satisfied by the feature points at the blade edge position in the union of R1 and R2 in the i-th frame and the event corresponding to the feature points at the blade edge position in the (i + 1)-th frame. During a specific state transition process, the model triggers the blade counting action to update the counter.

[0157] Specifically, when using the spatio-temporal correlation model for aero-engine blade counting, the distribution of the feature points at the blade edge position in the i-th frame in R0, R1, and R2 represents three states S 0 , S 1 , and S 2 . In the (i + 1)-th frame, the distribution of R0, R1, and R2 represents three events E0 , E 1 and E 2 . A 1 and A 2 are the forward and reverse counting actions that need to be triggered during the specified state transition process. When event E 2 occurs and triggers the state S 2 to undergo a state transition, action A 2 is executed; when event E 1 occurs and triggers the state S 1 to undergo a state transition, action A 1 is executed.

[0158] In the specific embodiments described above, the purpose, technical solutions, and beneficial effects of the present invention have been further described in detail. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for visually counting rotor blades of an aircraft engine, characterized in that: The following steps are involved: Step 1: Read the video stream through the industrial video endoscope and initialize the region of interest for the first frame image; Step 2: Extract feature points from each frame of the image, use a feature point matching algorithm to track the displacement of feature points between two consecutive frames, and sense the rotation amplitude and direction of the blades; Step 3: For images whose rotation degree of the blade is greater than the threshold, image enhancement is performed through histogram equalization, and then the blade edge is detected; Step 4: Extract the position feature points of the leaf prediction box from the detection results output in step 3, then check the counting conditions, trigger the counting and update the counting value.

2. A method for visually counting aircraft engine rotor blades according to claim 1, characterized in that: The step 1 is to read the video stream through the industrial video endoscope and initialize the region of interest for the first frame image, as follows: The video stream is read through an industrial video endoscope to obtain continuous images of the blades; Initialize the region of interest (ROI) for the first frame image so that the edges of two adjacent leaves have a constraint effect on the two adjacent counting ROIs; Initialize the rotation direction of the blade according to the first frame image: if it rotates horizontally, initialize the rotation factor D=0; if it rotates vertically, initialize the direction factor D=1.

3. The method for visually counting rotor blades of an aircraft engine according to claim 1, characterized in that: The step 2 is to extract feature points from each frame of the image, use a feature point matching algorithm to track the displacement of feature points between two consecutive frames, and sense the rotation amplitude and direction of the blades. The specific steps are as follows: For each frame of image read, the FAST algorithm is used to extract feature points; Perform quadtree equalization on these feature points to improve the distribution uniformity of feature points; The displacement of feature points is tracked in two consecutive frames using feature point matching algorithm; The blade rotation amplitude and angle are determined based on the mean of the displacement and direction of the feature points.

4. A method for visually counting aircraft engine rotor blades according to claim 3, characterized in that: The method of judging the blade rotation amplitude and angle according to the mean of the characteristic point displacement and direction comprises the following steps: 1) Calculate the blade rotation offset: For each pair of feature points Calculate the deviation of its spatial position, that is, the displacement on the image coordinates: the coordinates of the matching point in the current frame are The coordinates of the matching point in the previous frame are P2 j =(x2 j ,y2 j ), then the position deviation According to the calculated feature point position deviation ΔP i , get the average deviation of feature point position i and j represent feature point indexes, and N represents the total number of feature points; For each pair of feature points, calculate its displacement vector v i =(v xi ,v yi ), its deviation ΔP i With the displacement direction through θ i =atan(v yi ,v xi ) calculation; rotation direction Among them, θ avg is the average rotation angle of all feature point pairs; 2) Rotation direction perception: If ΔP avg If it is greater than the set threshold, the rotation factor D and the cosine value cosθ of the average angle are used. avg and the sine value sinθ avg Determine the direction of rotation of the blade: When the rotation factor D = 0, if cosθ avg >0, the blade is judged to be right-handed; if cosθ avg <0, it is judged as left-handed; When the rotation factor D = 1, if sinθ avg >0, it is judged as topspin; if sinθ avg <0, it is judged as backspin.

5. The method for visually counting rotor blades of an aircraft engine according to claim 1, characterized in that: In step 3, for images whose rotation degree of the blade is greater than a threshold, image enhancement is performed by histogram equalization, and then the blade edge is detected, as follows: 3.1) Histogram equalization: The input image I(x, y) is processed to obtain the grayscale histogram H(k) of the image; based on the grayscale histogram H(k), the cumulative distribution function C(k) of the image is calculated, and the cumulative distribution function C(k) is normalized to obtain the normalized cumulative distribution function C′(k); through the normalized cumulative distribution function C′(k), each pixel value of the input image is mapped to a new grayscale value to generate the enhanced image I′(x, y); 3.2) Image feature extraction: The enhanced image I′(x,y) is resized to make it consistent with the input of GEResNet, and features are extracted to output the feature map F: in, Indicates feature extraction through GEResNet, x and y represent the position index of the pixel in the input image I′, and x' and y' represent the position index of the pixel in the output feature map; The GEResNet uses the GS-ECA module to replace the second 3x3 convolution in the residual structure of the original Resnet34 stage 1 to stage 4; wherein the GS-ECA module is based on the Ghost convolution and ECA attention mechanism, and performs the following steps: Ghost convolution is expressed as: F ghost =[Conv(F in ),A·Conv(F in )]; where F ghost is a redundant feature map, Conv is a convolution operation, A is a linear transformation matrix, and F in is the input feature map; The ECA attention mechanism obtains the channel average through global average pooling and generates an adaptive channel attention coefficient F through one-dimensional convolution att =α c ·F in ; Among them, α c is the channel attention coefficient; Finally, the GS-ECA module fuses the two feature maps by element-by-element multiplication to obtain the final output feature map F out : F out =F ghost ⊙F att ; 3.3) Hybrid encoder processing: The feature map F(x',y') extracted by GEResNet is processed by the hybrid encoder to generate the final hybrid feature map Z to enhance the local and global information of the feature; 3.4) Decoder processing: The input feature map Z is processed by the decoder to generate the bounding box position and category prediction of the target; 3.5) Query selector processing: get the candidate box position output by the decoder and the predicted category Finally, the candidate boxes are screened, and the redundant boxes are removed by the non-maximum suppression algorithm, and finally the optimal bounding box and predicted category are output.

6. The method for visually counting aircraft engine rotor blades according to claim 1, characterized in that: The step 4 extracts the position feature points of the leaf prediction frame from the detection results output in step 3, then checks the counting conditions, triggers the counting and updates the counting value. The specific steps are as follows: Step 4-1, position feature point extraction: After obtaining the optimal bounding box and predicted category, the center point of each leaf bounding box is used as the edge position feature point; Step 4-2, leaf counter update: The leaf position feature points are used as the input of the spatiotemporal correlation counting model, and the counter is updated through the spatiotemporal correlation counting model, as follows: The spatiotemporal correlation counting model is represented as a four-tuple M = (S, Σ, δ, s0), where S is a finite state set, Σ is an event set, δ is a state transition function, S0 is the initial state, and s0∈S; The feature points of the leaf edge positions in the i-th frame are defined as a state set, and the feature points of the leaf edge positions in the i+1-th frame are defined as an event set. The spatiotemporal association counting model performs effective state transitions according to the states satisfied by the feature points of the leaf edge positions in the i-th frame and the events corresponding to the feature points of the leaf edge positions in the i+1-th frame. In the specific state transition process, the leaf counting action is triggered to update the counter.

7. A method for visually counting aircraft engine rotor blades according to claim 6, characterized in that: In step 4-2, when the spatiotemporal correlation model is used to count aeroengine blades, the distribution of the blade edge position feature points in R0, R1 and R2 in the i-th frame respectively represents three states S0, S1 and S2; in the i+1-th frame, the distribution of R0, R1 and R2 respectively represents three events E0, E1 and E2; A1 and A2 are the forward counting action and reverse counting action that need to be triggered during the specified state transition process; when event E2 triggers state S2 to transition, action A2 is executed; when event E1 triggers state S1 to transition, action A1 is executed.

8. A visual counting system for aircraft engine rotor blades, characterized in that: include: An image initialization module, used for initializing a region of interest for a first frame image of a video stream read by an industrial video endoscope; The blade rotation amplitude and direction sensing module is used to extract feature points from each frame of the image, track the displacement of feature points between two consecutive frames using a feature point matching algorithm, and sense the blade rotation amplitude and direction; The blade edge detection module is used to enhance the image of the blade whose rotation degree is greater than the threshold value through histogram equalization, and then detect the blade edge; The counting module is used to extract the position feature points of the leaf prediction box from the output detection results, and then check the counting conditions, trigger the counting and update the counting value.

Images