Mark-free phase identification and phase mapping method for large-scale rotational motion structure

Abstract

The invention discloses an unmarked phase identification phase mapping method for a large-scale rotary motion structure, which comprises the following steps of: fusing a phase mapping algorithm with quadrant multi-scale identification characteristics in a video sequence and a sparse optical flow method to respectively obtain large-scale rotary motion displacement measurement results; when inter-frame rotation is small enough and meets the'space continuity hypothesis' of an optical flow algorithm, a periodic rotating blade optical flow tracking measurement method based on a large-scale rotating motion structure unmarked phase identification phase mapping method is established, and displacement tracking of the large-scale rotating motion structure by a traditional optical flow algorithm is achieved under the unmarked condition. According to the unmarked phase identification phase mapping method and the shift driving method for the large-scale rotary motion structure, the problems that large-scale rotary motion tracking cannot be realized and displacement measurement cannot be realized by a traditional optical flow algorithm are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual displacement measurement, and particularly relates to a method for unmarked phase recognition phase mapping of large-scale rotational motion structures. Background Art

[0002] Rotating structures play a crucial role in the field of major equipment. Since they often operate under extreme conditions, they are more vulnerable to damage than other components. Therefore, tracking and monitoring the motion state of rotating equipment is crucial for predicting the normal operation of the structure. Monitoring and maintaining the equipment during operation is a key means to ensure the correct service of the equipment. Replacing faulty components through system monitoring before a fault occurs can save a large amount of downtime and prevent further major damage caused by the fault. Therefore, implementing various maintenance strategies in actual working conditions can effectively prevent or minimize the faults of rotating machinery during operation. For traditional optical flow algorithms, the inter-frame structural motion requirements are small, which severely limits the application of this technology in large-scale rotation cases. Therefore, this patent mainly proposes a phase mapping method for unmarked phase recognition of large-scale rotational motion structures, mainly solving the problem that the traditional optical flow algorithm cannot meet the small motion assumption for tracking large-scale rotational motion. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for unmarked phase recognition phase mapping of large-scale rotational motion structures, solving the problems that the traditional optical flow algorithm cannot achieve tracking of large-scale rotational motion and cannot achieve displacement measurement.

[0004] To achieve the above purpose, the technical solution adopted by the present invention is: a method for unmarked phase recognition phase mapping of large-scale rotational motion structures, which fuses the large-scale rotational motion displacement measurement results obtained by a phase mapping algorithm with quadrant multi-scale recognition characteristics and a sparse optical flow method in a video sequence. By reordering the rotation frames of the rotating structure according to the phase sequence, when the inter-frame rotation is small enough and satisfies the "spatial continuity assumption" of the optical flow algorithm, a periodic rotating blade optical flow tracking measurement method based on the unmarked phase recognition phase mapping method of large-scale rotational motion structures is established to achieve displacement tracking of large-scale rotational motion structures by the traditional optical flow algorithm without markers.

[0005] As a preferred technical solution of the present invention, it is specifically implemented according to the following steps:

[0006] Step 1, minimize the large displacement caused by the overall rotational motion of the structure to ensure that the rotation between frames satisfies the spatial coherence assumption;

[0007] Step 2, when collecting large-scale rotational motion images, to improve the accuracy of edge detection, it is necessary to effectively filter the noise in the image before performing image edge detection to obtain a more complete edge contour and form the required straight edge points finally;

[0008] Step 3, after obtaining the required straight edge points, the Hough transform converts the straight line detection problem in the image space into a curve intersection problem in the parameter space by converting the coordinates between the parameter space and the normal coordinate system, and detects the straight lines in the image;

[0009] Step 4, based on the process and results of Step 3, obtain the videos after phase sorting and recombination of the periodic rotating blades at different rotational speeds;

[0010] Step 5, based on the videos after phase sorting and recombination of the periodic rotating blades at different rotational speeds obtained in Step 4, use the pyramid LK algorithm under ORB feature point detection to track the rotating structure at rest, so as to quantify the error performance of the vision detection system under static conditions, verify the robustness of the algorithm, and provide a benchmark and basis for subsequent dynamic analysis and subsequent system optimization.

[0011] As a preferred technical solution of the present invention, in the said Step 1, specifically:

[0012] By rearranging the image set to reduce the inter-frame rotation of the structure and mapping it to the range of [0, 2π] in the order of phase; the structure phase is used to describe the rotation of the structure, β is regarded as the rotation angle of the structure, that is, the angle by which the structure rotates within one time step, then the rotating structure angle can be expressed as follows:

[0013]

[0014] In the formula, β I represents the rotation angle at the I-th frame, A is the cumulative rotation angle, and j is the number of the current frame;

[0015] When the rotation direction of the structure is normal, it will not change. In this case, β remains positive, and as time goes by, A j is monotonically increasing, so the phase A of the entire structure n , can be represented by the average rotation angle of each time step:

[0016]

[0017] In the formula, θ a is the average rotation angle of all time steps, and n is the number of the sampled frames;

[0018] Due to the periodic nature of rotational motion, all possible positions of the structure are covered in a full rotation; this means that any phase angle β I can be mapped to the range [0, 2π]; to ensure the accuracy of this mapping process, the maximum phase value A m should ideally approach 2π or 360° after mapping. This mapping process can be expressed as follows:

[0019]

[0020] where θ a′ is the rearranged average rotation angle, and A m is the maximum phase value after mapping;

[0021] By comparing θ a′ in Equation (3) with θ a in Equation (2), the phase mapping method can reduce θ a′ to 1 / N of its original value, where N represents the total number of rotation revolutions. This result shows that the more structural rotation revolutions are sampled, the more significant the reduction in the average rotation angle; when θ a′ is reduced to a small value, the inter-frame motion state caused by the structural rotation will be correspondingly reduced, thus better meeting the "spatial continuity assumption" of traditional optical flow algorithms, that is, the motion between frames should be smooth and consistent.

[0022] As a preferred technical solution of the present invention, in step 2, a Gaussian filter is used to smooth the noisy image to reduce the influence of noise.

[0023] As a preferred technical solution of the present invention, in step 2, specifically: an appropriate threshold is found based on experiments to retain the true edge information in the image;

[0024] First, the two-dimensional Gaussian distribution function G(x, y) is used as the smoothing factor, and its expression is:

[0025]

[0026] where σ 2 is the mean square deviation, which is used to control the smoothing degree of the filter. x and y are the coordinates of each point in the filter, representing the values of the Gaussian function at these coordinates; the center of the filter is generally located at (0, 0), that is, the origin position, which means that the peak of the Gaussian function appears at the center of the filter, and the function value gradually decays as the distance from the origin increases, thereby achieving the smoothing of the image;

[0027] The horizontal and vertical directions of the original image f(x,y) are respectively convolved using a smoothing factor to obtain the filtered image I(x,y), and its mathematical expression is as follows:

[0028] I(x,y) = G(x,y) * f(x,y) (5)

[0029] In the formula, G(x,y) is a two-dimensional Gaussian smoothing factor, * represents the convolution operation, I(x,y) is the filtered image, and f(x,y) is the original image;

[0030] This process effectively removes noise and retains the main structural information of the image by performing convolution calculations in the horizontal and vertical directions of the image, making the filtered image smoother and more continuous, and suitable for subsequent edge detection processing;

[0031] The gradient magnitude A(i,j) and direction α(i,j) are obtained through the horizontal gradient and vertical gradient:

[0032]

[0033] In the formula, G x (i,j) and G y (i,j) are respectively the gradient components in the horizontal and vertical directions of the image at the point (i,j), and i and j are the coordinates of the positions that locate these gradient values in the image;

[0034] After obtaining the gradient magnitude A(i,j) and direction α(i,j), non-maximum suppression is required to retain the true edge structure. The gradient magnitude A(i,j) of the current pixel is compared with the gradient magnitudes of the adjacent pixels on both the positive and negative sides along the gradient direction α(i,j). If the gradient magnitude of the current pixel is the maximum compared to the gradient magnitudes of these two pixel points, then this pixel is retained as an edge point; otherwise, this pixel point will be suppressed. This process ensures that only the pixels with the maximum gradient magnitude are retained by removing local non-maxima, thereby improving the accuracy and clarity of edge detection and retaining the true edge structure;

[0035] At the same time, in order to ensure that there are no pseudo-edge points among the detected edge points, pseudo-edge points refer to false edge information caused by factors such as noise, texture or details, and illumination changes. It is necessary to perform thresholding on the image after non-maximum suppression. By setting a fixed threshold, the pixel values with gradient magnitudes lower than this threshold are set to zero. However, too low or too high a threshold may lead to a decrease in edge contrast or the loss of some edge information. Therefore, the double-threshold method is used to segment the image after non-maximum suppression to optimize the edge detection effect;

[0036] Two gradient thresholds are used: a low threshold and a high threshold, and two thresholds δ 1 and δ2 , where δ 1 is the low threshold, and δ 2 is the high threshold. First, segment the image according to the high threshold to obtain the high-threshold edge image This image contains fewer false edges, but the edges may be discontinuous or unclosed; subsequently, segment the image according to the low threshold to obtain the low-threshold edge image The core idea of the double-threshold method is to form a complete edge contour by connecting the edges in; at the endpoints of the edge contour, search for edge points in the neighborhood of the corresponding low-threshold edge image that may continue the contour; this algorithm continuously collects edge information in and finally connects the edges in to obtain a more complete edge contour, forming the final required straight-edge points.

[0037] As a preferred technical solution of the present invention, in step 3, specifically:

[0038] In the normal coordinate system of the image, each straight line can be mapped to a point in the parameter space. At the same time, each point in the image corresponds to a curve in the parameter space; any straight line can be represented in polar coordinates as a straight line:

[0039] ρ = xcosθ + ysinθ (8)

[0040] where: ρ represents the perpendicular distance from the point to the origin, θ is the angle between the perpendicular line and the x-axis, and (x, y) are the coordinates of each edge point detected by edge detection;

[0041] In the image space, each edge point can be mapped to a curve in the parameter space, that is, a curve in the ρ-θ space. When multiple such curves intersect in the parameter space, these intersection points correspond to straight lines in the image space; therefore, by searching for such intersection points in the parameter space, the straight lines in the image can be effectively detected.

[0042] As a preferred technical solution of the present invention, in step 4, specifically:

[0043] Among the straight-line parameters returned by the Hough transform, ρ is the perpendicular distance from the point to the origin, and θ is the angle between the perpendicular line and the x-axis. Therefore, θ is not the angle between the straight line itself and the x-axis, but the angle between the normal direction of the straight line and the x-axis; the angle β of the straight line is the complement of the angle between its normal direction and the x-axis. To obtain the angle between the straight line and the x-axis, the calculation method is as follows:

[0044]

[0045] Since large-scale rotational motion involves 360° periodic motion, in the Hough transform, to describe all possible lines, the two possible directions of a line are distinguished by the positive and negative values of θ, reducing the redundancy of θ in the range [0, 2π]. Therefore, the angle θ between the perpendicular line and the x-axis is Therefore, to accurately describe the line angle in large-scale rotational motion, the calculated line angle β needs to be corrected according to the actual rotation problem;

[0046] In a rotating structure, the blades often have symmetry, similarity in texture and shape. For two symmetric blades, assume the angles of blade 1 and blade 2 are β 1 and β 2 , they are on the same radius and are symmetric with respect to the center;. Due to the symmetry of the structure, there is a relationship: β 2 = β 1 + 180°. In this case, the detected angles β 1 and β 2 will be confused in some cases. The Hough transform may detect similar lines at symmetric positions, and these lines will give similar ρ and θ values at symmetric positions; Using spatial localization in geometry, this problem is solved by quadrant region constraint. This method ensures that only one blade is allowed to be detected within a specific range, thus avoiding interference between blades;

[0047] Through the quadrant multi-scale method, the quadrant space where the line is located in the initial frame can be preferentially detected, and in subsequent frames, the same quadrant can be preferentially detected according to the previous detection result. Finally, other quadrants are detected in sequence according to the rotational motion direction, ensuring the continuity and stability of the phase angle calculation; For a symmetric rotating structure, the continuity detection and phase angle jump exclusion strategy can effectively avoid calculation errors caused by the interference of symmetric blades; When the structure rotates, by the change of the quadrant space where the line detected on the rotating blade is located, the phase of the structure can be finally determined; In addition, when the phase angle gradually increases and approaches 360°, the phase angle value of the next moment will change and approach 0°, which means that the rotating blade has completed a full circle and started a new round of rotation.

[0048] As a preferred technical solution of the present invention, in step 5, specifically:

[0049] Through step 4, the video after phase sorting and recombination of the periodically rotating blades at different rotational speeds is obtained, and the pyramid LK algorithm with ORB feature point detection characteristics is used to perform optical flow tracking on the recombined video;

[0050] The pyramid LK optical flow method assumes that the motion vector is constant within a neighborhood Ω in a space. Then, within the neighborhood Ω of n pixels, each pixel satisfies the following formula:

[0051] I xi u + I yi v + I ti = 0 for i = 1, 2, … n (10)

[0052] In the formula, and are the spatial gradients in the x and y directions respectively. u and v represent the moving components of the optical flow in the horizontal and vertical directions, that is, the optical flow information to be obtained. is the change rate of the image grayscale with respect to time t, and i represents the i-th pixel point;

[0053] The final optical flow displacement vector d result is

[0054]

[0055] In the formula, g 0 is the optical flow guess value of the 0th layer, and the superscript 0 represents the 0th layer of the pyramid. d 0 is the optical flow result calculated for the 0th layer, and the superscript 0 represents the 0th layer of the pyramid. N is the number of pyramid layers, L represents the layer at which the optical flow is being calculated, and d L is the optical flow result calculated for the Lth layer;

[0056] Then the corresponding point of the pixel U in the previous frame image I a in the next frame image I b is:

[0057] ν = u + d (12)

[0058] In the formula, ν is the pixel position in I b and u is the pixel position in I a and d is the final optical flow displacement vector;

[0059] Through the object tracking using the optical flow algorithm, the pixel displacement time history curve of the specified target point can be obtained. The actual displacement of the target in the world coordinate system can be converted through a scale factor:

[0060]

[0061] In the formula, S represents the scale factor, that is, the actual size of the target corresponding to the unit pixel length in the image coordinate system. D and d P represent the actual size and pixel length respectively, Z represents the distance between the lens and the photographed object, and f represents the focal length of the lens.

[0062] The beneficial effects of the present invention are as follows: Through the research on the visual vibration displacement measurement of large-scale rotational motion, the present invention proposes a phase mapping method for markerless phase recognition of large-scale rotational motion structures, which solves the problems that the existing traditional optical flow algorithms cannot achieve the tracking of large-scale rotational motion and cannot achieve displacement measurement. Through the improved optical flow tracking algorithm, the present invention successfully realizes the accurate tracking of the phase arrangement video sequence that satisfies the small motion hypothesis, and further solves the problem that the traditional optical flow algorithm is difficult to satisfy the small motion hypothesis in large-scale rotational motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0064] Figure 1 It is the schematic diagram of the markerless phase recognition phase mapping method for the large-scale rotational motion structure of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0065] The technical solutions of the present invention will be further described in detail below in conjunction with the drawings and specific embodiments.

[0066] Embodiment 1

[0067] As Figure 1 shown, a markerless phase recognition phase mapping method for a large-scale rotational motion structure of the present invention combines the displacement measurement results of large-scale rotational motion obtained by the phase mapping algorithm with quadrant multi-scale recognition characteristics and the sparse optical flow method in the video sequence. By reordering the rotation frames of the rotating structure according to the phase order, when the inter-frame rotation is small enough and satisfies the "spatial continuity hypothesis" of the optical flow algorithm, a periodic rotating blade optical flow tracking measurement method based on the markerless phase recognition phase mapping method for the large-scale rotational motion structure is established, and the displacement tracking of the large-scale rotational motion structure by the traditional optical flow algorithm is realized without markers. The specific implementation steps are as follows:

[0068] Step 1: Minimize the large displacement caused by the overall rotational motion of the structure to ensure that the rotation between frames satisfies the spatial coherence hypothesis;

[0069] Step 2: When collecting images of large-scale rotational motion, in order to improve the accuracy of edge detection, it is necessary to effectively filter the noise in the image before performing image edge detection to obtain a more complete edge contour and form the final required straight edge points;

[0070] Step 3, after obtaining the required straight edge points, the Hough transform converts the straight line detection problem in the image space into a curve intersection problem in the parameter space by transforming the coordinates between the parameter space and the normal coordinate system, and detects the straight lines in the image;

[0071] Step 4, based on the process and results of Step 3, obtain the videos of the periodically rotating blades at different rotational speeds after phase sorting and recombination;

[0072] Step 5, based on the videos of the periodically rotating blades at different rotational speeds obtained in Step 4 after phase sorting and recombination, use the pyramid LK algorithm under ORB feature point detection to track the rotating structure at rest, thereby quantifying the error performance of the vision detection system under static conditions, verifying the robustness of the algorithm, and providing a benchmark and basis for subsequent dynamic analysis and subsequent system optimization.

[0073] The present invention proposes a phase mapping method for markerless phase recognition of large-scale rotating motion structures. Aiming at the problem that the traditional optical flow algorithm cannot meet the "spatial continuity assumption" of large-scale rotating motion, a method for measuring the optical flow tracking of periodically rotating blades based on the local recognition characteristics of quadrant space in the recombined video is invented. By dividing the video sequence of the large-scale rotating structure into quadrants, the displacement tracking of the large-scale rotating motion by the traditional optical flow algorithm is realized without markers. By rearranging the image set to reduce the inter-frame rotation of the structure, and by reordering the frames of the rotating structure, the displacement tracking of the large-scale rotating motion by the traditional optical flow algorithm is realized without markers. And the present invention can be easily embedded into various current mainstream object detectors to improve their detection accuracy for rotating targets.

[0074] Embodiment 2

[0075] As Figure 1 shown, different from Embodiment 1, in Step 1 of a method for phase mapping of markerless phase recognition of a large-scale rotating motion structure of the present invention:

[0076] To minimize the large displacement caused by the overall rotational motion of the structure and ensure that the rotation between frames satisfies the spatial coherence assumption, which is used to describe that in an image sequence, the motion of pixels or regions is usually relatively smooth and consistent. By rearranging the image set to reduce the inter-frame rotation of the structure, map it to the range of [0, 2π] according to the order of phases. The structure phase is used to describe the rotation of the structure. β is regarded as the rotation angle of the structure, that is, the angle by which the structure rotates within a time step. Then the rotation angle of the rotating structure can be expressed as follows:

[0077]

[0078] In the formula, β IRepresents the rotation angle at the I-th frame, A is the cumulative rotation angle, and j is the number of the current frame.

[0079] The rotation direction of the structure generally does not change. In this case, β remains positive, and over time, A j is monotonically increasing, so the phase A of the entire structure n , can be represented by the average rotation angle for each time step:

[0080]

[0081] In the formula, θ a is the average rotation angle for all time steps, and n is the number of the sampled frames.

[0082] Due to the periodic nature of the rotational motion, all possible positions of the structure are covered in a complete rotation. This means that any phase angle β I can be mapped to the range of [0, 2π]. To ensure the accuracy of this mapping process, the maximum phase value A after mapping m should ideally approach 2π (or 360°). This mapping process can be represented as follows:

[0083]

[0084] In the formula, θ a′ is the rearranged average rotation angle, and A m is the maximum phase value after mapping.

[0085] By comparing θ a′ in Equation (3) with θ a in Equation (2), through the phase mapping method, θ a′ can be reduced to 1 / N of the original value (where N represents the total number of rotation turns). This result shows that the more the number of rotation turns of the sampled structure, the more significant the reduction in the average rotation angle. When θ a′ is reduced to a smaller value, the inter-frame motion state caused by the structure rotation will be correspondingly reduced, thus better meeting the "spatial continuity assumption" of the traditional optical flow algorithm, that is, the motion between frames should be smooth and consistent.

[0086] Example 3

[0087] Different from Embodiment 2, in step 2 of the method for markerless phase recognition phase mapping of a large-scale rotational motion structure according to the present invention, when collecting large-scale rotational motion images, in order to improve the accuracy of edge detection, it is necessary to effectively filter the noise in the image before performing image edge detection. The noisy image is smoothed by applying a Gaussian filter to reduce the influence of noise. In the specific calculation process, a suitable threshold is found based on experiments to retain the true edge information in the image. First, the two-dimensional Gaussian distribution function G(x,y) is used as the smoothing factor, and its expression is:

[0088]

[0089] In the formula, σ 2 is the mean square deviation, which is used to control the smoothing degree of the filter. x and y are the coordinates of each point in the filter, representing the values of the Gaussian function at these coordinates. The center of the filter is generally located at (0,0), that is, the origin position, which means that the peak value of the Gaussian function appears at the center of the filter, and the function value gradually decays as the distance from the origin increases, thereby realizing the smoothing processing of the image.

[0090] The smoothing factor is used to perform convolution operations on the horizontal and vertical directions of the original image f(x,y) respectively to obtain the filtered image I(x,y). Its mathematical expression is as follows:

[0091] I(x,y) = G(x,y) * f(x,y) (5)

[0092] In the formula, G(x,y) is the two-dimensional Gaussian smoothing factor, * represents the convolution operation, I(x,y) is the filtered image, and f(x,y) is the original image.

[0093] This process effectively removes noise and retains the main structural information of the image by performing convolution calculations in the horizontal and vertical directions of the image, making the filtered image smoother and more continuous, which is suitable for subsequent edge detection processing.

[0094] The gradient magnitude A(i,j) and direction α(i,j) are obtained through the horizontal gradient and vertical gradient:

[0095]

[0096]

[0097] In the formula, G x (i,j) and G y (i,j) are the gradient components of the image in the horizontal and vertical directions at the point (i,j) respectively. i and j are the coordinates of the positions in the image that locate these gradient values.

[0098] After obtaining the gradient magnitude A(i,j) and direction α(i,j), non-maximum suppression is required to retain the true edge structure. The gradient magnitude A(i,j) of the current pixel is compared with the gradient magnitudes of the adjacent pixels on both the positive and negative sides along the gradient direction α(i,j). If the gradient magnitude of the current pixel is the maximum compared to the gradient magnitudes of these two pixel points, then the pixel is retained as an edge point; otherwise, the pixel point will be suppressed. This process improves the accuracy and clarity of edge detection by removing local non-maxima and ensuring that only the pixels with the maximum gradient magnitude are retained, thus retaining the true edge structure.

[0099] At the same time, to ensure that there are no false edge points among the detected edge points, false edge points refer to false edge information caused by factors such as noise, texture or details, and illumination changes. It is necessary to perform thresholding on the image after non-maximum suppression. By setting a fixed threshold, the pixel values with gradient magnitudes lower than this threshold are set to zero. However, too low or too high a threshold may lead to a decrease in edge contrast or the loss of some edge information. Therefore, a double-threshold method is used to segment the image after non-maximum suppression to optimize the edge detection effect.

[0100] The specific operation is as follows. Two gradient thresholds are used: the low threshold CannyThresholdMin and the high threshold CannyThresholdMax. Set two thresholds δ 1 and δ 2 , where δ 1 is the low threshold and δ 2 is the high threshold. First, segment the image according to the high threshold to obtain the high-threshold edge image This image contains fewer false edges, but the edges may be discontinuous or unclosed. Subsequently, segment the image according to the low threshold to obtain the low-threshold edge image The core idea of the double-threshold method is to connect the edges in to form a complete edge contour. At the endpoints of the edge contour, look for edge points in the neighborhood of the corresponding low-threshold edge image that may continue the contour. This algorithm continuously collects edge information in and finally connects the edges in to obtain a more complete edge contour and form the final required straight edge points.

[0101] Embodiment 4

[0102] Different from Embodiment 3, in step 3 of the method for markerless phase recognition phase mapping of a large-scale rotational motion structure according to the present invention, after obtaining the required straight edge points, the Hough transform is required to convert the straight line detection problem in the image space into the problem of curve intersection points in the parameter space by converting the coordinates in the parameter space and the normal coordinate system. The basic principle of the Hough line detection is that in the normal coordinate system of the image, each straight line can be mapped to a point in the parameter space. At the same time, each point in the image corresponds to a curve in the parameter space. More specifically, any straight line can be represented in polar coordinates as a straight line:

[0103] ρ = xcosθ + ysinθ (8)

[0104] In the formula: ρ represents the perpendicular distance from the point to the origin, θ is the angle between the perpendicular line and the x-axis, and (x, y) are the coordinates of each edge point obtained by edge detection.

[0105] In the image space, each edge point can be mapped to a curve in the parameter space (i.e., a curve in the ρ-θ space). When multiple such curves intersect in the parameter space, these intersection points correspond to the straight lines in the image space. Therefore, by searching for such intersection points in the parameter space, the straight lines in the image can be effectively detected.

[0106] Embodiment 5

[0107] As Figure 1 shown, different from Embodiment 4, in step 4 of the method for markerless phase recognition phase mapping of a large-scale rotational motion structure according to the present invention, among the straight line parameters returned by the Hough transform, ρ is the perpendicular distance from the point to the origin, and θ is the angle between the perpendicular line and the x-axis. Therefore, θ is not the angle between the straight line itself and the x-axis, but the angle between the normal direction (i.e., the perpendicular direction) of the straight line and the x-axis. The angle β of the straight line is the supplementary angle of the angle between its normal direction and the x-axis. To obtain the angle between the straight line and the x-axis, the calculation method is as follows:

[0108]

[0109] Since the large-scale rotational motion involves periodic motion of 360°, in the Hough transform, in order to describe all possible straight lines, the two possible directions of the straight line are distinguished by the positive and negative values of θ, reducing the redundancy of θ in the range of [0, 2π]. Therefore, the angle θ between the perpendicular line and the x-axis is Therefore, in order to accurately describe the straight line angle in the large-scale rotational motion, the calculated straight line angle β needs to be corrected according to the actual rotation problem.

[0110] In the rotational structure, the blades often have symmetry, similarity in texture and shape. For two symmetric blades, assuming the angles of blade 1 and blade 2 are β 1 and β2 , they are on the same radius and symmetric with respect to the center. Due to the symmetry of the structure, there is a relationship: β 2 = β 1 + 180°, in this case, the detected angles β 1 and β 2 will be confused in some cases. The Hough transform may detect similar lines at symmetric positions, and these lines will give similar ρ and θ values at symmetric positions. Using spatial localization in geometry, this problem is solved by quadrant region constraint. This method ensures that only one blade is allowed to be detected within a specific range, thus avoiding interference between blades.

[0111] The steps of the quadrant multi-scale method are as follows:

[0112] 1. Initial frame detection: Detect lines in the order of the quadrant space and retain the initially detected lines.

[0113] 2. Subsequent frame detection: In each frame, give priority to detecting within the quadrant space where the line was detected last time. This strategy helps to preferentially detect the same blade when the blade moves slowly or the angle changes little.

[0114] 3. Large-scale rotation direction detection strategy: Detect in sequence according to the rotation direction and the quadrant space where the line was detected last time.

[0115] 4. Continuity and stability strategy: ① Continuity detection: Compare the positions and angles of the lines in the same quadrant between consecutive frames to ensure that the phase angle calculated each time is for the same blade. ② Phase angle jump exclusion: By judging the continuity of the phase angle, exclude the jumps caused by other blades or interference, thus ensuring the accuracy of the phase angle calculation.

[0116] Through the quadrant multi-scale method, it is possible to preferentially detect the quadrant space where the line is located in the initial frame, and preferentially detect the same quadrant in subsequent frames according to the previous detection result. Finally, detect other quadrants in sequence according to the rotation direction of the movement, ensuring the continuity and stability of the phase angle calculation. For a symmetric rotation structure, the continuity detection and phase angle jump exclusion strategies can effectively avoid calculation errors caused by the interference of symmetric blades. When the structure rotates, by the change of the quadrant space where the line detected on the rotating blade is located, the phase of the structure can be finally determined. In addition, when the phase angle gradually increases and approaches 360°, the phase angle value at the next moment will change and approach 0°, which means that the rotating blade has completed a full circle and started a new round of rotation.

[0117] Example 6

[0118] Different from Embodiment 5, in step 5 of the method for markerless phase recognition phase mapping of a large-scale rotational motion structure in the present invention, in order to evaluate the motion of the monocular vision device, the uneven illumination intensity or the amplitude of the error of the optical flow algorithm, the pyramid LK algorithm under ORB feature point detection is used to track the rotational structure at rest to obtain the overall noise under the current conditions.

[0119] Through step 4, videos after phase sorting and recombination of periodically rotating blades at different rotational speeds are obtained, and the pyramid LK algorithm with ORB feature point detection characteristics is used for optical flow tracking of the recombined videos.

[0120] The pyramid LK optical flow method assumes that the motion vector is constant within a neighborhood Ω in a space. Then, within the neighborhood Ω of n pixels, each pixel satisfies the following formula:

[0121] I xi u + I yi v + I ti = 0 i = 1, 2, … n (10)

[0122] In the formula, and are the spatial gradients in the x and y directions respectively, u and v represent the moving components of the optical flow in the horizontal and vertical directions, that is, the optical flow information to be obtained, is the change rate of the image gray level with respect to time t, and i represents the i-th pixel point;

[0123] The final optical flow displacement vector d result is

[0124]

[0125] In the formula, g 0 is the optical flow guess value of the 0th layer, and the superscript 0 represents the 0th layer of the pyramid. d 0 is the optical flow result calculated for the 0th layer, and the superscript 0 represents the 0th layer of the pyramid. N is the number of pyramid layers, L represents the layer at which the optical flow is being calculated, and d L is the optical flow result calculated for the Lth layer;

[0126] Then, the corresponding point of the pixel point U in the previous frame image I a in the next frame image I b is:

[0127] ν = u + d (12)

[0128] In the formula, ν is the pixel point position in I b , u is the pixel point position in I a , and d is the final optical flow displacement vector.

[0129] Through target tracking using the optical flow algorithm, the time history curve of the pixel displacement of a specified target point can be obtained. The actual displacement of the target in the world coordinate system can be converted through a scale factor:

[0130]

[0131] In the formula, S represents the scale factor, that is, the actual size of the target corresponding to the unit pixel length in the image coordinate system, D and d P represent the actual size and pixel length respectively, Z represents the distance between the lens and the photographed object, and f represents the focal length of the lens.

[0132] The foregoing description has shown and described several preferred embodiments of the invention. However, as previously mentioned, it should be understood that the invention is not limited to the forms disclosed herein, should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications, and environments, and can be changed within the scope of the inventive concept described herein through the above teachings or the techniques or knowledge in related fields. And any changes and modifications made by those skilled in the art without departing from the spirit and scope of the invention shall fall within the protection scope of the appended claims of the invention.

Claims

1. A label-free phase identification and mapping method for large-scale rotating motion structures, characterized in that: The large-scale rotational motion displacement measurement results obtained by the phase mapping algorithm with quadrant multi-scale recognition characteristics and the sparse optical flow method in the fusion video sequence are respectively obtained. By reordering the rotation frames of the rotating structure according to the phase order, when the rotation between frames is small enough and meets the "spatial continuity assumption" of the optical flow algorithm, an optical flow tracking measurement method for periodic rotating blades based on the markerless phase recognition phase mapping method of large-scale rotational motion structure is established. The displacement tracking of large-scale rotational motion structure by traditional optical flow algorithm is realized without markers.

2. The label-free phase identification and mapping method for large-scale rotating motion structures according to claim 1 is characterized in that: Follow the steps below to implement it: Step 1: minimize the large displacement caused by the overall rotation of the structure, ensuring that the rotation between frames meets the spatial coherence assumption; Step 2: When collecting large-scale rotating motion images, in order to improve the accuracy of edge detection, it is necessary to effectively filter the noise in the image before performing image edge detection to obtain a more complete edge contour and form the final required straight line edge points; Step 3: After obtaining the required straight line edge points, the Hough transform transforms the straight line detection problem in the image space into the curve intersection problem in the parameter space by converting the parameter space coordinates and the normal coordinate system, and detects the straight line in the image; Step 4, based on the process and results of step 3, obtain the video of the periodically rotating blades at different speeds after phase sorting and reorganization; Step 5: Based on the video of the periodically rotating blades at different speeds obtained in step 4 and reorganized through phase sorting, the pyramid LK algorithm under ORB feature point detection is used to track the rotating structure at rest, thereby quantifying the error performance of the visual inspection system under static conditions, verifying the robustness of the algorithm, and providing a benchmark and basis for subsequent dynamic analysis and subsequent system optimization.

3. The label-free phase identification and mapping method for large-scale rotating motion structures according to claim 2 is characterized in that: In the step 1, specifically: The image set is rearranged to reduce the inter-frame rotation of the structure and mapped to the range of [0,2π] in the order of phase; the structural phase is used to describe the rotation of the structure, and β is regarded as the rotation angle of the structure, that is, the angle at which the structure rotates within a time step. The rotation angle of the structure can be expressed as follows: In the formula, β I represents the rotation angle at the Ith frame, A is the cumulative rotation angle, and j is the number of the current frame; The rotation direction of the structure does not change normally. In this case, β remains positive and A j is monotonically increasing, so the phase A of the entire structure n , can be expressed as the average rotation angle at each time step: In the formula, θ a is the average rotation angle of all time steps, n is the number of the sampling frame; Due to the periodic nature of the rotational motion, all possible positions of the structure are covered in one complete rotation; this means that for any phase angle β I can be mapped to the range of [0,2π]; to ensure the accuracy of the mapping process, the maximum phase value A after mapping m Ideally, it should approach 2π or 360°. This mapping process can be expressed as follows: In the formula, θ a′ is the average rotation angle after rearrangement, A m The maximum phase value after mapping; By comparing θ in equation (3) a′ and θ in equation (2) a , through the phase mapping method, θ a′ It is reduced to 1 / N of the original value, where N represents the total number of rotations. This result shows that the more the sampled structural rotations are, the more significant the reduction in the average rotation angle is. a′ When it is reduced to a smaller value, the inter-frame motion state caused by the structural rotation will be reduced accordingly, thereby better satisfying the "spatial continuity assumption" of the traditional optical flow algorithm, that is, the motion between frames should be smooth and consistent.

4. The label-free phase identification and mapping method for large-scale rotating motion structure according to claim 3 is characterized in that: In step 2, a Gaussian filter is used to smooth the noisy image to reduce the impact of noise.

5. The label-free phase identification and mapping method for large-scale rotating motion structure according to claim 4 is characterized in that: In the step 2, specifically: find a suitable threshold value based on experiments to retain the real edge information in the image; First, use the two-dimensional Gaussian distribution function G(x,y) as the smoothing factor, which is expressed as: In the formula, σ 2 is the mean square error, which is used to control the smoothness of the filter. x, y are the coordinates of each point in the filter, indicating the value of the Gaussian function at these coordinates. The center of the filter is generally located at (0,0), that is, the origin. This means that the peak of the Gaussian function appears at the center of the filter, and the function value gradually decays as the distance from the origin increases, thereby achieving smooth processing of the image. The smoothing factor is used to perform convolution operations on the original image f(x, y) in the horizontal and vertical directions to obtain the filtered image I(x, y), whose mathematical expression is as follows: I(x,y)=G(x,y)*f(x,y) (5) Where G(x,y) is the two-dimensional Gaussian smoothing factor, * represents the convolution operation, I(x,y) is the filtered image, and f(x,y) is the original image; This process effectively removes noise and retains the main structural information of the image by performing convolution calculations in the horizontal and vertical directions of the image, making the filtered image smoother and more continuous, which is suitable for subsequent edge detection processing; Through the horizontal gradient and vertical gradient, the gradient amplitude A(i,j) and direction α(i,j) are obtained: In the formula, G x (i,j) and G y (i, j) are the horizontal and vertical gradient components of the image at point (i, j), respectively, and i and j are the coordinates of the locations where these gradient values ​​are located in the image; After obtaining the gradient amplitude A(i,j) and direction α(i,j), non-maximum suppression is required to retain the true edge structure; the gradient amplitude A(i,j) of the current pixel is compared with the gradient amplitudes of the adjacent pixels on the positive and negative sides of the gradient direction α(i,j). If the gradient amplitude of the current pixel is the maximum compared to the gradient amplitudes of these two pixels, the pixel is retained as an edge point; otherwise, the pixel will be suppressed; this process ensures that only pixels with the largest gradient amplitude are retained by removing local non-maximum values, thereby improving the accuracy and clarity of edge detection and retaining the true edge structure; At the same time, in order to ensure that there are no pseudo edge points in the detected edge points, pseudo edge points refer to false edge information caused by noise, texture or details, illumination changes and other factors, it is necessary to perform threshold processing on the image after non-maximum suppression. By setting a fixed threshold, the pixel value with a gradient amplitude lower than this threshold is set to zero. However, too low or too high a threshold may cause a decrease in edge contrast or the loss of some edge information. Therefore, the double threshold method is used to segment the image after non-maximum suppression to optimize the edge detection effect. Use two gradient thresholds: low threshold and high threshold, set two thresholds δ1 and δ2, where δ1 is the low threshold and δ2 is the high threshold. First, segment the image according to the high threshold to obtain the high threshold edge image. The image contains fewer pseudo edges, but the edges may be discontinuous or not closed; then, the image is segmented according to the low threshold to obtain a low threshold edge image The core idea of ​​the dual threshold method is to connect The edge in the image is a complete edge contour; at the end point of the edge contour, the corresponding low threshold edge image Find edge points that may continue the contour in the neighborhood of Continuously collect edge information, and eventually The edges in the image are connected to obtain a more complete edge contour and form the final required straight edge points.

6. The label-free phase identification and mapping method for large-scale rotating motion structures according to claim 5 is characterized in that: In the step 3, specifically: In the normal coordinate system of the image, each straight line can be mapped to a point in the parameter space. At the same time, each point in the image corresponds to a curve in the parameter space; any straight line can be represented by polar coordinates: ρ=xcosθ+ysinθ (8) Where: ρ represents the vertical distance from the point to the origin, θ is the angle between the vertical line and the x-axis, (x, y) is the coordinates of each edge point obtained by edge detection; In the image space, each edge point can be mapped to a curve in the parameter space, that is, a curve in the ρ-θ space, and when multiple such curves intersect in the parameter space, these intersections correspond to straight lines in the image space; therefore, by finding such intersections in the parameter space, the straight lines in the image can be effectively detected.

7. The label-free phase identification and mapping method for large-scale rotating motion structures according to claim 6 is characterized in that: In the step 4, specifically: In the line parameters returned by the Hough transform, ρ is the vertical distance from the point to the origin, and θ is the angle between the vertical line and the x-axis. Therefore, θ is not the angle between the line itself and the x-axis, but the angle between the normal direction of the line and the x-axis; the angle β of the line is the supplementary angle of the angle between its normal direction and the x-axis. In order to obtain the angle between the line and the x-axis, the calculation method is as follows: Since large-scale rotational motion involves 360° periodic motion, in order to describe all possible straight lines in the Hough transform, the positive and negative values ​​of θ are used to distinguish the two possible directions of the straight line, reducing the redundancy of θ in the range of [0,2π]. Therefore, the angle θ between the vertical line and the x-axis is Therefore, in order to accurately describe the straight line angle in large-scale rotational motion, it is necessary to correct the calculated straight line angle β according to the actual rotation problem; In rotating structures, blades often have symmetry, texture and shape similarity. For two symmetrical blades, assume that the angles of blade 1 and blade 2 are β1 and β2, respectively. They are on the same radius and symmetrical with respect to the center. Due to the symmetry of the structure, there is a relationship: β2 = β1 + 180°. In this case, the detected angles β1 and β2 will be confused in some cases. The Hough transform may detect similar straight lines at symmetrical positions, and these straight lines will give similar ρ and θ values ​​at symmetrical positions. Spatial localization in geometry is used to solve this problem through quadrant region constraints. This method ensures that only one blade is allowed to be detected within a specific range, thereby avoiding interference between blades. Through the quadrant multi-scale method, the quadrant space where the straight line is located in the initial frame can be detected first, and the same quadrant can be detected first in the subsequent frames according to the previous detection results. Finally, other quadrants are detected in sequence according to the rotation direction, ensuring the continuity and stability of the phase angle calculation; For symmetrical rotating structures, continuity detection and phase angle jump elimination strategies can effectively avoid calculation errors caused by interference from symmetrical blades. When the structure rotates, the phase of the structure can be finally determined by the change in the quadrant space where the straight line detected on the rotating blade is located. In addition, when the phase angle gradually increases and approaches 360°, the phase angle value at the next moment will change and approach 0°, which means that the rotating blade has completed a full circle and started a new round of rotation.

8. The large-scale rotating motion structure label-free phase identification and phase mapping method according to claim 7 is characterized in that: In the step 5, specifically: The video of periodically rotating blades at different speeds reorganized by phase sorting is obtained through step 4, and the optical flow tracking of the reorganized video is performed using the pyramid LK algorithm containing ORB feature point detection characteristics; The pyramid LK optical flow method assumes that the motion vector in a neighborhood Ω in a space is constant, then in a neighborhood Ω of n pixels, each pixel satisfies the following formula: I xi u+I yi v+I ti =0 i=1,2,…n (10) In the formula, and are the spatial gradients in the x and y directions respectively, u and v represent the moving components of the optical flow in the horizontal and vertical directions, that is, the required optical flow information, is the rate of change of image grayscale relative to time t, i represents the i-th pixel; The final optical flow displacement vector d is In the formula, g 0 is the optical flow guess value of the 0th layer. The superscript 0 represents the 0th layer of the pyramid. 0 is the optical flow result calculated at the 0th layer. The superscript 0 represents the 0th layer of the pyramid, N is the number of pyramid layers, L represents the number of layers the optical flow is being calculated, and d L The optical flow result calculated for the L layer; Then the previous frame image I a The pixel point U in the next frame image I b The corresponding points in are: ν=u+d (12) Where ν is I b The pixel position in the middle, u is I a The pixel position in the middle, d is the final optical flow displacement vector; After target tracking using the optical flow algorithm, the pixel displacement time curve of the specified target point can be obtained; the actual displacement of the target in the world coordinate system can be converted by the scale factor: Where S represents the scale factor, that is, the actual size of the target corresponding to the unit pixel length in the image coordinate system, and D and d P They represent actual size and pixel length respectively, Z represents the distance between the lens and the object being photographed, and f represents the focal length of the lens.

Images