Process pipeline micro-vibration motion amplification detection method based on automatic tracking and deconvolution

Abstract

The invention discloses a process pipeline micro-vibration motion amplification detection method based on automatic tracking and deconvolution, and the method comprises the steps: carrying out the automatic frame selection of a vibration abnormal region through an inter-frame difference method, tracking displacement through combining with an LK optical flow method, and analyzing a frequency spectrum, so as to adaptively determine an amplification frequency band; performing vibration signal enhancement by selecting an Euler or phase video motion amplification technology according to the environmental complexity; and further converting the RGB image into an HSV color space, inhibiting motion blur and artifacts by adopting confocal deconvolution, and finally synthesizing a high-definition amplified video. The method has the advantages that automatic recognition of the vibration area and the frequency band is achieved, and the detection efficiency is improved; the image quality is remarkably improved through deconvolution post-processing, and noise interference is avoided; complex industrial environments are compatible, reliable technical support is provided for non-contact pipeline health monitoring, leakage and explosion accidents are effectively prevented, and the service life of facilities is prolonged.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and non-destructive testing, and in particular to a method for amplifying and detecting micro-vibration motion of a process pipeline based on automatic tracking and deconvolution. Background Art

[0002] Process pipelines, as core transportation facilities in industries like chemical, petroleum, and natural gas, are subject to long-term dynamic loads such as mechanical vibration and flow-induced vibration. These loads can easily lead to damage such as fatigue cracks and loosening of supports, resulting in abnormal vibrations and, in turn, increasing the risk of major accidents such as leaks and explosions. Therefore, efficient and accurate micro-vibration detection of pipelines is key to achieving early damage warnings and extending the life of these facilities.

[0003] Although traditional non-destructive testing technologies (such as radiographic testing, ultrasonic testing, and magnetic particle testing) can identify specific types of defects, they have significant limitations:

[0004] The contradiction between efficiency and cost: X-ray testing has high accuracy but is expensive and poses radiation risks; ultrasonic testing relies on the adaptability of complex structures and is difficult to meet the needs of multiple scenarios; magnetic particle testing is only applicable to ferromagnetic materials and cannot detect internal damage.

[0005] Interference in contact measurement: Traditional sensors will change the inherent vibration characteristics of the pipeline due to the mass loading effect, reducing data reliability.

[0006] Non-contact motion amplification technologies (such as Euler video amplification and phase video motion amplification) provide new solutions to the above problems, but they still have the following defects:

[0007] Manual reliance and noise interference: The region of interest needs to be manually selected, which is highly subjective. The noise is simultaneously enhanced during the amplification process, resulting in motion artifacts and blur, affecting detection accuracy.

[0008] Insufficient scene adaptability: Both the Euler method and the phase method are prone to artifacts. The Euler method is sensitive to lighting, and the phase method is computationally complex and has limited ability to handle non-rigid motion.

[0009] Frequency parameter dependence: The filter frequency band needs to be preset manually, and there is a lack of adaptive recognition of the main vibration frequency, making it difficult to cope with dynamic frequency changes under complex working conditions.

[0010] Therefore, there is an urgent need for a detection method that integrates automatic tracking, adaptive frequency band identification and image enhancement to overcome the shortcomings of existing technologies and realize efficient, accurate and automated detection of micro-vibrations in process pipelines. Summary of the Invention

[0011] In view of the defects of the prior art, the present invention provides a method for amplifying and detecting micro-vibration motion of process pipelines based on automatic tracking and deconvolution.

[0012] In order to achieve the above object of the invention, the technical solution adopted by the present invention is as follows:

[0013] A method for detecting micro-vibration motion amplification in process pipelines based on automatic tracking and deconvolution comprises the following steps:

[0014] Step 1: Use a high-speed camera to capture video of the process pipeline;

[0015] Step 2: Automatically select the abnormal vibration area in the video using the inter-frame difference method;

[0016] Step 3: Use the LK optical flow method to track the motion of the abnormal vibration area, obtain the displacement waveform and perform Fourier transform to determine the main frequency and the specified frequency band range;

[0017] Step 4: performing motion amplification processing on the video according to the specified frequency band range, and outputting the amplified video;

[0018] Step 5: Convert each frame of the amplified video from RGB color space to HSV color space;

[0019] Step 6: Perform confocal deconvolution on the converted image sequence;

[0020] Step 7: Synthesize the processed image sequence to generate an enhanced process pipeline motion magnification video.

[0021] Furthermore, the step 2 specifically includes:

[0022] Convert each color image of the video into a grayscale image, calculate the difference image between adjacent frames and perform binarization processing;

[0023] Perform morphological operations on the binarized image to remove noise and connect regions;

[0024] Extract the contour of the motion area and automatically select the abnormal vibration area with a rectangular frame.

[0025] Furthermore, the morphological operation includes at least one of erosion, dilation, opening operation and closing operation.

[0026] Furthermore, the method for determining the specified frequency band range in step 3 includes: calculating the instantaneous motion speed of each pixel in the vibration abnormality area by the LK optical flow method; performing time integration on the instantaneous motion speed to obtain a displacement waveform; performing Fourier transform on the displacement waveform to generate a spectrum diagram, and selecting the frequency with the highest amplitude as the center frequency f0; with f0 as the center, determining the frequency band range to be [0.8f0, 1.2f0].

[0027] Furthermore, the acquisition of the displacement waveform in step 3 is specifically as follows: averaging the displacement changes of each pixel in the vibration abnormality area to generate a displacement waveform for the entire area.

[0028] Furthermore, the motion magnification processing in step 4 includes: selecting Euler video magnification or phase video motion magnification according to the complexity of the environment; when the environment is simple and fill light is required, Euler video magnification is used; when the environment is complex and high magnification is required, phase video motion magnification is used.

[0029] Furthermore, the conversion formula from RGB to HSV in step 5 is:

[0030]

[0031] Among them, R, G, and B represent the components of red, green, and blue in the RGB color space respectively.

[0032] Normalize the RGB components and calculate the maximum value C max , minimum value C min and the difference Δ;

[0033]

[0034] Based on the above formula, hue H, saturation S, and brightness V are calculated respectively. The formulas are as follows:

[0035]

[0036] V=C max

[0037] Among them, R′=R / 255, G′=G / 255, B′=B / 255.

[0038] Furthermore, the formula for the cofocal deconvolution in step 6 is:

[0039] g=f*h+η

[0040]

[0041] Where g is the observed image, f is the original image, h is the PSF model of the Gaussian point spread function, then the observed image can be expressed as the convolution of the original image and the PSF, * represents the convolution operation, η is the noise term, represents the deconvolution operation, h -1 represents the inverse of the PSF, σ is the standard deviation, which controls the width of the spot, and x and y represent the spatial coordinates of the image plane.

[0042] Furthermore, the point spread function h(x, y) is determined by theoretical derivation through analysis of imaging system characteristics and motion states.

[0043] Furthermore, the motion amplification processing in step 4 includes: decomposing the video into a multi-scale sub-band image sequence using a complex controllable pyramid; filtering and amplitude weighted denoising the phase signal within the specified frequency band; and reconstructing the sub-band after adjusting the phase according to a preset amplification factor to generate an amplified video.

[0044] Compared with the prior art, the advantages of the present invention are:

[0045] 1. Automation and efficiency improvement

[0046] Automatic tracking of abnormal vibration areas and frequency band identification are achieved through the inter-frame difference method and LK optical flow method, avoiding the subjectivity of manual area selection and preset frequency in traditional methods, significantly improving detection efficiency and reducing operational complexity.

[0047] Combined with the adaptive frequency band calculation of the main vibration frequency, it can accurately capture micro-vibration characteristics, reduce invalid frequency band interference, and improve detection sensitivity.

[0048] 2. Image quality optimization

[0049] The introduction of confocal deconvolution technology can effectively suppress blur and artifacts after motion amplification, enhance image resolution and clarity, and improve the visualization of tiny vibration details, making it easier for manual or AI algorithms to identify early damage.

[0050] Through HSV color space conversion, the brightness and chromaticity information are separated, the interference of lighting changes on motion amplification is reduced, and the robustness of the algorithm in complex environments is enhanced.

[0051] 3. Technology integration and enhanced adaptability

[0052] The dual-mode motion amplification technology (Eulerian / phase video amplification) can be flexibly selected according to the complexity of the environment, taking into account both computational efficiency and noise resistance, and expanding the applicability of the method in scenarios such as poor fill lighting conditions and complex backgrounds.

[0053] Combining non-contact detection with image post-processing avoids the mass loading effect of traditional sensors, ensuring accurate and reliable vibration data while reducing hardware deployment costs.

[0054] 4. Engineering practical value

[0055] By shortening processing time, it meets the real-time or near-real-time detection needs of industrial sites and provides technical support for dynamic monitoring of pipeline health status.

[0056] It significantly improves the early warning capability for hidden dangers such as pipeline fatigue cracks and loose supports, reduces the risk of serious accidents such as leakage and explosion, and extends the service life of facilities, with significant economic and social benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a flow chart of a method for amplifying and detecting micro-vibration motion in a process pipeline according to an embodiment of the present invention.

[0058] Figure 2 Schematic diagram of a public source video according to an embodiment of the present invention.

[0059] Figure 3 This is the vibration abnormality area automatically selected by the pre-processing tracking in the method of the embodiment of the present invention.

[0060] Figure 4 It is a displacement and spectrum diagram of the vibration abnormal area automatically selected by pre-processing tracking in the embodiment method of the present invention.

[0061] Figure 5 Schematic diagram of maximum and minimum frames of a video after amplifying a process pipeline video using improved phase video motion amplification according to an embodiment of the present invention, where (a) is a maximum frame and (b) is a minimum frame.

[0062] Figure 6 3. It is a schematic diagram comparing the amplified video of the process pipeline micro-vibration motion amplification detection method of the embodiment of the present invention and the process pipeline video of the existing method, wherein (a) is the existing method and (b) is the method of the present invention. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and examples.

[0064] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting micro-vibration motion amplification in process pipelines based on automatic tracking and deconvolution. Taking a public source video as an example, the image enhancement process for improving motion amplification is the same for different objects being tested, specifically including the following steps:

[0065] Step 1: Use a high-speed camera to capture video of the process pipeline. The video read in the embodiment is as follows: Figure 2 As shown;

[0066] Step 2: Use the inter-frame difference method to automatically select the vibration abnormal area in the collected video;

[0067] Using the inter-frame difference method, each frame of the video is grayscale processed, converting the color image into a grayscale image. A difference operation is then performed on two adjacent grayscale frames to obtain a difference image. This difference image is then binarized to highlight the moving areas in the difference image. Morphological erosion, dilation, opening, and closing operations are then performed on the binarized image to initially remove noise, fill holes, and fill broken connections, thereby obtaining a more accurate moving area. A contour extraction algorithm is used to extract the contours of the moving area in the binarized image. Based on the extracted contours, a rectangular box is automatically selected to frame the moving area, thereby marking the moving target in the area of interest in the original video image.

[0068] The formula of the inter-frame difference method is:

[0069]

[0070] Among them, the nth frame is I n (x, y), the previous frame of the nth frame is I n-1 (x, y), n>1, the frame difference after binarization of the nth frame is J n (x,y), the binarization threshold is M.

[0071] Step 3: Use the LK optical flow method to track the motion of the automatically selected area, obtain the displacement waveform and corresponding main frequency of the automatically selected area, and then automatically obtain the specified frequency band;

[0072] The LK optical flow method is used to calculate the horizontal and vertical movement speed of each pixel in each frame of the automatically selected area of interest in the video; by integrating over the entire duration of the video, the displacement change of each pixel in the area of interest in the video over time is obtained; the average position change data of the pixels in the area is fast Fourier transformed to obtain the frequency spectrum of the area; in the frequency spectrum, the frequency with the highest amplitude is selected as the center frequency f0 of the filter, and the frequency band range of the filter given near f0 is [0.8f0, 1.2f0]. The final schematic diagram of the enlarged video obtained by pre-processing tracking is shown below. Figure 3 As shown in the figure, the displacement waveform and corresponding spectrum of the selected area obtained by pre-processing tracking are as follows Figure 4 shown.

[0073] Step 4: Motion amplification is performed on the process pipeline video captured by the high-speed camera using motion amplification technology. The amplified video is output and stored in AVI format. A complex steerable pyramid is used to decompose the video according to spatial scale, direction, and position, generating subband image sequences at different directions and scales. The phase and amplitude sequences of each pixel in each subband image sequence are then obtained. Using the automatically acquired frequency band [0.8f0, 1.2f0], the phase is independently filtered at each position, direction, and scale. Amplitude-weighted filtering is applied to the phase signal to denoise it, and amplitude-weighted spatial smoothing is applied to increase the signal-to-noise ratio of the phase. The phase is amplified by a preset amplification factor α, for example, set to 75, and added to the phase sequence of each pixel in the subband image sequence to obtain a new phase sequence. New subbands are reconstructed based on the new phase and amplitude sequences of each pixel in each subband, and the subbands are synthesized to produce the motion-amplified video. In this example, we extract the vibration frequency of the process pipeline of the source video. The frequency band range automatically obtained is [0.82, 1.23]. Phase-based motion amplification is performed on it. Finally, the improved phase video motion amplification is used to amplify the maximum and minimum frames of the process pipeline video. Figure 5 Table 1 shows a time comparison between the method of the present invention and the traditional phase motion amplification method. A unified calculation was performed on a laptop with a 4-core CPU and 8GB memory. It can be seen that the processing time of the method of the present invention is significantly shortened.

[0074] Table 1 Comparison of the running time of the method of the present invention and the traditional phase video motion amplification

[0075]

[0076] Step 5: Convert each frame of the amplified video sequence from the original RGB color space to the HSV color space. Assuming that R, G, and B represent the red, green, and blue components in the RGB color space, first normalize them between 0 and 255:

[0077]

[0078] For the normalized value, let:

[0079]

[0080] Based on the above formula, calculate the hue H, saturation S, and brightness V respectively:

[0081]

[0082] V=C max

[0083] Step 6: Perform confocal deconvolution on each frame of the converted video sequence. First, the HSV image sequence is analyzed and calculated using an image processing algorithm, and its specific PSF is determined through theoretical derivation. Then, using this precisely calculated PSF, a deconvolution operation is performed on the image sequence.

[0084] The confocal deconvolution formula equation is:

[0085] g=f*h+η

[0086]

[0087] Where g is the observed image, f is the original image, h is the PSF model of the Gaussian point spread function, then the observed image can be expressed as the convolution of the original image and the PSF, * represents the convolution operation, η is the noise term, represents the deconvolution operation, h -1 represents the inverse of the PSF, σ is the standard deviation, which controls the width of the spot, and x and y represent the spatial coordinates of the image plane.

[0088] Step 7: Synthesize the digital image sequences of each frame to obtain the motion magnified video after the process pipeline image is enhanced ( Figure 6 ). By observing Figure 6 a and Figure 6 As can be seen from FIG. 2 , the shadow portion on the left side of the process pipeline is clearer and the overall visual effect is enhanced, which shows that the method of the present invention has a certain improvement in image enhancement.

[0089] The method according to the present invention described above can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored in a remote recording medium or non-transitory machine-readable medium downloaded via a network and then stored in a local recording medium. Thus, the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It will be understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the process pipeline micro-vibration motion amplification detection method described herein is implemented. Furthermore, when a general-purpose computer accesses the code for implementing the process shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for performing the process shown herein.

[0090] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the implementation methods of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A method for detecting micro-vibration motion amplification in process pipelines based on automatic tracking and deconvolution, characterized in that: The following steps are involved: Step 1: Use a high-speed camera to capture video of the process pipeline; Step 2: Automatically select the abnormal vibration area in the video using the inter-frame difference method; Step 3: Use the LK optical flow method to track the motion of the abnormal vibration area, obtain the displacement waveform and perform Fourier transform to determine the main frequency and the specified frequency band range; Step 4: performing motion amplification processing on the video according to the specified frequency band range, and outputting the amplified video; Step 5: Convert each frame of the amplified video from RGB color space to HSV color space; Step 6: Perform confocal deconvolution on the converted image sequence; Step 7: Synthesize the processed image sequence to generate an enhanced process pipeline motion magnification video.

2. The process pipeline micro-vibration motion amplification detection method according to claim 1 is characterized in that , the step 2 specifically includes: Convert each color image of the video into a grayscale image, calculate the difference image between adjacent frames and perform binarization processing; Perform morphological operations on the binarized image to remove noise and connect regions; Extract the contour of the motion area and automatically select the abnormal vibration area with a rectangular frame.

3. The process pipeline micro-vibration motion amplification detection method according to claim 2, characterized in that: The morphological operation includes at least one of erosion, dilation, opening operation and closing operation.

4. The process pipeline micro-vibration motion amplification detection method according to claim 1, characterized in that: The method for determining the specified frequency band range in step 3 includes: calculating the instantaneous motion speed of each pixel in the vibration abnormal area by the LK optical flow method; performing time integration on the instantaneous motion speed to obtain a displacement waveform; performing Fourier transform on the displacement waveform to generate a spectrum diagram, selecting the frequency with the highest amplitude as the center frequency f0; and determining the frequency band range as [0.8f0, 1.2f0] with f0 as the center.

5. The process pipeline micro-vibration motion amplification detection method according to claim 4, characterized in that: The acquisition of the displacement waveform in step 3 is specifically as follows: averaging the displacement changes of each pixel in the vibration abnormality area to generate a displacement waveform for the entire area.

6. The process pipeline micro-vibration motion amplification detection method according to claim 1, characterized in that: The motion magnification processing in step 4 includes: selecting Euler video magnification or phase video motion magnification according to the complexity of the environment; when the environment is simple and fill light is required, Euler video magnification is used; when the environment is complex and high magnification is required, phase video motion magnification is used.

7. The process pipeline micro-vibration motion amplification detection method according to claim 1, characterized in that: The conversion formula from RGB to HSV in step 5 is: Among them, R, G, and B represent the components of red, green, and blue in the RGB color space respectively; Normalize the RGB components and calculate the maximum value C max , minimum value C min and the difference Δ; Based on the above formula, hue H, saturation S, and brightness V are calculated respectively. The formulas are as follows: V=C max Among them, R′=R / 255, G′=G / 255, B′=B / 255.

8. The process pipeline micro-vibration motion amplification detection method according to claim 1, characterized in that: The formula for the confocal deconvolution in step 6 is: g=f*h+η Where g is the observed image, f is the original image, h is the PSF model of the Gaussian point spread function, then the observed image can be expressed as the convolution of the original image and the PSF, * represents the convolution operation, η is the noise term, represents the deconvolution operation, h -1 represents the inverse of the PSF, σ is the standard deviation, which controls the width of the spot, and x and y represent the spatial coordinates of the image plane.

9. The process pipeline micro-vibration motion amplification detection method according to claim 8, characterized in that: The point spread function h(x, y) is determined by theoretical derivation through analysis of imaging system characteristics and motion state.

10. The process pipeline micro-vibration motion amplification detection method according to claim 6, characterized in that: The motion amplification process includes: decomposing a video into a multi-scale sub-band image sequence using a complex controllable pyramid; filtering and amplitude-weighted denoising of a phase signal within the specified frequency band; and reconstructing the sub-band after adjusting the phase according to a preset amplification factor to generate an amplified video.

Images