A method and apparatus for fault detection based on motion amplification

Abstract

Embodiments of the present application provide a fault detection method and device based on motion amplification, and relate to the field of fault detection. The fault detection method comprises: acquiring a video signal and obtaining continuous multiple frames of original image signals containing a fault detection object and corresponding digital image signals from the video signal; obtaining full-field vibration displacement information of the fault detection object after image multi-scale analysis; extracting phase information from the full-field vibration displacement information based on a preset amplification multiple and a preset modal order, replacing local phase information of the corresponding digital image signals, and obtaining image information of each frame of digital image signals; and synthesizing each frame of digital image signals with corresponding image information to obtain continuous multiple frames of digital image signals that are motion amplified. The present application performs conversion, analysis, amplification, synthesis and other processing means on the original image signals after the video signal is acquired, and obtains motion amplified images in which the fault of a component can be identified by the naked eye, so that fault detection is convenient and automated.

Description

Technical Field

[0001] This invention relates to the field of fault detection technology, specifically to a fault detection method and device based on motion amplification. Background Technology

[0002] The development of society and the economy is inseparable from various large-scale manufacturing equipment. In order to improve efficiency, most of these machines operate continuously from the start of production, which often leads to malfunctions such as equipment aging, fatigue damage to parts, and loosening of fasteners. These malfunctions can range from minor issues like hindering equipment operation or the normal use of basic functions, resulting in low production efficiency and inconvenience to social order and people's lives, to more serious issues like causing safety accidents. For example, the Challenger space shuttle exploded and disintegrated due to the loss of elasticity in its rubber O-rings, causing significant economic losses and casualties.

[0003] Fault monitoring of component structures is of great significance for the safe use of infrastructure and even the prevention of major safety accidents. Condition monitoring is also necessary for mechanical systems under minute vibrations to prevent unnecessary accidents. Traditional measurement methods include contact and non-contact methods. Contact measurements require the installation of sensing elements, such as displacement sensors and accelerometers, on the structure being measured. This preprocessing requires a large amount of labor, and the sensors attached to the component structure increase structural mass, inevitably altering the vibration characteristics and reducing measurement accuracy. While non-contact measurements avoid this problem—for example, laser vibrometers and vision-based motion methods (image correlation, pattern matching, point trajectory tracking)—these methods cannot visually identify structural faults. The advent of motion amplification technology allows even extremely minute movements to be directly identified visually after amplification, greatly improving monitoring efficiency. However, current motion amplification equipment is bulky, inefficient, and expensive.

[0004] The core technical challenge in applying motion amplification technology to component structural fault monitoring lies in achieving a portable, efficient, and low-cost fault detection device after considering the above factors. Based on these technical challenges, the applicant has proposed the technical solution in this application. Summary of the Invention

[0005] The purpose of this invention is to provide a motion-magnified fault detection method and device. By acquiring video signals of the motion environment of the fault detection object, the original image signals are converted, analyzed, amplified, and synthesized to obtain a motion-magnified image in which the fault can be identified by the naked eye.

[0006] To achieve the above objectives, the present invention provides a fault detection method based on motion amplification, comprising: acquiring a series of consecutive original image signals containing the fault detection object from a video signal of the acquired fault detection object, and acquiring a series of consecutive digital image signals corresponding to the series of consecutive original image signals; performing multi-scale image analysis based on the series of consecutive digital image signals to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signal; extracting the phase information of the fault detection object from the full-field vibration displacement information of each frame of the digital image signal based on a preset amplification factor and a preset modal order, and replacing the local phase information of the corresponding digital image signal with the phase information of the fault detection object in each frame of the digital image signal to obtain image information of each frame of the digital image signal; and synthesizing each frame of the digital image signal with the corresponding image information to obtain a series of consecutive digital image signals of the fault detection object after motion amplification.

[0007] The present invention also provides a fault detection device based on motion magnification, comprising: an image acquisition unit for acquiring video signals containing a fault detection object; an image conversion unit for acquiring multiple consecutive frames of original image signals containing the fault detection object from the video signals, and acquiring multiple consecutive frames of digital image signals corresponding to the multiple consecutive frames of original image signals; a signal analysis unit for performing multi-scale image analysis based on the multiple consecutive frames of digital image signals to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signals; a motion magnification unit for extracting the phase information of the fault detection object from the full-field vibration displacement information of each frame of the digital image signals based on a preset magnification factor and a preset modal order, and replacing the local phase information of the corresponding digital image signals with the phase information of the fault detection object in each frame of the digital image signals to obtain image information of each frame of the digital image signals; and an image synthesis unit for synthesizing each frame of the digital image signals with the corresponding image information to obtain multiple consecutive frames of digital image signals of the fault detection object after motion magnification.

[0008] In this embodiment of the invention, a motion-amplified fault detection method is applied to a portable head-mounted glasses. After receiving input parameters for amplification and frequency bandwidth, the system automatically begins acquiring video signals containing the fault detection object. It then obtains the digital image signal corresponding to the original image signal and performs analysis, amplification, and synthesis on the digital image signal to form a continuous multi-frame digital image signal with motion amplification, which can be visually identified by the naked eye. This signal is displayed on the two display frames of the head-mounted glasses for the inspector to intuitively perceive, making fault detection portable and automated. The entire process from video signal acquisition to motion amplification requires no manual intervention, making fault detection convenient and simple. The digital image signal processing incorporates motion structural feature extraction, micro-motion methods, and specific frequency band selection, further improving the fault detection efficiency and ensuring high accuracy and reliability.

[0009] In one embodiment, extracting the phase information of the fault detection object from the full-field vibration displacement information of each frame of the digital image signal based on a preset amplification factor and a preset modal order includes: performing blind source separation on the full-field vibration displacement information of each frame of the digital image signal to obtain displacement information of several single-frequency band components; selecting the displacement information of the single-frequency band component corresponding to the preset modal order from the displacement information of the several single-frequency band components as the displacement signal to be amplified; amplifying the displacement signal to be amplified using the preset amplification factor; and extracting the phase information of the fault detection object from the amplified displacement signal to be amplified.

[0010] In one embodiment, before amplifying the displacement signal to be amplified using the preset amplification factor, the motion amplification-based fault detection method further includes: performing a Fourier transform on the displacement signal to be amplified to obtain the frequency of the displacement signal to be amplified, and selecting a bandpass filter with the frequency as the center frequency to filter the displacement signal to be amplified; the amplification of the displacement signal to be amplified using the preset amplification factor includes: amplifying the filtered displacement signal to be amplified using the preset amplification factor.

[0011] In one embodiment, the step of performing multi-scale image analysis based on the continuous multi-frame digital image signals to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signals includes: performing multi-scale image analysis on the continuous multi-frame digital image signals to obtain the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signals; and obtaining the full-field vibration displacement information of the fault detection object in each frame of the digital image signals based on the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signals.

[0012] In one embodiment, obtaining the consecutive multi-frame digital image signals corresponding to the consecutive multi-frame original image signals includes: for each frame of the original image signal, obtaining the pixel YIQ value of the original image signal as the digital image signal corresponding to the original image signal.

[0013] In one embodiment, before acquiring a series of original image signals containing the fault detection object from the acquired video signal of the fault detection object, the motion magnification-based fault detection method further includes: identifying the fault detection object from the acquired reference image, and after adjusting the image acquisition range with the fault detection object as the center position of the reference image, acquiring the video signal containing the fault detection object.

[0014] In one embodiment, the method for performing multi-scale image analysis on consecutive multi-frame digital image signals is: multi-scale image analysis based on dual complex wavelet transform.

[0015] In one embodiment, the motion-magnified fault detection device further includes an image display unit for displaying a series of digital image signals of the fault detection object after motion magnification.

[0016] In one embodiment, the motion amplification unit is further configured to perform blind source separation on the full-field vibration displacement information of each frame of the digital image signal to obtain displacement information of several single-frequency band components, and select the displacement information of the single-frequency band component corresponding to the preset modal order from the displacement information of the several single-frequency band components as the displacement signal to be amplified; the motion amplification unit is further configured to amplify the displacement signal to be amplified using the preset amplification factor, and extract the phase information of the fault detection object from the amplified displacement signal to be amplified.

[0017] In one embodiment, the motion amplification unit is further configured to perform a Fourier transform on the displacement signal to be amplified to obtain the frequency of the displacement signal to be amplified, select a bandpass filter with the frequency as the center frequency to filter the displacement signal to be amplified, and amplify the filtered displacement signal to be amplified using a preset amplification factor.

[0018] In one embodiment, the signal analysis unit is further configured to perform multi-scale image analysis on the continuous multi-frame digital image signals to obtain the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signals; the signal analysis unit is further configured to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signals based on the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signals.

[0019] In one embodiment, the image conversion unit is used to obtain the pixel YIQ values ​​of the original image signal for each frame of the original image signal as the digital image signal corresponding to the original image signal.

[0020] In one embodiment, the motion-amplified fault detection device further includes: a control input unit, configured to identify a fault detection object from a reference image acquired by the image acquisition unit, adjust the image acquisition range of the image acquisition unit with the fault detection object as the center position of the reference image, and control the image acquisition unit to acquire a video signal containing the fault detection object.

[0021] The fault detection device is a head-mounted glasses, and the image display unit includes two displays, which are the two lenses of the head-mounted glasses. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a motion-amplified fault detection device used in the motion-amplified fault detection method according to the first embodiment of the present invention.

[0023] Figure 2 This is a detailed flowchart of the fault detection method based on motion amplification according to the first embodiment of the present invention;

[0024] Figure 3 yes Figure 2 The detailed flowchart of step 103 in the motion amplification-based fault detection method;

[0025] Figure 4 This is a schematic diagram of the original image of the fault detection object according to the first embodiment of the present invention;

[0026] Figure 5 According to the first embodiment of the present invention, Figure 4 A schematic diagram of the magnified image after the original image is magnified at different frequency bandwidths;

[0027] Figure 6 This is a schematic diagram of a head-mounted glasses as the motion-amplified fault detection device according to the second embodiment of the present invention. Detailed Implementation

[0028] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings to provide a clearer understanding of the purpose, features, and advantages of the present invention. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of the present invention, but are merely illustrative of the essential spirit of the technical solution of the present invention.

[0029] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known apparatuses, structures, and techniques associated with this application may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0030] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.

[0031] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0032] The singular forms “a” and “the” used in this specification and the appended claims include plural references unless otherwise expressly stated herein. It should be noted that the term “or” is generally used to include the meaning of “or / and” unless otherwise expressly stated herein.

[0033] In the following description, in order to clearly demonstrate the structure and working method of the present invention, a number of directional terms will be used. However, terms such as "front", "back", "left", "right", "outside", "inside", "outward", "inward", "up", and "down" should be understood as convenient terms and not as limiting terms.

[0034] The first embodiment of the present invention relates to a fault detection method based on motion amplification, which is applied to a fault detection device based on motion amplification and is used to detect faults in the structure of components.

[0035] like Figure 1As shown, the motion-amplified fault detection device includes an image acquisition unit 11, a control input unit 12, an image conversion unit 13, a signal analysis unit 14, a motion amplification unit 15, an image synthesis unit 16, an image display unit 17, and a power supply unit 18. The image acquisition unit 11 is connected to the control input unit 12, the image conversion unit 13, and the image synthesis unit 16. The control input unit 12 is connected to the image acquisition unit 11, the signal analysis unit 14, and the motion amplification unit 15. The image conversion unit 13 is connected to the image acquisition unit 11 and the signal analysis unit 14. The signal analysis unit 14 is connected to the control input unit 12, the image conversion unit 13, and the motion amplification unit 15. The motion amplification unit 15 is connected to the signal analysis unit 14 and the image synthesis unit 16. The image synthesis unit 16 is connected to the motion amplification unit 15, the control input unit 12, and the image display unit 17. The image display unit 17 is connected to the image acquisition unit 11 and the image synthesis unit 16. The power supply unit 18 supplies power to all the above units.

[0036] The specific process of the motion amplification-based fault detection method in this embodiment is as follows: Figure 2 As shown.

[0037] Step 101: Identify the fault detection object from the acquired reference image, and adjust the image acquisition range with the fault detection object as the center position of the reference image.

[0038] Specifically, taking a component as an example, when acquiring video signals containing the component, it is necessary to keep the component at the center of the video signal acquisition range to facilitate observation of the component's fault image. After the motion-amplified fault detection device is worn, the motion video information of the component is acquired by the image acquisition unit 11 and displayed as an image by the image display unit 17. The user moves the component to center it in the image display, such as... Figure 4 As shown.

[0039] Step 102: Obtain multiple consecutive frames of original image signals containing the fault detection object from the video signal of the collected fault detection object, and obtain multiple consecutive frames of digital image signals corresponding to the multiple consecutive frames of original image signals.

[0040] Specifically, by controlling the input unit 12 to start video signal acquisition, a series of consecutive multi-frame raw image signals are acquired. For each frame of the raw image signal, the pixel YIQ value of the raw image signal is obtained, and the luminance signal in the first channel Y of the pixel YIQ value of the raw image signal is used as the corresponding digital image signal. The second channel I and the third channel Q of the pixel YIQ value of the raw image signal are not used in this step, but will be used in the subsequent image synthesis step. In this step, they can be temporarily stored in the digital image signal and not used. Only the luminance signal in the first channel Y is used. The video signal is converted by the image conversion unit 13 into the raw image signal and the digital image signal represented by the pixel YIQ value. Then, the converted digital image signal is sent to the signal analysis unit 14 for analysis.

[0041] Step 103: Perform multi-scale image analysis based on the continuous multi-frame digital image signals to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signals.

[0042] In one example, please refer to Figure 3 Step 103 includes the following sub-steps:

[0043] Sub-step 1031 involves performing multi-scale image analysis on the continuous multi-frame digital image signals to obtain the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signals.

[0044] Sub-step 1032: Based on the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signal, obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signal.

[0045] Specifically, multi-scale image analysis is performed using dual complex wavelets to obtain complex domain image information at different directions and multiple scales, including local amplitude and local phase. The difference in local phase between image frames is then calculated to obtain the local phase difference at each scale. Spatial differentiation of the two-dimensional discrete local phase is performed to obtain the spatial frequency at the corresponding scale. Finally, the ratio of the phase difference to the spatial frequency at each scale is used to obtain the local motion at the corresponding scale, thus obtaining the full-field vibration displacement information of the fault detection object. Since this embodiment obtains the physical relationship between phase and motion from an Euler perspective, the full-field vibration displacement of the fault detection object can be obtained. The full-field vibration displacement information reflects the minute motion changes of the fault detection object under working conditions.

[0046] Step 104: For each frame of the digital image signal, perform blind source separation on the full-field vibration displacement information of the digital image signal to obtain displacement information of several single-frequency band components, and select the displacement information of the single-frequency band component corresponding to the preset mode order from the displacement information of the several single-frequency band components as the displacement signal to be amplified.

[0047] Specifically, blind source separation first employs principal component analysis to reduce the order of the full-field vibration displacement information, thereby reducing data dimensionality and retaining vibration information with strong modal correlation. Secondly, to identify closely spaced and highly damped modes with minimal manual supervision and parameter adjustment, a complexity tracking algorithm is used to perform blind source separation on the reduced-order full-field vibration displacement information to obtain single-frequency component displacement information. Furthermore, this algorithm does not require structural surface preprocessing and can be implemented in a relatively efficient and autonomous manner. From the single-frequency component displacement information, the displacement information corresponding to the predefined preset modal order is selected as the displacement signal to be amplified.

[0048] Step 105: Filter the displacement signal to be amplified and then amplify it. Extract the phase information of the fault detection object from the amplified displacement signal. Replace the local phase information of the corresponding digital image signal with the phase information of the fault detection object in each frame of the digital image signal to obtain the image information of each frame of the digital image signal.

[0049] Specifically, the vibration signal after blind source separation carries modal information of single-frequency components. The corresponding natural frequencies are obtained through Fourier transform. A bandpass filter is then selected using these natural frequencies as the center frequency to further filter out noise frequencies surrounding the natural frequencies. The Fourier transform converts the time-domain vibration signal into a frequency-domain vibration signal, thus obtaining the frequency of the displacement signal. In simpler terms, this means the structure reciprocates a certain number of times per second; for example, 1 Hz means the structure reciprocates once per second. Hertz (Hz) is the unit of frequency. The processed vibration signal is then linearly amplified using a predefined amplification factor, and a Hilbert transform is performed to obtain the local phase of the amplified vibration signal. Finally, the local phase of the source signal is replaced to obtain the image information of each frame of digital image signal, which can be used for further image synthesis.

[0050] Step 106: Combine the digital image signals of each frame with the corresponding image information to obtain a series of digital image signals of the fault detection object after motion magnification.

[0051] Specifically, the image information obtained in step 105 is combined with the other two channels of the three-channel digital image signal (pixel YIQ value) and reconstituted into an RGB signal playable on the display. The motion amplification signal processed by the motion amplification unit 15 is then converted into an amplified image, and images from each different image channel are combined to obtain a motion amplified image of the component, such as... Figure 5 As shown, with the same magnification, the magnified images of the motion of components exhibit different changes at different frequency bandwidths. The inherent frequency and frequency bandwidth mentioned in this embodiment are frequency parameters corresponding to the time domain. After the magnified motion images are synthesized to form visualized image information, they are displayed visually by the image display unit 17. Users can determine whether there are potential faults in the components by observing the magnified motion deformation.

[0052] A second embodiment of the present invention relates to a fault detection device based on motion amplification, comprising: a control input unit 12, configured to identify a fault detection object from a reference image acquired by an image acquisition unit, adjust the image acquisition range of the image acquisition unit with the fault detection object as the center position of the reference image, and control the image acquisition unit to acquire a video signal containing the fault detection object; and to receive the set motion amplification motion vibration parameters, the motion vibration parameters including at least amplification factor and frequency bandwidth; an image acquisition unit 11, configured to acquire a video signal containing the fault detection object; and an image conversion unit 13, configured to obtain multiple consecutive frames containing the fault detection object from the video signal. The system first obtains an initial image signal and acquires consecutive multi-frame digital image signals corresponding to the consecutive multi-frame original image signals. The signal analysis unit 14 is used to perform multi-scale image analysis based on the consecutive multi-frame digital image signals to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signal. The signal analysis unit 14 is also used to perform multi-scale image analysis on the consecutive multi-frame digital image signals to obtain the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signal. The signal analysis unit 14 is also used to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signal based on the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signal. The motion amplification unit 15 is used to extract the phase information of the fault detection object from the full-field vibration displacement information of each frame of the digital image signal based on a preset amplification factor and a preset modal order, and to replace the local phase information of the corresponding digital image signal with the phase information of the fault detection object in each frame of the digital image signal to obtain the image information of each frame of the digital image signal; the motion amplification unit 15 is also used to perform blind source separation on the full-field vibration displacement information of the digital image signal for each frame of the digital image signal to obtain the displacement information of several single-frequency band components, and to extract the displacement information of the several single-frequency band components from the displacement information of the digital image signal. The displacement information of the single-band component corresponding to the preset modal order is selected as the displacement signal to be amplified; the motion amplification unit 15 is also used to amplify the displacement signal to be amplified using the preset amplification factor, and extract the phase information of the fault detection object from the amplified displacement signal to be amplified; the motion amplification unit 15 is also used to perform Fourier transform on the displacement signal to be amplified to obtain the frequency of the displacement signal to be amplified, select a bandpass filter with the frequency as the center frequency to filter the displacement signal to be amplified, and amplify the filtered displacement signal to be amplified using the preset amplification factor.The image synthesis unit 16 is used to acquire the pixel YIQ value of the original image signal for each frame of the original image signal as the corresponding digital image signal; the image display unit 17 is used to display multiple consecutive frames of original image signals, as well as multiple consecutive frames of digital image signals after the fault detection object has been motion-amplified; the power supply unit 18 is used to supply power to the image acquisition unit, image conversion unit, signal analysis unit, motion amplification unit, and image synthesis unit.

[0053] like Figure 1 As shown, the image acquisition unit 11 is connected to the control input unit 12, the image conversion unit 13, and the image synthesis unit 16, respectively. The control input unit 12 is connected to the image acquisition unit 11, the signal analysis unit 14, and the motion amplification unit 15, respectively. The image conversion unit 13 is connected to the image acquisition unit 11 and the signal analysis unit 14, respectively. The signal analysis unit 14 is connected to the control input unit 12, the image conversion unit 13, and the motion amplification unit 15, respectively. The motion amplification unit 15 is connected to the signal analysis unit 14 and the image synthesis unit 16, respectively. The image synthesis unit 16 is connected to the motion amplification unit 15 and the image display unit 17, respectively. The image display unit 17 is connected to the image acquisition unit 11 and the image synthesis unit 16, respectively. The power supply unit 18 supplies power to all the above units.

[0054] In one example, motion-amplified fault detection equipment is used on portable head-mounted glasses, such as... Figure 6 As shown, the image acquisition unit 11 is positioned at the front of the head-mounted glasses in the form of a miniature camera for video acquisition. The image synthesis unit 16 consists of two Micro-LED displays, each mounted on one of the lenses of the head-mounted glasses. The control input unit 12 and the image conversion unit 13 are integrated on one temple of the head-mounted glasses, while the signal analysis unit 14, the motion amplification unit 15, and the power supply unit 18 are integrated on the other temple of the head-mounted glasses. Users only need to wear the head-mounted glasses to perform component fault detection, making it convenient to use and portable.

[0055] Since the first embodiment corresponds to this embodiment, this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment remain valid in this embodiment, and the technical effects achievable in the first embodiment can also be achieved in this embodiment. To reduce repetition, they will not be repeated here. Correspondingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0056] The preferred embodiments of the present invention have been described in detail above, but it should be understood that, if necessary, aspects of the embodiments can be modified to utilize aspects, features, and concepts from various patents, applications, and publications to provide other embodiments.

[0057] In light of the detailed description above, these and other changes can be made to the embodiments. Generally, the terminology used in the claims should not be considered limited to the specific embodiments disclosed in the specification and claims, but should be understood to include all possible embodiments together with the full scope of equivalents enjoyed by these claims.

Claims

1. A motion amplification based fault detection method, characterized in that, Applied to head-mounted glasses devices, the method includes: Obtain multiple consecutive frames of original image signals containing the fault detection object from the video signal of the fault detection object, and obtain multiple consecutive frames of digital image signals corresponding to the multiple consecutive frames of original image signals. Based on the continuous multi-frame digital image signals, multi-scale image analysis is performed to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signals. Based on a preset magnification factor and a preset modal order, the phase information of the fault detection object is extracted from the full-field vibration displacement information of the digital image signal in each frame, and the phase information of the fault detection object in each frame of the digital image signal is used to replace the local phase information of the corresponding digital image signal to obtain the image information of the digital image signal in each frame. Each frame of the digital image signal is combined with the corresponding image information to obtain a series of consecutive frames of digital image signals of the fault detection object after motion magnification; wherein, the head-mounted glasses device displays the series of consecutive frames of digital image signals of the fault detection object after motion magnification. The step of performing multi-scale image analysis based on the continuous multi-frame digital image signals to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signals includes: Multi-scale image analysis is performed on the continuous multi-frame digital image signals to obtain the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signal; wherein, the spatial frequency at each scale is obtained by spatially differentiating the local phase of each frame of the digital image signal. Based on the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signal, the full-field vibration displacement information of the fault detection object in each frame of the digital image signal is obtained; wherein, the ratio of the local phase difference to the spatial frequency at each scale is used to obtain the local motion at the corresponding scale, and the local motion at each scale in each frame of the digital image signal is fused to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signal.

2. The motion-amplified fault detection method of claim 1, wherein, The extraction of phase information of the fault detection object from the full-field vibration displacement information of each frame of the digital image signal based on a preset magnification factor and a preset modal order includes: For each frame of the digital image signal, blind source separation is performed on the full-field vibration displacement information of the digital image signal to obtain displacement information of several single-frequency band components. The displacement information of the single-frequency band component corresponding to the preset mode order is selected from the displacement information of the several single-frequency band components as the displacement signal to be amplified. The displacement signal to be amplified is amplified using the preset amplification factor, and the phase information of the fault detection object is extracted from the amplified displacement signal to be amplified.

3. The motion amplification based fault detection method of claim 2, wherein, Before amplifying the displacement signal to be amplified using the preset amplification factor, the method further includes: The frequency of the displacement signal to be amplified is obtained by performing a Fourier transform on the displacement signal to be amplified, and a bandpass filter is selected with the frequency as the center frequency to filter the displacement signal to be amplified. Amplifying the displacement signal to be amplified using a preset amplification factor includes: The filtered displacement signal to be amplified is amplified using a preset amplification factor.

4. The motion-amplified fault detection method of claim 1, wherein, The step of acquiring the consecutive multi-frame digital image signals corresponding to the consecutive multi-frame original image signals includes: For each frame of the original image signal, the pixel YIQ value of the original image signal is obtained as the digital image signal corresponding to the original image signal.

5. The motion amplification based fault detection method of claim 1, wherein, Before obtaining the continuous multi-frame raw image signal containing the fault detection object from the acquired video signal of the fault detection object, the method further includes: The fault detection object is identified from the acquired reference image, and after adjusting the image acquisition range with the fault detection object as the center position of the reference image, the video signal containing the fault detection object is acquired.

6. The motion-amplified fault detection method of claim 1, wherein, The method for performing multi-scale image analysis on the continuous multi-frame digital image signal is: multi-scale image analysis based on dual-tree complex wavelet transform.

7. A motion amplification based fault detection apparatus, characterized by, The fault detection device is a head-mounted glasses device, including: The image acquisition unit is used to acquire video signals containing the fault detection object; An image conversion unit is used to acquire a series of consecutive original image signals containing the fault detection object from the video signal, and to acquire a series of consecutive digital image signals corresponding to the series of consecutive original image signals. The signal analysis unit is used to perform multi-scale image analysis based on the continuous multi-frame digital image signals to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signals. The motion amplification unit is used to extract the phase information of the fault detection object from the full-field vibration displacement information of each frame of the digital image signal based on a preset amplification factor and a preset modal order, and to replace the local phase information of the corresponding digital image signal with the phase information of the fault detection object in each frame of the digital image signal to obtain the image information of each frame of the digital image signal. An image synthesis unit is used to synthesize each frame of the digital image signal with the corresponding image information to obtain a series of consecutive frames of digital image signals after the fault detection object has been motion-magnified; the head-mounted glasses device displays the series of consecutive frames of digital image signals after the fault detection object has been motion-magnified. The signal analysis unit is further configured to perform multi-scale image analysis on the continuous multi-frame digital image signals to obtain the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signals; wherein, the spatial frequency at each scale is obtained by spatially differentiating the local phase of each frame of the digital image signals. The signal analysis unit is further configured to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signal based on the local phase difference and spatial frequency of the fault detection object at each scale in each frame of the digital image signal; wherein, the ratio of the local phase difference to the spatial frequency at each scale is used to obtain the local motion at the corresponding scale, and the local motion at each scale in each frame of the digital image signal is fused to obtain the full-field vibration displacement information of the fault detection object in each frame of the digital image signal.

8. The motion-amplified fault detection apparatus of claim 7, wherein, The motion amplification unit is further configured to perform blind source separation on the full-field vibration displacement information of the digital image signal for each frame of the digital image signal, to obtain displacement information of several single-frequency band components, and select the displacement information of the single-frequency band component corresponding to the preset mode order from the displacement information of the several single-frequency band components as the displacement signal to be amplified. The motion amplification unit is also used to amplify the displacement signal to be amplified using the preset amplification factor, and to extract the phase information of the fault detection object from the amplified displacement signal to be amplified; The motion amplification unit is further configured to perform a Fourier transform on the displacement signal to be amplified to obtain the frequency of the displacement signal to be amplified, select a bandpass filter with the frequency as the center frequency to filter the displacement signal to be amplified, and amplify the filtered displacement signal to be amplified using a preset amplification factor.

Images