Full-automatic cable vibration high-precision measurement method based on machine vision technology

Abstract

The invention discloses a full-automatic cable vibration high-precision measurement method based on a machine vision technology, and relates to the technical field of structural vibration measurement. The method comprises the following steps: installing and calibrating equipment, and ensuring that a cable plane coincides with an imaging plane; high-frame-rate video acquisition is carried out, and comparison verification is carried out in combination with an accelerometer; screening effective pixels based on the pixel intensity time sequence variance, and constructing a pixel intensity space-time matrix by using the time sequence of the pixels; sVD decomposition is adopted to extract an accumulated energy ratio gt; 90% of the main modes are reconstructed, and real signals are revealed; a plurality of inhaul cables are automatically positioned and coded through gradient features and Hough transform; sub-pixel-level vertical displacement is calculated by applying an optical flow method, and median filtering is combined to resist noise; fourier transform is carried out to obtain a spectrogram, and a fundamental frequency is identified; and calculating cable force based on a string pulling theorem. According to the method, remote, full-field and multi-target synchronous monitoring is realized, the displacement measurement precision reaches a sub-pixel level, and a high-precision and full-automatic solution is provided for health monitoring of structures such as bridges.

Description

Technical Field

[0001] The present invention belongs to the technical field of structural vibration measurement, and particularly relates to a high-precision full-automatic cable vibration measurement method based on machine vision technology. Background Art

[0002] In engineering structures, accurately measuring cable vibration is crucial, especially in applications such as bridges, towers, and large-span roofs. The change of cable vibration directly affects the safety and stability of the structure. The most commonly used current cable vibration measurement method is mainly the accelerometer, but this method often has disadvantages such as complex installation, high cost, sparse measuring points, and insufficient measurement accuracy for low-frequency vibration. The emerging cable vibration measurement technology based on machine vision, using high-precision cameras and image processing algorithms, can achieve non-contact monitoring of cable vibration. This technology realizes the vibration measurement of the cable by accurately tracking and positioning the targets arranged on the cable, and can further obtain cable force data. However, most of the current cable vibration measurement methods based on machine vision rely heavily on the targets arranged on the cable, and the number and quality of the targets will directly affect the measurement accuracy and the number of measuring points. At the same time, in some application scenarios, such as bridge cable vibration measurement, the camera is usually arranged at a position far from the cable, which directly leads to the measurement accuracy not meeting the requirements.

[0003] Combined with the actual engineering requirements, when using a camera for non-contact measurement of cable vibration and cable force, overcoming the multi-cable synchronous full-field high-precision displacement measurement in the case of no targets is the key to achieving high-precision measurement of cable vibration and cable force. The present invention breaks through the accuracy, efficiency, and multi-target monitoring bottlenecks of traditional methods through the deep integration of machine vision and spatio-temporal signal processing, and provides a high-precision and full-automatic solution for large-scale structural vibration analysis. Summary of the Invention

[0004] In order to overcome the above-mentioned disadvantages and deficiencies of the prior art, the present invention adopts the following technical solutions: A high-precision full-automatic cable vibration measurement method based on machine vision technology has the following process: Equipment installation: Adjust the viewing angle and focal length of a monocular or binocular camera to make the cable vibration plane coincide with the camera imaging plane, and fix the camera at a stable position; Image acquisition: Shoot the cable vibration video at a frame rate greater than or equal to 100 frames per second, and record the pixel intensity time history data; Effective pixel screening: Screen the effective pixels by calculating the variance of the pixel intensity time history in the target area, and construct a pixel intensity spatio-temporal matrix; Ultra-sensitive calculation: Perform singular value decomposition on the pixel intensity spatio-temporal matrix, select the first k modes with a singular value cumulative energy ratio > 90% to reconstruct the matrix, and reveal the true vibration signal; Cable positioning coding: Based on the pixel intensity gradient of a static reference image, the straight-line feature of the cable is located through Hough transform or morphological operations, and a unique code is assigned and the physical parameters are stored; Vibration displacement calculation: Using the optical flow method, combined with direction constraints and anti-noise processing, calculate the time history curve of the vibration displacement of each cable; Frequency calculation: Perform Fourier transform on the time history of the vibration displacement, and extract the fundamental frequency through the harmonic identification algorithm; Cable force calculation: Based on the chord theorem, combined with the fundamental frequency, cable length, and unit mass, calculate the cable force.

[0005] Preferably, during equipment installation, the camera focal length is determined according to the actual distance between the camera and the cable to be measured, and the image signal bit depth is not less than 8 bits; During installation, adjust the lens direction through a tripod to make the vibration plane of the cable perpendicular to the camera optical axis, ensuring that the imaging resolution ≥ 1920 × 1080 pixels.

[0006] Preferably, the effective pixel screening is specifically: Select the target area in the captured video, and screen the effective pixels by calculating the pixel intensity variance of the pixels in the selected area within a set time period ; The pixel intensity variance formula is: ; where is the number of video frames captured within the time period; is the mean value of the pixel intensity within this time period; is the pixel intensity of the i-th frame within this time period; The larger the variance, the more significant the fluctuation of the pixel intensity over time; Construct the time history of the pixel intensity of the effective pixels into a pixel intensity spatio-temporal matrix , its size is , where and are the spatial pixel number and time sampling number of the spatio-temporal pixel intensity matrix respectively.

[0007] Preferably, in singular value decomposition, the singular value decomposition expression is: ; , where , are unitary matrices, representing the spatial mode and time mode respectively; and are unitary matrices that retain the first k modes; is the diagonal matrix of singular values; is the reconstructed pixel intensity matrix; k is the first k modes retained during reconstruction, and it is required to satisfy that the cumulative energy ratio > 90%, which is used to reveal the true vibration signal.

[0008] Preferably, the cable positioning coding, specifically: S51. Obtain data, including the first target area image and the pixel intensity gradient. The first target area image is the first frame image of the selected effective area. The pixel intensity gradient is calculated by an edge detection algorithm to obtain the gradient amplitude and direction of each pixel in the image, reflecting the severity and direction of the brightness change. S52. Automatically determine the cable position according to the gradient distribution, including gradient feature extraction. Use the Hough transform or morphological operations to identify the linear features of the cables from the high-gradient areas, parameterize the detected lines, and record the geometric positions of each cable in the image.

[0009] S53. Cable coding. According to the positions of the cables, assign a unique number to each cable (Cable 1, Cable 2, Cable 3, Cable 4... Cable n), and additionally store the physical parameters of the cables, including the preset cable length and unit mass.

[0010] Preferably, the vibration displacement calculation is based on the optical flow constraint equation: ; where is the spatial gradient, reflecting the change rate of pixel brightness in the x and y directions and used to detect the spatial displacement trend of cable vibration; is the time change rate of pixel brightness, directly related to the displacement of cable vibration; is the pixel velocity, and the vibration displacement is solved through the constraint equation. For vertical vibration, only the velocity in the y direction is calculated when u = 0, which is simplified to: ; Eliminate the pixels with gradient values less than the threshold, use median filtering to eliminate outliers, and calculate the average vibration displacement time history of cable i by grouping according to the cable coding : .

[0011] Preferably, the frequency calculation process is as follows: Perform Fourier transform on the calculated cable vibration time history ; Obtain the frequency spectrum diagram of cable vibration, and identify the fundamental frequency of cable vibration through the harmonic frequency identification algorithm ; The fundamental frequency is the frequency component with the largest energy in the frequency spectrum; is the frequency-domain spectrum; is the angular frequency.

[0012] Preferably, the cable force calculation formula is: where is the cable force, is the mass per unit length, is the cable length, is the fundamental frequency; and verify the measurement results by comparing with the accelerometers arranged on the bridge cables.

[0013] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are as follows: 1. Through binocular cameras and high-frame-rate video acquisition, the present invention can complete vibration monitoring without physically contacting the cable, avoiding interference to the structure or installation limitations of traditional contact sensors (such as accelerometers); combined with SVD spatio-temporal weighted reconstruction, optical flow method, and edge detection algorithms, it can achieve vibration displacement measurement accuracy at the sub-micron level, improving the reliability of cable force calculation; through anti-noise processing such as pixel intensity variance screening and median filtering, the influence of light changes, background interference, etc. on the measurement results can be reduced.

[0014] 2. The entire process from image acquisition to cable force calculation of the present invention is automatically completed by algorithms without manual intervention. Both cable positioning coding and vibration displacement calculation rely on machine learning and image processing technologies, significantly reducing manual operation time and costs; high-frame-rate cameras combined with efficient algorithms can process vibration data in real time, meeting the high-frequency dynamic monitoring requirements of large-scale bridge structures; through cable coding and parametric calibration, multiple cables of different bridge structures can be flexibly adapted to achieve multi-target parallel monitoring.

[0015] 3. Through spatial gradient constraint and direction simplification (vertical displacement first), the present invention adapts to complex vibration modes; at the same time, the SVD singular value energy ratio threshold (>90%) is used to screen key modes, improving data robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 Shows the flow chart of a fully automatic high-precision cable vibration measurement method based on machine vision technology of the present invention; Figure 2 Shows the schematic diagram of calculating displacement by the optical flow method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0019] In addition, the described features, structures, or characteristics may be combined in any suitable manner in one or more example embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the example embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, steps, etc. may be adopted. In other cases, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0020] Example:

[0021] Refer to Figure 1 As shown, a fully automatic high-precision measurement method for cable vibration based on machine vision technology in this embodiment is as follows: Step 1. Equipment installation: Adjust the viewing angle and focal length of the binocular camera to make the plane where the cable vibration is located be within the camera imaging plane as much as possible; To measure the vibration of the cable, fix a camera on the shore with a tripod, and ensure that the measured cable structure is within the sight range of the camera by adjusting the lens direction, so as to ensure clear imaging and complete capture of the cable vibration area.

[0022] Step 2. Image acquisition: Start the camera to shoot the cable vibration video; To compare and evaluate the measurement results of the method of the present invention, select the accelerometer originally installed on the bridge cable as a reference. As a contact measurement method, the measurement accuracy and acquisition frequency of this accelerometer are much higher than those of the camera.

[0023] Step 3. Effective pixel screening: Select the target area in the captured video, and screen the effective pixels by calculating the pixel intensity variance of the pixels in the selected area within a set time period The formula for pixel intensity variance is:

[0024] ; where is the number of video frames captured within the time period; is the average value of the pixel intensity within this time period; is the pixel intensity of the i-th frame within this time period; the larger the variance, the more significant the fluctuation of the pixel intensity over time. Construct the pixel intensity time history of the effective pixels into a pixel intensity spatio-temporal matrix

[0025] , , The size of the pixel intensity spatio-temporal matrix is , where and They are the number of spatial pixels and the number of temporal samplings of the spatio-temporal pixel intensity matrix respectively; the spatio-temporal matrix is constructed from the pixel intensity time series of spatial pixels. For example, for a 5*5 image, the number of pixels in the image is 25. When constructing the spatio-temporal matrix, these 25 pixels are taken as one coordinate and time as the other coordinate. For instance, if the number of temporal samplings is 10, then the size of this spatio-temporal matrix is 25*10.

[0026] 。

[0027] Step Four, Ultra-sensitive calculation: For the pixel intensity time-course matrix perform singular value decomposition (SVD) spatio-temporal weighted averaging (reconstruction) to obtain where represents the first singular modes, which are selected by the cumulative energy ratio of singular values being greater than 90%, and restore the averaged (reconstructed) pixel intensity matrix to the corresponding position in the corrected video; 、 are unitary matrices, representing the spatial mode and the temporal mode respectively; and are unitary matrices retaining the first k modes.

[0028] 。

[0029] Step Five, Lasso positioning coding: Based on the pixel intensity gradient of the first target area, locate and code the lasso.

[0030] S51. Obtain data, including the first target area image and the pixel intensity gradient.

[0031] The first target area image: That is, the first frame image of the effective area selected in Step Three (the static reference image when not vibrating).

[0032] Pixel intensity gradient: Calculate the gradient amplitude and direction of each pixel in the image through edge detection algorithms (such as Sobel, Canny), which reflects the severity and direction of brightness change.

[0033] S52. Lasso positioning.

[0034] Automatically judge the lasso position according to the gradient distribution.

[0035] Gradient feature extraction: As a linear structure, the lasso has a relatively high gradient amplitude in its edge area, and the gradient direction is perpendicular to the lasso axis.

[0036] Line detection: Use the Hough Transform or morphological operations (such as skeleton extraction) to identify the linear features of the lasso from the high-gradient areas.

[0037] Position calibration: Parameterize the detected straight lines (such as representing them by the starting and ending coordinates or polar coordinates), and record the geometric positions of each cable in the image.

[0038] S53. Cable coding.

[0039] Unique identifier assignment: Assign a unique number (Cable 1, Cable 2, Cable 3, Cable 4... Cable n) to each cable according to its position.

[0040] Attribute recording: Additionally store the physical parameters of the cables, including the preset cable length and unit mass, for facilitating subsequent cable force calculation.

[0041] Step Six. Vibration displacement calculation: Refer to Figure 2 As shown, use the optical flow method, select the first frame of the target area as the reference frame, and calculate the vibration displacement of each cable ; where S represents the displacement at time t at coordinate ; represents the pixel intensity at coordinate ; represents the pixel intensity gradient at this coordinate at time 1.

[0042] Obtain the reference frame, current frame, and cable coding information.

[0043] Mark the first static image (t = 0) of the framed target area in Step Three as the reference frame; mark the vibration images at subsequent time points (t = 1, 2,..., T) as the current frame; the cable coding information includes the pixel coordinate ranges of each cable located in Step Five.

[0044] Spatial gradient : Use the Sobel operator to calculate the horizontal and vertical gradients of the reference frame: ; ; Calculate the pixel intensity difference between the current frame and the reference frame: ; Perform the following operations on the pixel area of each coded cable: Direction constraint: Assume the vibration direction of the cable is the vertical direction (d = [0, 1]), then the horizontal displacement can be ignored, and only the vertical displacement needs to be calculated .

[0045] Single-degree-of-freedom simplification: ; where is the vertical gradient, is the time gradient.

[0046] Anti-noise processing: For the gradient values Pixels less than the set threshold are eliminated; the displacement results within the same cable area are median-filtered to eliminate outliers.

[0047] Group the displacement calculation results of each pixel according to the cable code to generate the vibration displacement time history curve of each cable: ; Step 7: Frequency calculation: Perform Fourier transform on the calculated cable vibration time history ; Obtain the frequency spectrum diagram of the cable vibration, and identify the fundamental frequency of the cable vibration through the harmonic frequency identification algorithm ; The fundamental frequency is the frequency component with the largest energy in the frequency spectrum.

[0048] Step 8: Cable force calculation: Based on the known cable parameters of mass per unit length and cable length , combined with the identified cable vibration frequency , calculate the cable force through the string vibration theorem .

[0049] The beneficial effects of this embodiment are as follows: Through non-contact measurement by binocular vision, sensor additional interference is avoided, and non-destructive monitoring of cables is realized; SVD spatio-temporal reconstruction combined with the optical flow method effectively suppresses environmental noise, improves the signal-to-noise ratio and sensitivity of vibration signals; The dynamic area is screened based on pixel intensity variance to accurately lock the cable vibration characteristics; Automatic coding positioning and Fourier spectrum analysis are used to realize rapid identification of the fundamental frequency and cable force calculation, and the accuracy meets the engineering-level requirements; The whole process is automatically processed, adapting to complex lighting and vibration environments, and can monitor the vibration states of multiple cables in real time, providing a high-precision and high-efficiency solution for the structural health monitoring of bridges, cableways, etc.

[0050] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0051] Those of ordinary skill in the art will realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present application.

[0052] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0053] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only for some logical function divisions, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings, direct couplings, or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0054] The unit described as a separation component may or may not be physically separated. The component displayed as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0055] As described above, the foregoing are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A fully automatic high-precision measurement method for cable vibration based on machine vision technology, characterized in that, The method process is as follows: Equipment installation: Adjust the perspective and focal length of the monocular or binocular camera so that the vibration plane of the cable coincides with the camera imaging plane, and fix the camera at a stable position; Image acquisition: Shoot the cable vibration video at a frame rate greater than or equal to 100 frames per second, and record the pixel intensity time history data; Effective pixel screening: Screen the effective pixels by calculating the variance of the pixel intensity time history in the target area, and construct the pixel intensity spatio-temporal matrix; Ultra-sensitive calculation: Perform singular value decomposition on the pixel intensity spatio-temporal matrix, select the first k-order modes with the cumulative energy ratio of singular values > 90% to reconstruct the matrix, and reveal the true vibration signal; Cable positioning and coding: Based on the pixel intensity gradient of the static reference image, locate the straight line feature of the cable through the Hough transform or morphological operation, assign a unique code and store the physical parameters; Vibration displacement calculation: Adopt the optical flow method, combine the direction constraint and anti-noise processing, and calculate the vibration displacement time history curve of each cable; Frequency calculation: Perform Fourier transform on the vibration displacement time history, and extract the fundamental frequency through the frequency doubling identification algorithm; Cable force calculation: Based on the chord theorem, combine the fundamental frequency, cable length and unit mass to calculate the cable force.

2. The fully automatic high-precision measurement method for cable vibration based on machine vision technology according to claim 1, wherein In the equipment installation, the camera focal length is determined according to the actual distance between the camera and the cable to be measured, and the image signal bit depth is not less than 8 bits; During installation, adjust the lens direction through the tripod so that the cable vibration plane is perpendicular to the camera optical axis, ensuring that the imaging resolution ≥ 1920×1080 pixels.

3. A fully automatic high-precision measurement method for cable vibration based on machine vision technology according to claim 1, characterized in that, The specific effective pixel screening is as follows: Select a target area in the captured video, and screen the effective pixels by calculating the pixel intensity variance of the pixels in the selected area within a set time period ; The pixel intensity variance formula is as follows: ; where is the number of video frames collected within the time period; is the mean value of the pixel intensity within this time period; is the pixel intensity of the i-th frame within this time period; The greater the variance, the more significant the fluctuation of pixel intensity over time; the time course of pixel intensity of valid pixels is constructed into a spatio-temporal matrix of pixel intensity , whose size is , where and are the number of spatial pixels and the number of temporal samplings of the spatio-temporal pixel intensity matrix, respectively.

4. A fully automatic high-precision measurement method for cable vibration based on machine vision technology according to claim 1, characterized in that, In the singular value decomposition, the singular value decomposition expression is: ; , where and are unitary matrices, representing spatial mode and temporal mode respectively; and are unitary matrices that retain the first k modes; is a singular value diagonal matrix; is the reconstructed pixel intensity matrix; k is the first k modes retained during reconstruction, which needs to satisfy that the cumulative energy ratio > 90% and is used to reveal the true vibration signal.

5. A fully automatic high-precision measurement method for cable vibration based on machine vision technology according to claim 1, characterized in that Regarding the cable positioning and coding, specifically: S51. Obtain data, including the first target area image and the pixel intensity gradient; The first target area image: The first frame image of the selected effective area; Pixel intensity gradient: Calculate the gradient amplitude and direction of each pixel in the image through the edge detection algorithm, reflecting the severity and direction of the brightness change; S52. Automatically judge the cable position according to the gradient distribution; Include gradient feature extraction, identify the straight line feature of the cable from the high-gradient area by using the Hough transform or morphological operation, parameterize the detected straight line, and record the geometric position of each cable in the image; S53. Cable coding; According to the position of the cable, assign a unique number to each cable, and additionally store the physical parameters of the cable, including the preset cable length and unit mass.

6. The full-automatic cable vibration high-precision measurement method based on machine vision technology according to claim 1, characterized in that The vibration displacement calculation is based on the optical flow constraint equation: ； Among them, is the spatial gradient, reflecting the change rate of pixel brightness in the x and y directions, and is used to detect the spatial displacement trend of the cable vibration; is the time change rate of pixel brightness, which is directly related to the displacement of the cable vibration; is the pixel velocity, and the vibration displacement is solved through the constraint equation; for vertical vibration, only the velocity in the y direction is calculated when u = 0, which is simplified to: ; pixels with gradient values less than the threshold are removed, and median filtering is used in to eliminate outliers, and the average vibration displacement time history of cable i is calculated by grouping according to the cable code : .

7. A fully automatic high-precision measurement method for cable vibration based on machine vision technology according to claim 1, characterized in that, The frequency calculation process is: Perform Fourier transform on the calculated cable vibration time history ; Obtain the frequency spectrum diagram of the cable vibration, and identify the fundamental frequency of the cable vibration through the harmonic frequency identification algorithm ; The fundamental frequency is the frequency component with the largest energy in the frequency spectrum; is the frequency-domain spectrum; is the angular frequency.

8. A fully automatic high-precision measurement method for cable vibration based on machine vision technology according to claim 1, characterized in that The cable force calculation formula is as follows: ; where is the cable force, is the mass per unit length, is the cable length, is the fundamental frequency; and the measurement results are verified by comparing with the accelerometers installed on the bridge cables.

Images