Video detection method and system based on spatial domain and time domain characteristic cable frequency

Abstract

The application discloses a video detection method and system based on spatial domain and time domain characteristic cable frequency, and belongs to computer vision technology. The method comprises the following steps: inputting an image sequence; extracting phase information, including extracting phase information of a detection object and a background in a phase space; filtering out background phase, including calculating local spatial feature information after an adaptive direction filter to screen out the phase of the detection object and the background, filtering out the phase information of the background, and reserving the phase information of the detection object; and extracting a vibration signal and a frequency, including extracting a frequency value of vibration of the detection object. Compared with the prior art, the application has the advantages that weak vibration signals and vibration frequencies of a cable structure can be well extracted in a complex background, so that the cable force of the cable can be accurately calculated.

Description

Technical Field

[0001] This invention relates to computer vision technology, and more specifically, to a video detection method and system based on the frequency of features in the spatial and temporal domains. Background Technology

[0002] Cables are crucial load-bearing components of cable-stayed bridges, bearing the load of the main girder. The proper functioning of the cables has a significant impact on the bridge's lifespan and overall safety. Therefore, it is necessary to monitor cable tension in cable-stayed bridges to ensure smooth bridge closure and structural safety during construction, and to monitor cable tension changes during operation to understand structural damage and prevent redistribution of internal forces and alterations in bridge alignment due to changes in cable tension over time, which could affect structural safety and service life. Therefore, cable tension testing is an important aspect of cable-stayed bridge health monitoring and a vital indicator for assessing the bridge's health status.

[0003] Currently, methods for measuring cable tension in cable-stayed bridges include pressure sensor method, hydraulic gauge test method, magnetic flux method, and frequency method. Video phase-based cable measurement is an emerging frequency measurement technology that requires no auxiliary markings and is highly efficient, with an ideal frequency detection error of ≤0.05%. However, in practical applications, it has been found that existing video phase-based methods fail when the cables are surrounded by complex backgrounds such as buildings and vegetation, and cannot accurately extract the cable frequency information.

[0004] In related technologies, for example, Chinese patent document CN101055216A provides a monitoring method for installing pressure sensors on stay cables. This method has high accuracy, but the equipment is expensive and heavy. Another example is Chinese patent document CN109668658A, which provides a method for monitoring cable force using a magnetic flux sensor. This method requires prior calibration to obtain the correlation between the magnetic permeability of the stay cable and the cable force and external temperature, and the magnetic flux method testing technology is currently not mature enough. Yet another example is Chinese patent document CN1150964. 94A provides a cable force monitoring system that uses an accelerometer to calculate the cable force value based on the correlation between vibration frequency and cable force. The drawbacks of this method are that the equipment is inconvenient to operate and the detection efficiency is low. In addition, there is the hydraulic pressure gauge test method, which converts the hydraulic pressure gauge reading into the tension force of the jack, i.e., the cable force of the stay cable, based on the direct proportionality between the tension force of the hydraulic jack and the reading of the hydraulic pressure gauge. This method is often used for cable force control during construction, but the accuracy is not high enough, and it cannot measure the cable force of the stay cable after it has been tensioned. It is not suitable for long-term monitoring of cable force after the bridge is completed.

[0005] In summary, no effective solution has yet been proposed for the problem of accurately extracting the frequency information of cables in complex backgrounds in related technologies. Summary of the Invention

[0006] 1. Technical problems to be solved

[0007] To address the issue of frequency detection failure in complex backgrounds such as buildings and vegetation using video phase-based methods, this invention provides a video detection method and system based on spatial and temporal domain features of cable frequencies. This method can accurately separate the cable from the background in the phase space of the cable, thereby calculating the cable's vibration frequency, and is robust to complex backgrounds.

[0008] 2. Technical Solution

[0009] The objective of this invention is achieved through the following technical solutions.

[0010] A video detection method based on cable frequency features in the spatial and temporal domains includes the following steps:

[0011] Input image sequence;

[0012] Extracting phase information from image sequences: Extracting phase information of the detected object and background in phase space;

[0013] Extracting vibration signals and frequencies from image sequences: Extracting the frequency values ​​of vibration of the detected object;

[0014] The following steps are included before extracting the vibration signal and frequency:

[0015] Background phase filtering: Calculating the local spatial feature information after the adaptive directional filter can filter out the phase of the detected object and the background, thereby filtering out the phase information of the background and retaining the phase information of the detected object.

[0016] Furthermore, the specific steps for inputting the image sequence are as follows: capture a video of the bridge cable vibration, with the camera frame rate being greater than twice the maximum vibration frequency of the cable, and the video frame count between 500 and 600 frames, and save the video.

[0017] Furthermore, the specific steps for extracting phase information are as follows: Define R as the number of directed lines equally divided at angles within a window centered on the current phase point and with side length C, where R is a natural number greater than 0. The formula for calculating the variance of the phase gradient on each line is:

[0018]

[0019] The window size is C×C, and p0(n,m) is the phase value of the m-th pixel in direction n, where 1≤n≤R and 1≤m≤P. sum P sum p is a natural number greater than 0, representing the total number of pixels in direction n. ave(n) represents the average phase value along direction n, and the fringe direction is the direction with the minimum phase gradient variance. D n Let be the variance of the phase gradient in the m-th direction;

[0020] The horizontal coordinate corresponding to the minimum value is the desired fringe direction, and the angle θ between the fringe direction and the phase fringe is obtained by fitting using the least squares method. r ;

[0021] The angle θ of the obtained phase fringes r Formula for directional input of Gabor filter bank:

[0022]

[0023] The calculated filtered phase information is a relatively clean background and cable information;

[0024] Where x0, y0 are the center points of the Gaussian kernel, and θ is the clockwise rotation direction of the Gaussian kernel. r That is, the direction angle of the filter, σ x ,σ y Let u0 and v0 be the scales in the two directions of the Gaussian kernel, respectively. Let K be the amplitude scale of the Gaussian kernel, and x and y be the horizontal and vertical coordinates.

[0025] Furthermore, the specific steps for filtering out background phase are as follows: calculate the absolute average deviation of the phase value of the filtered image from the average value μ within a window of size C×C, and design the formula for the spatial local feature E(x, y) as follows:

[0026]

[0027] Where R(a,b) is the Gabor transform result of the phase intensity I(x,y) at a certain point in the input image, and μ is the average phase value of all points within the C×C window after the Gabor transform, i.e.

[0028] Furthermore, the specific steps for filtering out the background phase are as follows, where R(a,b) is calculated using the following formula:

[0029]

[0030] Where * represents a two-dimensional linear convolution, C and N are the sizes of the Gabor filter window, (s,z) are the horizontal and vertical coordinates of the pixels within the filter, 0≤s≤C-1, 0≤z≤N-1, and g(x,y) is a two-dimensional Gabor function.

[0031] Furthermore, the specific steps for filtering out the background phase are as follows, where the two-dimensional Gabor function g(x,y) is calculated using the following formula:

[0032] g(x,y)=s(x,y)*w(x,y)

[0033] Where s(x,y) is the complex sine function and w(x,y) is the envelope of the Gaussian function.

[0034] Furthermore, the specific steps for filtering out the background phase are as follows, where the complex sine function s(x,y) is calculated using the following formula:

[0035] s(x,y)=cos(2π·(u0·x+v0·y)+φ)+i·sin(2π·(u0·x+v0·y)+φ)

[0036] Where φ is the phase of the complex sine function.

[0037] Furthermore, the specific steps for filtering out the background phase are as follows, where the Gaussian function envelope w(x,y) is calculated using the following formula:

[0038]

[0039] Furthermore, the specific steps for extracting the vibration signal and frequency are as follows: after obtaining the pure cable phase information in the phase space, the vibration signal of the cable is extracted using a video phase-based vibration analysis method.

[0040]

[0041] Where V(x, t) is the vibration signal, and A ω Let ω be the vibration amplitude, ω be the vibration frequency, and δ(x,t) be a displacement function with a small amplitude.

[0042] The system based on the video detection method using spatial and temporal domain feature cable frequencies described above.

[0043] Includes an acquisition module for acquiring image sequences;

[0044] The phase information extraction module is used to extract the phase information of the detected object and the background in phase space;

[0045] The background phase filtering module is used to filter out the phase information of the background and retain the phase information of the detected object;

[0046] The vibration signal and frequency extraction module is used to extract the frequency value of the vibration of the detected object.

[0047] 3. Beneficial effects

[0048] Compared with the prior art, the advantages of this invention are:

[0049] The present invention relates to a video detection method and system based on the spatial and temporal domain features of cable frequencies. By combining a designed phase adaptive direction filter and feature selection, the method selects phase information in the phase space and filters local features, thereby filtering out background phase information and obtaining pure cable phase information. This method can effectively extract weak vibration signals and vibration frequencies of cable structures in complex backgrounds, thus accurately calculating the cable force. It effectively reduces the workload of technicians, improves work efficiency, and is suitable for long-term bridge cable frequency detection and health monitoring. Attached Figure Description

[0050] Figure 1 This is a flowchart of a bridge cable frequency measurement method in one embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of a phase line feature extraction method in one embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of a direction adaptive filter and its filtering effect in one embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the local feature filtering results in one embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of the time-domain measurement results of bridge cables in one embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram of the frequency domain measurement results of bridge cables in one embodiment of the present invention. Detailed Implementation

[0056] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0057] Example 1

[0058] Combination Figures 1 to 6 The video detection method based on the frequency characteristics of cable in the spatial and temporal domains of the present invention includes the following steps:

[0059] Input image sequence:

[0060] A Canon camera was used to capture video of bridge cable vibration in a real-world scenario. The camera frame rate was adjusted according to the actual situation. To satisfy the Nyquist sampling theorem, the camera frame rate should be greater than twice the maximum vibration frequency of the cable; in this embodiment, 50fps was used. The video frame rate was between 500 and 600 frames per second and saved in a video format. Generally, the captured video data can be in common video formats such as AVI, MOV, and MP4.

[0061] Extracting phase information:

[0062] Step (1): Define R (R is a natural number greater than 0, in this embodiment R = 8) as the number of directed lines that are equally spaced in a window W with the current phase point as the center and C as the side length.

[0063] The variance of the phase gradient on each straight line is calculated using formula (a):

[0064]

[0065] Where the window W has a size of C×C, D n Let p0(n, m) be the variance of the phase gradient in a certain direction, and p0(n, m) be the m-th phase value in direction n, where 1 ≤ n ≤ R and 1 ≤ m ≤ P. sum (P sum p is the total number of pixels in direction n (a natural number greater than 0). ave (n) represents the average phase value in direction n.

[0066] Since the fringe direction is the direction with the minimum phase gradient variance, D n Let be the variance of the phase gradient in the m-th direction. Therefore, the horizontal coordinate corresponding to the minimum value is the desired fringe direction, and the angle θ between the fringe direction and the phase fringe is obtained by fitting using the least squares method. r This step utilizes the characteristic that the phase gradient variance of a straight line is minimized to search for stripes in the image that indicate the direction of the cable and calculate the corresponding angle information, providing parameters for the subsequent filter input.

[0067] Step (2): The included angle θ of the phase fringes obtained in step (1) r Formula (b) as the directional input for the Gabor filter bank:

[0068]

[0069] x0, y0 are the center points of the Gaussian kernel, and θ r σ represents the rotation direction of the Gaussian kernel (clockwise), i.e., the filtering direction angle of the filter. x ,σ y Let u0 and v0 be the frequency domain coordinates, K be the amplitude scale of the Gaussian kernel, and x and y be the horizontal and vertical coordinates, respectively. The parameters designed in this embodiment are as follows:

[0070] The filtered phase information is a relatively clean background and cable information.

[0071] Background phase filtering:

[0072] By calculating the phase values ​​of the filtered image within a C×C window W, the absolute average deviation from the average value μ is determined, where μ is the average phase value of all points within window W after Gabor transformation.

[0073]

[0074] The formula (c) for designing the local features of the space is:

[0075]

[0076] Where R(a,b) is the result of Gabor transforming the phase intensity I(x,y) at a certain point in the input image, and the calculation formula (d) is:

[0077]

[0078] * represents a two-dimensional linear convolution, C and N are the sizes of the Gabor filter window, (s,z) are the x and y coordinates of the pixels within the filter, 0≤s≤C-1, 0≤z≤N-1, and g(x,y) is the two-dimensional Gabor function, calculated using formula (e):

[0079] g(x,y)=s(x,y)*w(x,y)

[0080] Where s(x,y) is a complex sine function, and its calculation formula (f) is:

[0081] s(x,y)=cos(2π·(u0·x+v0·y)+φ)+i·sin(2π·(u0·x+v0·y)+φ)

[0082] φ is the phase of the complex sine function; in this embodiment, φ = 0.

[0083] Where w(x,y) is the envelope of the Gaussian function, and the calculation formula (g) is:

[0084]

[0085] After calculating the local spatial feature information through the adaptive directional filter using formula (c), the phase of the cable and the background can be filtered out, thereby extracting the pure cable phase information. This step defines a local feature formula in the filtered image. Since the local features of the background and cable stripes differ, the local features of each stripe (which could be a background stripe or a cable stripe) are calculated to distinguish the background and cable in terms of local features, thus spatially segmenting the cable and background and extracting the pure cable spatial phase information.

[0086] Extracting vibration signals and frequencies:

[0087] The cable frequency is calculated based on the phase information. After obtaining the pure cable phase information in phase space, the vibration signal of the cable is extracted using the video phase-based vibration analysis (PVE) method. Where the vibration signal V(x,t), A ω Let ω be the vibration amplitude, ω be the vibration frequency, and δ(x,t) be a displacement function with a small amplitude.

[0088] The local temporal vibration of the cable is related to the local phase φ(x,t)=2πω(x+δ(x,t))=2πωx+2πωδ(x,t). By filtering the phase φ(x,t) through a DC filter to remove the DC component 2πωx, and then subtracting the local phase of each frame from that of the reference frame, the phase difference signal φ'(x,t)=2πωδ(x,t) of each frame can be obtained. By calculating the frequency domain response of the small displacement function δ(x,t), the frequency value of the cable vibration can be obtained.

[0089] The video detection system based on the spatial and temporal domain feature cable frequencies of the present invention includes the following steps:

[0090] Data Acquisition Module:

[0091] The video of the bridge cables to be measured is captured using a video acquisition device, with approximately 10 seconds of video recording. The acquisition device satisfies the Nyquist sampling theorem, meaning the acquisition frequency (frame rate) is greater than or equal to twice the vibration frequency of the cables. In this embodiment, a 50fps camera is used.

[0092] Phase information extraction module:

[0093] A complex controllable pyramid is used to spatially decompose the cable-stayed video image sequence. By performing spatial bandpass filtering and downsampling on each frame, amplitude and phase spectra at different scales and directions are obtained. The phase spectrum information at this point mainly contains phase information from the cable and background. The phase line feature extraction method is as follows: Figure 2 As shown.

[0094] Background phase filtering module:

[0095] By combining a designed phase-adaptive-directional filter (PAD filter) with feature selection (FS), the phase information of the background can be automatically filtered out while retaining the phase information of the cable. The PAD-FS design consists of the following steps: a) extraction of spatial phase information; b) screening of local spatial features of the phase.

[0096] Adaptive directional filter and its filtering effect are as follows Figure 3 The results of local feature filtering are as follows: Figure 4 .

[0097] Vibration signal and frequency extraction module:

[0098] After obtaining the pure cable phase information in phase space, the vibration signal of the cable is extracted using the video phase-based vibration analysis (PVE) method. Where the vibration signal V(x,t), A ω Let ω be the vibration amplitude, ω be the vibration frequency, and δ(x,t) be a displacement function with a small amplitude.

[0099] The local temporal vibration of the cable is related to the local phase φ(x,t)=2πω(x+δ(x,t))=2πωx+2πωδ(x,t). By filtering the phase φ(x,t)ω(x+δ(t)) through a DC filter to remove the DC component 2πωx, and then subtracting the local phase of each frame from that of the reference frame, the phase difference signal φ′(x,t)=2πωδ(x,t) of each frame can be obtained. By calculating the frequency domain response of the small displacement function δ(x,t), the frequency value of the cable vibration can be obtained.

[0100] The time-domain measurement results of the bridge cables are as follows Figure 5 As shown, the frequency domain measurement results of the bridge cables are as follows: Figure 6 As shown.

[0101] The present invention relates to a video detection method and system based on the spatial and temporal domain features of cable frequencies. By combining a designed phase adaptive direction filter and feature selection, the method selects phase information in the phase space and filters local features, thereby achieving the goal of filtering out background phase information and obtaining pure cable phase information. It can effectively extract weak vibration signals and vibration frequencies of cable structures in complex backgrounds, thus accurately calculating the cable force. This effectively reduces the workload of technicians, improves work efficiency, and is suitable for long-term bridge cable frequency detection and health monitoring.

[0102] The invention and its embodiments have been described above illustratively. This description is not restrictive, and the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. The accompanying drawings are only one embodiment of the invention, and the actual structure is not limited thereto. No reference numerals in the claims should limit the scope of the claims. Therefore, if a person skilled in the art is inspired by this description and designs a similar structure and embodiment without departing from the spirit of the invention, such design should fall within the scope of protection of this patent. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Multiple elements stated in the product claims may also be implemented by a single element through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A video detection method based on cable frequency features in the spatial and temporal domains, comprising the following steps: Input image sequence; Extracting phase information from image sequences: extracting phase information of the detected object and background in phase space; Extracting vibration signals and frequencies from image sequences: extracting the frequency values ​​of vibration of the detected object; The feature is that, before extracting the vibration signal and frequency, it further includes the following steps: Background phase filtering: Calculating the local spatial feature information after the adaptive directional filter can filter out the phase of the detected object and the background, thereby filtering out the phase information of the background and retaining the phase information of the detected object; The specific steps for extracting phase information from an image sequence are as follows: Define R as the number of directed lines equally spaced within a window centered at the current phase point and with side length C, where R is a natural number greater than 0. The formula for calculating the variance of the phase gradient on each line is: ； The window size is C×C, and p0(n,m) is the phase value of the m-th pixel in direction n, where 1≤n≤R and 1≤m≤P. sum P sum P is a natural number greater than 0, representing the total number of pixels in direction n. ave (n) represents the average phase value along direction n, and the fringe direction is the direction with the minimum phase gradient variance. D n Let be the variance of the phase gradient in the m-th direction; The horizontal coordinate corresponding to the minimum value is the desired fringe direction, and the angle θ between the fringe direction and the phase fringe is obtained by fitting using the least squares method. r ; The angle θ of the obtained phase fringes r Formula for directional input of Gabor filter bank: ； The calculated filtered phase information is a relatively clean background and cable information; Where x0, y0 are the center points of the Gaussian kernel, and θ is the clockwise rotation direction of the Gaussian kernel. r That is, the direction angle of the filter, σ x ,σ y Let u0 and v0 be the scales in the two directions of the Gaussian kernel, respectively. Let K be the amplitude scale of the Gaussian kernel, and x and y be the x-axis and y-axis, respectively.

2. The video detection method based on spatial and temporal domain feature cable frequency according to claim 1, characterized in that: The specific steps for inputting the image sequence are as follows: capture a video of the bridge cable vibration, the camera frame rate should be greater than twice the maximum vibration frequency of the cable, the number of video frames should be between 500 and 600, and save the video.

3. The video detection method based on spatial and temporal domain feature cable frequency according to claim 1, characterized in that: The specific steps for filtering out background phase are as follows: calculate the absolute average deviation of the phase value of the filtered image from the average value μ within a window of size C×C, and design the formula for the spatial local feature E(x, y) as follows: ； Where R(a, b) is the Gabor transform result of the phase intensity I(x, y) at a certain point in the input image, and μ is the average phase value of all points within the C×C window after the Gabor transform, i.e. .

4. The video detection method based on spatial and temporal domain feature cable frequency according to claim 3, characterized in that: The specific steps for filtering out the background phase are as follows, where R(a,b) is calculated using the following formula: ； Where * represents a two-dimensional linear convolution, C and N are the sizes of the Gabor filter window, (s, z) are the horizontal and vertical coordinates of the pixels within the filter, 0 ≤ s ≤ C-1, 0 ≤ z ≤ N-1, and g(x, y) is a two-dimensional Gabor function.

5. The video detection method based on spatial and temporal domain feature cable frequencies according to claim 3, characterized in that: The specific steps for filtering out the background phase are as follows, where the two-dimensional Gabor function g(x,y) is calculated using the following formula: g(x,y)=s(x,y)*w(x,y); Where s(x,y) is the complex sine function and w(x,y) is the envelope of the Gaussian function.

6. The video detection method based on spatial and temporal domain feature cable frequencies according to claim 3, characterized in that: The specific steps for filtering out the background phase are as follows, where the complex sine function s(x,y) is calculated using the following formula: s(x,y)=cos(2π·(u0·x+v0·y)+φ)+i·sin(2π·(u0·x+v0·y)+φ); Where φ is the phase of the complex sine function.

7. The video detection method based on spatial and temporal domain feature cable frequencies according to claim 3, characterized in that: The specific steps for filtering out the background phase are as follows, where the Gaussian function envelope w(x,y) is calculated using the following formula: 。 8. The video detection method based on spatial and temporal domain feature cable frequencies according to claim 1, characterized in that: The specific steps for extracting vibration signals and frequencies are as follows: After obtaining pure cable phase information in the phase space, the vibration signals of the cable are extracted using a video phase-based vibration analysis method. ； Where V(x, t) is the vibration signal, and A ω Let ω be the vibration amplitude, ω be the vibration frequency, and δ(x,t) be a displacement function with a small amplitude.

9. A system based on the video detection method for spatial and temporal domain feature cable frequencies as described in any one of claims 1-8, characterized in that, Includes an acquisition module for acquiring image sequences; The phase information extraction module is used to extract the phase information of the detected object and the background in phase space; The background phase filtering module is used to filter out the phase information of the background and retain the phase information of the detected object; The vibration signal and frequency extraction module is used to extract the frequency value of the vibration of the detected object.

Images