Real-time cable force identification method and device for inhaul cable based on deep learning

Abstract

The invention belongs to the technical field of bridge health monitoring, and aims at solving the problem that a traditional real-time cable force calculation method is low in recognition precision. The invention provides an inhaul cable real-time cable force identification method and device based on deep learning. The method comprises the following steps that vibration image information of a target inhaul cable is acquired; obtaining a target image sequence based on the vibration image information; adopting an optical flow estimation algorithm based on deep learning to obtain vibration displacement of a target inhaul cable in the target image sequence; variational mode decomposition transformation is conducted on the vibration displacement, and the mode component and the real-time frequency of the target inhaul cable are obtained; performing fast Fourier transform on the modal component, and determining the inherent frequency of each order and the corresponding order of the target inhaul cable; and calculating the real-time cable force of the target cable based on the real-time frequency and the order of the inherent frequency corresponding to the real-time frequency. According to the method, the vibration displacement identification precision of the target inhaul cable is improved, and the robustness of real-time cable force identification is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of bridge health monitoring, and in particular relates to a method and device for real-time cable force identification based on deep learning. Background Art

[0002] For long-span cable-stayed bridges, cables are a key factor in the overall safety of the structure. Under the influence of moving vehicles and other environmental influences, cables will inevitably suffer from corrosion, wire breakage, prestress loss and other damages, which will weaken the stiffness of the cables, reduce the bearing capacity, and eventually lead to cable breakage and bridge accidents. In cable bridges, cable force directly reflects the working status of the bridge. Therefore, identifying real-time cable force is indispensable for early damage detection, effectively predicting its service life, and controlling further deterioration, and is crucial for the status and health monitoring of cable bridges.

[0003] Traditional cable force identification is mainly carried out through contact sensors, including oil pressure gauges, pressure sensors, fiber Bragg grating sensors, piezoelectric sensors, and acceleration sensors. These contact measuring instruments have the disadvantages of difficult sensor installation, high equipment cost, and low test efficiency, and they also require continuous supervision; they are easily affected by complex environments such as temperature, wind and rain, and it is difficult to repair damaged instruments, and the maintenance cost is high, which gradually limits the application of this method in actual engineering.

[0004] With the development of computer vision technology and image acquisition equipment, non-contact structural vibration measurement methods based on computer vision have received widespread attention in cable force identification. This method has the characteristics of low cost, flexibility, high accuracy, and strong real-time performance. In recent years, non-contact structural vibration measurement methods have mainly focused on template matching, feature point matching, digital image correlation, optical flow, and deep learning. Among them, template matching, feature point matching, and digital image correlation are mainly suitable for target recognition with artificial markings, otherwise it will lead to recognition failure. It is very difficult to stick artificial marks on the cables, so these methods are not very suitable for cable vibration recognition in bridges.

[0005] Optical flow and deep learning-based methods can identify the vibration displacement of cables without any manual labeling, but traditional optical flow algorithms are prone to loss when tracking feature points, resulting in many outliers in the displacement identification results. In addition, most of them aim to identify the constant cable force of the cable, and pay less attention to its real-time cable force. The traditional real-time cable force calculation method is through short-time Fourier transform or Hilbert-Huang transform, among which short-time Fourier transform is easily limited by the size of the window function and has low resolution. Hilbert-Huang transform is prone to modal aliasing problems when processing nonlinear and non-stationary signals, resulting in inaccurate decomposition results, and improper parameter selection can easily lead to instability and distortion of the decomposition results. Summary of the invention

[0006] In order to solve at least one of the above-mentioned technical problems existing in the prior art, the present invention provides a method and device for real-time cable force identification based on deep learning.

[0007] The present invention is implemented by the following technical solution: a real-time cable force identification method based on deep learning, comprising the following steps:

[0008] Acquiring vibration image information of a target cable, wherein the target cable is a cable of a cable-stayed bridge to be identified;

[0009] Obtaining a target image sequence based on the vibration image information, wherein the target image sequence is an image sequence of an area of ​​a target cable to be identified;

[0010] The optical flow estimation algorithm based on deep learning is used to obtain the vibration displacement of the target cable in the target image sequence;

[0011] Performing variational modal decomposition transformation on the vibration displacement to obtain modal components and real-time frequencies of the target cable;

[0012] Performing fast Fourier transform on the modal components to determine the natural frequencies of each order and the corresponding orders of the target cable;

[0013] The real-time cable force of the target cable is calculated based on the real-time frequency and the order of its corresponding natural frequency.

[0014] Preferably, the step of acquiring the vibration displacement of the target cable in the target image sequence by using an optical flow estimation algorithm based on deep learning comprises:

[0015] Using a recursive full-field transform based on deep learning to perform optical flow estimation on the target image sequence to obtain an optical flow estimation result;

[0016] The vibration displacement of the target cable is extracted based on the optical flow estimation result.

[0017] Preferably, the step of obtaining the optical flow estimation result comprises:

[0018] Extracting feature vectors for each pixel from consecutive frame images through feature and context encoders;

[0019] By using the eigenvectors, the inner products of all eigenvector pairs are calculated to construct a 4D correlation space body;

[0020] A search operator is defined to search for required features from the relevant space volume, and then used as input for iterative updating;

[0021] Based on the GRU recurrent update operator, the full-field optical flow of the image sequence of the target cable area is identified.

[0022] Preferably, the step of obtaining the modal components and real-time frequency of the target cable includes:

[0023] Performing variational modal decomposition on the vibration displacement to obtain each modal component of the target cable;

[0024] Perform Hilbert transform on each modal component to obtain the real-time frequency of the target cable.

[0025] Preferably, the step of obtaining each modal component of the target cable comprises:

[0026] For the vibration displacement signal, the initial number of decomposition modes, penalty factors and iteration parameters are set, and the objective function is obtained through updating iteration;

[0027] Based on the objective function and the constructed variational modal decomposition transformation model, the optimal center frequency and finite bandwidth of each mode are adaptively searched and solved, so as to achieve effective separation of inherent modal components, frequency domain division of signals, and then obtain the effective modal decomposition components of a given signal.

[0028] Preferably, the step of obtaining the target cable real-time frequency includes:

[0029] The signal of the selected modal component is transformed into an analytical signal on a two-dimensional complex plane, wherein the mapping method of the analytical signal is that the real part and the imaginary part of the complex number are Hilbert transforms of each other;

[0030] By analyzing the ratio of the imaginary part and the real part of the signal, the instantaneous phase is obtained;

[0031] The instantaneous phase is differentiated in the time domain to obtain the real-time frequency of the target cable.

[0032] Preferably, the step of determining each order of natural frequencies and corresponding orders of the target cable comprises:

[0033] Performing fast Fourier transformation on the modal components to obtain frequency domain data of each component, wherein the frequency domain data includes the peak frequency of each component;

[0034] According to the characteristics that the natural frequency values ​​of each order of the cable vibration satisfy the arithmetic progression and the peak frequency values, the natural frequencies of each order of the cable and the corresponding orders are determined;

[0035] Preferably, the real-time cable force of the target cable is calculated based on the real-time frequency and the order of its corresponding natural frequency, including:

[0036] Substitute the real-time frequency and the order of its corresponding natural frequency into the cable force equation to obtain the real-time cable force of the target cable;

[0037] The cable tension equation is expressed as follows:

[0038]

[0039] Where T is the real-time cable force, ρ is the linear density of the cable, L is the length of the cable, and f n is the nth order vibration real-time frequency of the cable, and n is the order of the natural frequency of the cable.

[0040] The present invention also provides a real-time cable force identification device based on deep learning, comprising:

[0041] The vibration image information acquisition module is used to collect the original video of the punctuation camera parameters and the vibration of the cable of the cable-stayed bridge, and correct the original video to obtain the vibration image information of the target cable; the target image sequence extraction module is used to rotate, crop and filter the vibration image information to obtain the image sequence of the area of ​​the target cable to be identified; the vibration displacement extraction module is used to obtain the vibration displacement of the target cable in the target image sequence based on the optical flow estimation algorithm based on deep learning; the modal component and real-time frequency calculation module is used to perform Hilbert-Huang transform on the vibration displacement of the target cable to obtain the modal component and real-time frequency of the target cable; the module for determining the natural frequencies of each order and the corresponding order of the target cable is used to perform fast Fourier transform on the modal components to determine the natural frequencies of each order and the corresponding order of the target cable; the real-time cable force calculation module is used to calculate the real-time cable force of the target cable based on the real-time frequency and the order of its corresponding natural frequency.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention realizes the method of combining the optical flow algorithm based on deep learning and the variational mode decomposition transformation to identify the real-time cable force of the cable, improves the recognition accuracy of the target cable vibration displacement, and improves the robustness of the real-time cable force recognition. It has the advantages of strong anti-noise ability, simple operation, strong real-time performance and secondary optimization update.

[0044] In addition, the present invention can identify target displacement through simple structure, surface texture and edge information, will not be affected by characteristic points and does not require manual marking, is very suitable for non-contact target vibration displacement measurement, improves measurement accuracy, and facilitates obtaining accurate modal parameters, instantaneous phase and instantaneous frequency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 is a flow chart of the method of the present invention;

[0047] Figure 2 is a schematic diagram of a real-time cable force recognition scenario in the present invention;

[0048] Figure 3 is a schematic diagram of a vibration image information acquisition module in the present invention;

[0049] Figure 4 is a schematic diagram of the optical flow estimation process based on deep learning in the present invention;

[0050] Figure 5 It is a schematic diagram of extracting the vibration displacement of the target cable based on the full-field optical flow estimation in the present invention;

[0051] Figure 6 It is a schematic diagram of the modal component and real-time frequency calculation process of the target cable in the present invention;

[0052] Figure 7 is a real-time cable force identification result diagram of the target cable in the present invention;

[0053] Figure 8 This is a comparison diagram of the real-time cable force identification results of the target cable in the present invention.

[0054] Fig. 9 It is a schematic diagram of a module of a real-time cable force identification device in the present invention;

[0055] Fig.10 It is a computer device structure diagram corresponding to the present invention. DETAILED DESCRIPTION

[0056] In conjunction with the drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size should fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention. It should be noted that in this specification, relational terms such as first and second are only used to distinguish one entity from several other entities, and do not necessarily require or imply any actual relationship or order between these entities.

[0058] The present invention provides an embodiment:

[0059] like Figures 1 to 8 As shown, a real-time cable force identification method based on deep learning includes the following steps:

[0060] S101: Acquire vibration image information of a target cable, where the target cable is a cable of a cable-stayed bridge to be identified;

[0061] S102: obtaining a target image sequence based on the vibration image information, wherein the target image sequence is an image sequence of an area of ​​a target cable to be identified;

[0062] S103: Acquire the vibration displacement of the target cable in the target image sequence using an optical flow estimation algorithm based on deep learning;

[0063] S104: performing variational modal decomposition transformation on the vibration displacement to obtain modal components and real-time frequency of the target cable;

[0064] S105: performing fast Fourier transform on the modal components to determine the natural frequencies of each order and the corresponding orders of the target cable;

[0065] S106: Calculate the real-time cable force of the target cable based on the real-time frequency and its corresponding natural frequency order.

[0066] In this embodiment, the steps of obtaining the vibration image information of the target cable are specifically as follows:

[0067] Acquiring the vibration video image of the cable of the cable-stayed bridge to be identified includes: calibrating the video acquisition device to obtain the internal and external parameters of the camera; and correcting the vibration video of the cable of the cable-stayed bridge to be identified using the calibrated parameters to obtain the corrected vibration video image. Figure 2 The schematic diagram of a real-time cable force recognition scenario based on deep learning is shown in the figure. The video acquisition device is a Sony A9M3 camera with a resolution of 1920×1080 pixels and a frame rate of 60fps. The camera is fixed on a tripod about 100 meters away from the cable-stayed bridge.

[0068] To ensure the accuracy and reliability of the computer vision system, the potential distortion introduced by the camera lens must be considered. These distortions can cause linear distortion or displacement in the image, thus affecting the accuracy. Therefore, the collected cable vibration video needs to be corrected before preprocessing the video image. First, the camera was calibrated using the Zhang Zhengyou chessboard calibration method. The calibration board used an 8×7 chessboard as the calibration board, which was placed at different angles and a total of 30 images were taken. These images were then input into the MATLAB toolbox for calibration. The internal and external parameters of the camera were obtained, as described in Table 1 below. Secondly, the camera was used to collect the original vibration video of the cable, with a total acquisition time of 16 seconds. The cable vibration of the cable-stayed bridge was mainly affected by the combined effects of wind and vehicles. Finally, the vibration video of the cable-stayed bridge cable to be identified was corrected using the calibrated parameters to obtain the vibration image information of the corrected target cable.

[0069] α β γ <![CDATA[μ0]]> <![CDATA[v0]]> <![CDATA[k1]]> <![CDATA[k2]]> <![CDATA[p1]]> <![CDATA[p2]]> 9662.1 9589.0 0 1768.3 2177.0 1305.7 0.01 -0.02 0.01

[0070] Note: α and β are the scale factors of the image in the horizontal and vertical directions; γ represents the tilt between the two image axes; (μ0, v0) represents the coordinates of the camera principal point; k1 and k2 represent the radial distortion coefficients of the camera lens; p1 and p2 represent the tangential distortion coefficients of the camera lens.

[0071] The steps to obtain the target image sequence are as follows:

[0072] The corrected vibration image is rotated and cropped to obtain a video image of the area of ​​the target cable to be identified; the video image of the area of ​​the target cable to be identified is processed using non-local mean filtering to obtain an image sequence under low noise. This method can reduce computational complexity and minimize background interference. The specific steps are as follows:

[0073] (1) Considering the complexity of the algorithm, the search area is 21×21 and the block for similarity comparison is 7×7. This parameter can also be adjusted appropriately according to the actual application.

[0074] (2) The similarity comparison block slides in the search area, and the similarity between all sliding similarity comparison blocks and the pixel blocks to be estimated is calculated and classified, and weighted w(i, j) is assigned. The sum of all pixel weights is equal to 1. The similarity between the block domains centered on two pixels i and j is estimated by the Euclidean distance of the square of the grayscale brightness difference between the two pixels, as shown below:

[0075]

[0076] In the formula, is the pixel block to be estimated centered at pixel i; is the pixel value in the pixel block to be estimated centered at pixel i; is the similarity comparison block centered on pixel j; is the pixel value of the similarity comparison block centered on pixel j, a>0 is the standard deviation of the Gaussian kernel, h is the degree of filtering, that is, the smoothing parameter, which controls the attenuation degree of the Gaussian function, Z(i) is the normalization coefficient; e is a natural constant;

[0077] (3) By performing weighted averaging on these regions, we can obtain the denoised image sequence, as shown below:

[0078]

[0079] In the formula, represents the pixel value of pixel i after denoising; v(j) represents the pixel value of pixel j; {v(j)} represents the set of v(j) generated by the similarity comparison block in the process of sliding in the search area.

[0080] The steps for obtaining the vibration displacement of the target cable in the target image sequence are as follows:

[0081] In this embodiment, a deep learning algorithm is used to obtain the vibration displacement of the target cable in the image sequence, including: using a recursive full-field transform based on deep learning to estimate the optical flow of the image sequence in the area of ​​the target cable to be identified; and extracting the vibration displacement of the target cable through the full-field optical flow estimation result. Figure 4 This is a schematic diagram of the optical flow estimation process based on deep learning of the present invention, including: extracting the feature vector of each pixel from the continuous frame image through the feature and context encoder; constructing a 4D correlation space volume by calculating the inner product of all feature vector pairs, and performing multi-scale pooling on the last 2 dimensions of the 4D correlation volume to obtain a set of multi-scale volumes; defining a search operator and searching for the required features from the correlation space pyramid, and then using them as input for iterative updates; based on the GRU cyclic update operator, retrieving values ​​from the correlation volume and iteratively updating the optical flow field initialized to zero to obtain the full-field optical flow estimation of the image sequence in the area of ​​the target cable to be identified. The specific steps are as follows:

[0082] (1) Feature encoding extracts dense features of two consecutive images pixel by pixel. This process will gradually downsample to 1 / 8 resolution at the original image resolution. The feature extraction module architecture of context encoding is the same as feature extraction, but it will not be used for optical flow loop iteration.

[0083] (2) A 4D correlation space volume is constructed by calculating the inner product of all feature vector pairs. Then, the correlation space volume is pooled with pooling kernels of size 1, 2, 4, and 8 to construct a set of multi-scale volumes, a 4-layer pyramid {C 1 ,C 2 ,C 3 ,C 4}. It is worth noting that the dimension of C is H×W×H / 2q×W / 2q, which provides information about the large and small displacements of the object. By keeping the first two dimensions of C unchanged, the high-resolution information of the image is maintained, so that the movement of small objects moving quickly can be captured, which is beneficial for identifying the vibration displacement of the cable. The 4D correlation space can be expressed as follows:

[0084]

[0085] C=∑g θ (I1) g θ (I2)

[0086] In the formula, g θ (I1) and g θ (I2) represents the features of images I1 and I2 respectively;

[0087] (3) Assume that the optical flow of each pixel in the p-th frame image is x = (u, v), and the optical flow estimation state of the p+1-th frame image is x = (f u ,f v ), then the corresponding optical flow of each pixel in the pth frame mapped to the p+1th frame image is x′=(u+f u ,v+f v ), u is the optical flow value in the horizontal direction, v is the optical flow value in the vertical direction, f u is the horizontal optical flow value of the optical flow estimation of the p+1th frame image; f v The vertical optical flow value of the optical flow estimation of the p+1th frame image. Then, the search is performed on all levels of the pyramid through the defined search operator to find the location of the neighborhood of x′ in the relevant space volume to generate a feature map, which is then used as the input for iterative update.

[0088] (4) A recurrent update operator based on GRU (gated activation unit) retrieves values ​​from the relevant spatial volume and iteratively updates the optical flow field initialized to zero to obtain the full-field optical flow estimation of the image sequence in the area of ​​the target cable to be identified.

[0089] Figure 5 This is a schematic diagram of extracting the vibration displacement of the target cable based on the full-field optical flow estimation in the present invention. The optical flow result represents the motion vector of each pixel in the first frame on the second frame. The color represents the motion direction, and the depth of the color represents the modulus of the motion vector. Therefore, the motion displacement of the target cable can be determined by extracting the modulus of the motion vector of the optical flow at the target cable position. Here, the vibration displacement of the cable is calculated by averaging the optical flow of the target cable area.

[0090] The steps to obtain the modal components and real-time frequencies of the target cable are as follows:

[0091] In this embodiment, the vibration displacement is subjected to variational modal decomposition transformation to obtain the modal components and real-time frequency of the target cable, including: performing variational modal decomposition on the vibration displacement to obtain each modal component of the target cable; performing Hilbert transformation on each modal component to obtain the real-time frequency of the target cable. Figure 6 The figure is a schematic diagram of the modal components and real-time frequency calculation process of the target cable in the present invention. The variational modal decomposition is used to decompose the cable vibration displacement time history to obtain the response data of all modal components. The specific steps are as follows:

[0092] (1) Provide the vibration displacement signal F(t), set the initial number of decomposition modes k, penalty factor α, number of iterations b and iteration tolerance σ, and initialize the first modal function and the first mode center frequency

[0093] (2) When satisfying ω K ≥0, update each modal function Each center frequency ω k , as shown below:

[0094]

[0095] (3) Update the Lagrange multiplier λ, where ε represents the noise tolerance parameter, as shown below:

[0096]

[0097] Where ω is the frequency; To analyze the signal; is the Lagrange multiplier of the iterative process;

[0098] (4) Update the number of iterations b = b + 1;

[0099] (5) Repeat steps (2) to (4) until the termination objective function is satisfied, as shown below:

[0100]

[0101] (6) Through the above steps, we can obtain the k modal components with the smallest sum of bandwidths: f(t) = F(t) - Σ k u k (t), k = k + 1, which is the variational mode decomposition result in the figure.

[0102] The steps for calculating the real-time cable force of the target cable are as follows:

[0103] Perform fast Fourier transform on the modal components to determine the natural frequencies and corresponding orders of the cables; select the real-time frequency and the order of the corresponding natural frequency to calculate the real-time cable force of the target cable. The expression for fast Fourier transform of each modal component is as follows:

[0104]

[0105] Where f(t) represents the input modal component, M represents the length of the time domain data, f(ω) represents the frequency domain data of each modal component after fast Fourier transform, and J is a complex number.

[0106] In this embodiment, the frequency domain data after fast Fourier transformation, and the frequency domain data of each modal component will have a peak frequency, and it is difficult to distinguish these peak frequencies manually to which the natural frequencies of the cable belong. Therefore, the frequency in the cable force equation can be divided by the order corresponding to the frequency as a constant to estimate the natural frequencies of the cable to which these peak frequencies belong. The vibration equation of the cable and the cable force equation of the cable are as follows:

[0107]

[0108] Where EI is the bending stiffness of the cable, T is the real-time cable force, ρ is the linear density of the cable, L is the length of the cable, and f n is the nth order vibration real-time frequency of the cable, n is the order of the natural frequency of the cable, u is the vibration displacement of the cable, x is the vibration direction of the cable, and t is time.

[0109] It can be seen that by using this embodiment, as long as we know the order of vibration response of each modal component to which the cable belongs, we can select the order and its corresponding real-time frequency to calculate the real-time cable force of the cable using the cable force vibration equation. Figure 7 The real-time cable force identification result diagram of the target cable of the present invention is obtained by performing Hilbert transform on each modal component. n And the corresponding natural frequency order n is calculated.

[0110] The modal components are subjected to fast Fourier transform to obtain the natural frequencies of each order and the corresponding orders, as shown in Table 2 below.

[0111]

[0112] In addition, the vibration of the target cable was measured by a traditional contact acceleration sensor, and its measured natural frequency and corresponding order were compared with the judgment results of each order of natural frequency and corresponding order of this embodiment. It can be seen from the table that the judgment results are in good agreement, and the maximum error is 1.14%. Figure 8This is a comparison diagram of the real-time cable force identification results of the target cable in the present invention. It can be seen that the accuracy of the identified real-time cable force of the cable is high, which can provide a simple and reliable measurement method for the real-time cable force measurement of the cable under environmental excitation or vehicle load.

[0113] The present invention also provides a real-time cable force identification device based on deep learning. Fig. 9 The present invention is a module schematic diagram of the real-time cable force identification device, including: a vibration image information acquisition module, which is used to collect the original video of the punctuation camera parameters and the vibration of the cable of the cable-stayed bridge, and correct the original video to obtain the vibration image information of the target cable; a target image sequence extraction module, which is used to rotate, crop and filter the vibration image information to obtain the image sequence of the area of ​​the target cable to be identified; a vibration displacement extraction module, which is used to obtain the vibration displacement of the target cable in the target image sequence based on the optical flow estimation algorithm of deep learning; a modal component and real-time frequency calculation module, which is used to perform Hilbert-Huang transform on the vibration displacement of the target cable to obtain the modal component and real-time frequency of the target cable; a module for determining the natural frequencies of each order and the corresponding order of the target cable, which is used to perform fast Fourier transform on the modal components to determine the natural frequencies of each order and the corresponding order of the target cable; a real-time cable force calculation module, which is used to calculate the real-time cable force of the target cable based on the real-time frequency and the order of the corresponding natural frequency.

[0114] An embodiment of the present application also provides an electronic device. Fig.10 It is a schematic diagram of the structure of the computer device provided in the embodiment of the present application, including: at least one collector, at least one processor, a memory, a power supply, an input interface, an output interface and a communication bus. Among them, the memory is used to store operating systems, computer programs, deep learning models, training data, processed data files and other necessary configuration files, and the memory content is executed by the processor to implement the relevant steps in the real-time cable force identification method based on deep learning performed by the electronic device disclosed in the embodiment. In addition, the memory, as a carrier for resource storage, can be a storage medium such as a read-only memory, a hard disk or an optical disk. The memory can include random access as a running memory and storage purposes of an external memory. The storage resources thereon include operating systems, computer programs, deep learning models, other files, etc.

[0115] In this embodiment, the power supply is used to provide voltage for various hardware devices on the electronic device; the input interface and the output interface are used to create a data transmission channel between the electronic device and the external device, and the specific interface type can be selected according to the specific situation and is not specifically limited here; the processor can include a multi-core processor to increase the operating speed; the collector is used to capture the ambient light intensity information recorded by the camera lens through an optical photosensitive element and convert it into an electrical signal. These signals are processed by an analog-to-digital converter and converted into digital data, that is, the pixel number on each frame of the image. Modern cameras use image sensors (such as CCD or CMOS), which are composed of multiple photosensitive units (pixels), each pixel capturing light of a specific color. The image data is saved in the memory and can be displayed on the screen or output as a file, such as JPEG, RAW and other formats. The operating system is used to manage the computer's hardware resources, such as CPU, memory and storage devices; control and schedule running programs or processes; provide users with an interactive interface to facilitate computer operation; manage and control various hardware devices and computer programs on the electronic device on the source host. The operating system used in this embodiment is Windows system, which can also be Unix, Linux, etc.

[0116] In addition, the deep learning model is mainly used to realize the optical flow estimation of the image sequence of the area of ​​the target cable to be identified by recursive full-field transformation based on deep learning. This model is a specific optical flow estimation pre-training model using a deep learning algorithm in this embodiment, and can also be applied to embodiments of different scenarios through secondary optimization updates.

[0117] The present application accordingly provides a method combining an optical flow algorithm based on deep learning and a variational mode decomposition transformation to identify the real-time cable force, which improves the recognition accuracy of the target cable vibration displacement and the robustness of the real-time cable force recognition. It has the advantages of strong anti-noise ability, simple operation, strong real-time performance and secondary optimization update.

[0118] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A real-time cable force identification method based on deep learning, characterized in that: The following steps are involved: Acquiring vibration image information of a target cable, wherein the target cable is a cable of a cable-stayed bridge to be identified; Obtaining a target image sequence based on the vibration image information, wherein the target image sequence is an image sequence of a target cable region to be identified; The optical flow estimation algorithm based on deep learning is used to obtain the vibration displacement of the target cable in the target image sequence; Performing variational modal decomposition transformation on the vibration displacement to obtain modal components and real-time frequencies of the target cable; Performing fast Fourier transform on the modal components to determine the natural frequencies of each order and the corresponding orders of the target cable; The real-time cable force of the target cable is calculated based on the real-time frequency and the order of its corresponding natural frequency.

2. The method for real-time cable force identification based on deep learning according to claim 1 is characterized in that: The steps of using the deep learning-based optical flow estimation algorithm to obtain the vibration displacement of the target cable in the target image sequence include: Using a recursive full-field transform based on deep learning to perform optical flow estimation on the target image sequence to obtain an optical flow estimation result; The vibration displacement of the target cable is extracted based on the optical flow estimation result.

3. The method of obtaining the vibration displacement of the target cable in the target image sequence by using the deep learning-based optical flow estimation algorithm according to claim 2, characterized in that: The optical flow of the target image sequence is estimated by using a recursive full field transform based on deep learning, and the optical flow estimation results include: Extracting feature vectors for each pixel from consecutive frame images through feature and context encoders; By using the eigenvectors, the inner products of all eigenvector pairs are calculated to construct a 4D correlation space body; A search operator is defined to search for required features from the relevant space volume, and then used as input for iterative updating; Based on the GRU recurrent update operator, the full-field optical flow of the image sequence of the target cable area is identified.

4. The method for real-time cable force identification based on deep learning according to claim 1 is characterized in that: The steps of obtaining the modal components and real-time frequencies of the target cable include: Performing variational modal decomposition on the vibration displacement to obtain each modal component of the target cable; Perform Hilbert transform on each modal component to obtain the real-time frequency of the target cable.

5. The method for obtaining the modal components and real-time frequency of the target cable according to claim 4, characterized in that: The steps to obtain each modal component of the target cable include: For the vibration displacement signal, the initial number of decomposition modes, penalty factors and iteration parameters are set, and the objective function is obtained through updating iteration; Based on the objective function and the constructed variational modal decomposition transformation model, the optimal center frequency and finite bandwidth of each mode are adaptively searched and solved, so as to achieve effective separation of inherent modal components, frequency domain division of signals, and then obtain the effective modal decomposition components of a given signal.

6. The method for obtaining the modal components and real-time frequency of the target cable according to claim 4, characterized in that: The steps of obtaining the target cable real-time frequency include: The signal of the selected modal component is transformed into an analytical signal on a two-dimensional complex plane, wherein the mapping method of the analytical signal is that the real part and the imaginary part of the complex number are Hilbert transforms of each other; By analyzing the ratio of the imaginary part and the real part of the signal, the instantaneous phase is obtained; The instantaneous phase is differentiated in the time domain to obtain the real-time frequency of the target cable.

7. The method for real-time cable force identification based on deep learning according to claim 1 is characterized in that: The steps of determining the natural frequencies of the target cable and the corresponding orders include: Performing fast Fourier transformation on the modal components to obtain frequency domain data of each component, wherein the frequency domain data includes the peak frequency of each component; According to the characteristic that the natural frequency values ​​of each order of the cable vibration satisfy the arithmetic progression and the peak frequency values, the natural frequencies of each order of the cable and the corresponding orders are determined.

8. The method for real-time cable force identification based on deep learning according to claim 1, characterized in that: The real-time cable force of the target cable is calculated based on the real-time frequency and the order of its corresponding natural frequency, including: Substitute the real-time frequency and the order of its corresponding natural frequency into the cable force equation to obtain the real-time cable force of the target cable; The cable tension equation is expressed as follows: Where T is the real-time cable force, ρ is the linear density of the cable, L is the length of the cable, and f n is the nth order vibration real-time frequency of the cable, and n is the order of the natural frequency of the cable.

9. A real-time cable force identification device based on deep learning, used to implement the real-time cable force identification method based on deep learning as claimed in claims 1 to 8, characterized in that: include: The vibration image information acquisition module is used to collect the original video of the punctuation camera parameters and the vibration of the cable-stayed bridge cable, and correct the original video to obtain the vibration image information of the target cable; A target image sequence extraction module is used to rotate, crop and filter the vibration image information to obtain an image sequence of the area of ​​the target cable to be identified; A vibration displacement extraction module is used to obtain the vibration displacement of the target cable in the target image sequence based on the optical flow estimation algorithm based on deep learning; A modal component and real-time frequency calculation module is used to perform Hilbert-Huang transform on the vibration displacement of the target cable to obtain the modal component and real-time frequency of the target cable; A module for determining each order of natural frequencies and corresponding orders of a target cable, for performing fast Fourier transform on modal components to determine each order of natural frequencies and corresponding orders of the target cable; The real-time cable force calculation module is used to calculate the real-time cable force of the target cable based on the real-time frequency and the order of its corresponding natural frequency.

Images