Automatic measurement system and method based on computer vision

Abstract

The invention discloses an automatic measurement system and method based on computer vision, and the method comprises the steps: carrying out the target tracking of a vibration video of a bridge cable containing a target point through employing a computer vision technology, so as to obtain the motion time-history data of the cable, and further obtaining the motion time-history data of the cable; frequency domain analysis based on variational mode decomposition is carried out on motion time history data of the inhaul cable to capture waveform semantic features of each frequency domain intrinsic mode component of the motion time history data of the inhaul cable, then significance modulation dynamic aggregation is carried out on the feature energy distribution based on each frequency domain intrinsic mode component, and the motion time history data of the inhaul cable is obtained. And a full-frequency-domain waveform oscillation mode of time-history data of the movement of the inhaul cable is captured, so that intelligent estimation of the vibration frequency of the inhaul cable and cable force calculation are realized. In this way, the limitation of a traditional contact type measurement method can be effectively avoided, meanwhile, dependence on physical attribute information of the inhaul cable is reduced, and the accuracy and efficiency of inhaul cable force measurement are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent measurement, and more specifically, to an automated measurement system and method based on computer vision. Background Art

[0002] In the construction of long-span bridges, cable-stayed bridges have become the preferred structure for their excellent spanning capacity, beautiful appearance, and high cost-effectiveness. As the main load-bearing component of a cable-stayed bridge, the accurate measurement of the cable force is crucial to the safe operation of the bridge. Traditional cable force measurement methods usually rely on contact sensors, which are not only costly to install and maintain, but may also cause damage to the bridge structure during long-term use.

[0003] Traditional bridge cable force measurement methods mainly include tensiometer measurement, strain gauge measurement, pressure sensor measurement, and acceleration sensor measurement. However, these methods are contact measurement methods, which have problems such as difficult installation and high maintenance costs.

[0004] In recent years, with the rapid development of computer vision technology, measurement methods based on computer vision have gradually become a research hotspot. For example, the invention patent with publication number CN118505743A discloses a bridge cable force measurement method and system based on computer vision, which tracks the target point in the vibration video of the bridge cable through a preset tracking algorithm to obtain the motion time-history data of the cable, and uses the EEMD method to process the motion time-history data of the cable to obtain the corresponding multiple intrinsic mode functions, and according to the vibration frequency characteristics of the cable, eliminates the intrinsic mode functions that are not related to the cable vibration, reconstructs the remaining intrinsic mode functions, and then determines the vibration frequency and cable force of the cable based on the reconstructed motion time-history data. In this way, the physical contact requirement for equipment in traditional measurement methods is avoided, remote operation is achieved, and the safety measures and equipment investment required for on-site operations are reduced, reducing labor costs and equipment maintenance costs.

[0005] However, in the above-mentioned bridge cable force measurement scheme, the vibration frequency range of the cable is determined by collecting the physical property information such as the cable model, material, diameter, calculated length, line density, and inclination angle, so as to eliminate the false eigenmode function components. The effectiveness of this method is highly dependent on the accurate understanding of the physical properties of the cable, which may be difficult to obtain or contain errors in practical applications, which may lead to inaccurate determination of the frequency range. In addition, affected by the modal aliasing phenomenon of the EEMD algorithm, some useful IMF components may be misjudged as noise or false components, resulting in the loss of useful information and affecting the calculation accuracy of the vibration frequency and cable force of the cable.

[0006] Therefore, an optimized computer vision-based automated measurement method is expected. Summary of the invention

[0007] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an automated measurement system and method based on computer vision, which uses computer vision technology to track the vibration video of the bridge cable containing the target point to obtain the motion time history data of the cable, and then, by performing frequency domain analysis based on variational mode decomposition on the motion time history data of the cable, the waveform semantic features of each frequency domain intrinsic mode component of the cable motion time history data are captured, and then, based on the characteristic energy distribution of each frequency domain intrinsic mode component, it is significantly modulated and dynamically aggregated to capture the full frequency domain waveform oscillation mode of the cable motion time history data, so as to realize the intelligent estimation of the cable vibration frequency and the calculation of the cable force. In this way, the limitations of the traditional contact measurement method can be effectively avoided, while reducing the dependence on the physical property information of the cable, and improving the accuracy and efficiency of the cable force measurement.

[0008] According to one aspect of the present application, there is provided an automated measurement method based on computer vision, comprising:

[0009] Obtain a vibration video of the bridge cable including the target point collected by the camera;

[0010] Based on a preset tracking algorithm, target tracking is performed on the target point in the vibration video to obtain motion time history data of the cable;

[0011] Performing frequency domain analysis based on modal decomposition on the motion time history data of the cable to obtain a set of semantic feature vectors of waveforms of frequency domain intrinsic modal components of the motion time history;

[0012] Performing full-frequency-domain feature significant aggregation on the set of semantic feature vectors of the frequency-domain intrinsic modal component waveform of the motion time course to obtain a full-frequency-domain significant aggregation representation vector of the motion time course;

[0013] The cable force value of the cable is calculated based on the significant aggregation representation vector of the motion time course in the full frequency domain.

[0014] According to another aspect of the present application, there is provided an automated measurement system based on computer vision, comprising:

[0015] A vibration video acquisition module is used to acquire a vibration video of a bridge cable including a target point collected by a camera;

[0016] A target tracking module, used for tracking the target point in the vibration video based on a preset tracking algorithm to obtain the motion time history data of the cable;

[0017] A frequency domain analysis module, used for performing frequency domain analysis on the motion time history data of the cable based on modal decomposition to obtain a set of semantic feature vectors of waveforms of intrinsic modal components in the motion time history frequency domain;

[0018] A full-frequency domain feature significant aggregation module is used to perform full-frequency domain feature significant aggregation on the set of semantic feature vectors of the intrinsic modal component waveform of the motion time course frequency domain to obtain a full-frequency domain significant aggregation representation vector of the motion time course;

[0019] The cable force value calculation module is used to calculate the cable force value of the cable based on the significant aggregation representation vector of the motion time course in the full frequency domain.

[0020] Compared with the prior art, the present application provides an automated measurement system and method based on computer vision, which uses computer vision technology to track the vibration video of the bridge cable containing the target point to obtain the motion time history data of the cable, and then, by performing frequency domain analysis based on variational mode decomposition on the motion time history data of the cable, the waveform semantic features of each frequency domain intrinsic mode component of the cable motion time history data are captured, and then, based on the characteristic energy distribution of each frequency domain intrinsic mode component, the significant modulation dynamic aggregation is performed to capture the full frequency domain waveform oscillation mode of the cable motion time history data, so as to realize the intelligent estimation of the cable vibration frequency and the calculation of the cable force. In this way, the limitations of the traditional contact measurement method can be effectively avoided, while reducing the dependence on the physical property information of the cable, and improving the accuracy and efficiency of the cable force measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 is a flowchart of an automated measurement method based on computer vision according to an embodiment of the present application;

[0023] Figure 2 A schematic diagram of data flow of an automated measurement method based on computer vision according to an embodiment of the present application;

[0024] Figure 3 is a flowchart of sub-step S4 of the computer vision-based automated measurement method according to an embodiment of the present application;

[0025] Figure 4 4 is a block diagram of an automated measurement system based on computer vision according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0027] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0028] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0029] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0030] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0031] In the technical solution of the present application, an automated measurement method based on computer vision is proposed. Figure 1 The figure is a flowchart of an automated measurement method based on computer vision according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of data flow of an automated measurement method based on computer vision according to an embodiment of the present application. Figure 1 and Figure 2As shown, according to the embodiment of the present application, the automatic measurement method based on computer vision includes the steps of: S1, obtaining a vibration video of a bridge cable containing a target point collected by a camera; S2, based on a preset tracking algorithm, tracking the target point in the vibration video to obtain the motion time-history data of the cable; S3, performing frequency domain analysis based on modal decomposition on the motion time-history data of the cable to obtain a set of semantic feature vectors of the waveform of the frequency domain eigenmodal component of the motion time-history; S4, performing full-frequency domain feature significant aggregation on the set of semantic feature vectors of the waveform of the frequency domain eigenmodal component of the motion time-history to obtain a full-frequency domain significant aggregation representation vector of the motion time-history; S5, calculating the cable force value of the cable based on the full-frequency domain significant aggregation representation vector of the motion time-history.

[0032] In particular, the S1, obtains a vibration video of the bridge cable including the target point collected by the camera. It should be understood that using the camera to collect the video can realize a non-contact measurement method, avoiding the complexity of the installation of traditional physical sensors and possible interference or damage to the bridge structure. Among them, the target point can be a specific mark or natural feature point on the cable, which has a clear geometric shape and is easy to identify and track.

[0033] In particular, the S2, based on a preset tracking algorithm, tracks the target point in the vibration video to obtain the motion time-course data of the cable. That is, in the technical solution of the present application, by tracking the target point in the vibration video to obtain the position change of the target point in each video frame, the motion trajectory information of the cable is obtained, providing a reliable data basis for subsequent vibration frequency analysis and cable force calculation. In a specific implementation of the present application, the vibration video is firstly sampled based on a predetermined frequency, and then a target detection network (such as YOLO, SSD or Faster R-CNN, etc.) is used to detect the target point, and its position coordinates in each frame are extracted to obtain the motion time-course data of the cable.

[0034] In particular, the S3 performs frequency domain analysis based on modal decomposition on the motion time history data of the cable to obtain a set of waveform semantic feature vectors of the motion time history frequency domain intrinsic modal components. In a specific example of the present application, the specific process of performing frequency domain analysis based on modal decomposition on the motion time history data of the cable motion includes: first, performing variational modal decomposition on the motion time history data of the cable motion to obtain a set of motion time history frequency domain intrinsic modal components; since the motion time history data of the bridge cable usually contains multiple different frequency and amplitude components, reflecting different physical phenomena and dynamic behaviors. Therefore, in order to capture the vibration behavior mode of the cable more carefully, the present application further uses the variational modal decomposition (VMD) method to decompose the motion time history data of the cable to extract multiple motion time history frequency domain intrinsic modal components with fixed center frequency and bandwidth. Among them, each motion time history frequency domain intrinsic modal component represents the cable vibration mode within a specific frequency range, which helps to more clearly analyze and understand the various components of the cable motion time history data, thereby more accurately estimating the vibration frequency of the cable. Those skilled in the art should know that, compared with the EEMD method, the VMD method has better performance in processing nonlinear and non-stationary signals. It solves the optimization problem by using the alternating direction multiplier method (ADMM) to ensure that the bandwidth of each modal component is minimized, which can more effectively reduce the occurrence of modal aliasing and provide a more reliable basis for subsequent frequency analysis. Then, each motion time frequency domain intrinsic mode component in the set of motion time frequency domain intrinsic skyscraper components is input into the intrinsic mode component waveform semantic feature extractor based on the hole convolution layer to obtain the set of semantic feature vectors of the motion time frequency domain intrinsic mode component waveform. Here, in order to further extract the cable vibration waveform pattern characteristics contained in each motion time frequency domain intrinsic mode component, the present application adopts a hole convolutional neural network to construct an intrinsic mode component waveform semantic feature extractor to enhance the depth and breadth of feature extraction. The atrous convolution layer can expand the receptive field without increasing the number of parameters by introducing holes (i.e., gaps) in the convolution kernel, and capture a wider range of contextual information, so as to better understand the waveform oscillation patterns of each motion time-domain intrinsic mode component in the frequency domain, and obtain a set of semantic feature vectors of the waveform of the motion time-domain intrinsic mode component in the frequency domain.

[0035] In particular, the S4 performs full-frequency domain feature significant aggregation on the set of semantic feature vectors of the waveform of the frequency domain intrinsic modal component of the motion time course to obtain a full-frequency domain significant aggregation representation vector of the motion time course. That is, in the technical solution of the present application, in order to comprehensively analyze the motion characteristics of the cable from a global perspective, the present application further performs feature aggregation on the set of semantic feature vectors of the waveform of the frequency domain intrinsic modal component of the motion time course to form a full-frequency domain cable vibration waveform feature representation. It is worth mentioning that, considering that different intrinsic modal components in the set of semantic feature vectors of the waveform of the frequency domain intrinsic modal component of the motion time course have different importance, therefore, in order to reduce the influence of redundant information and noise, the present application proposes a dynamic aggregation method based on feature energy distribution, which analyzes the relative importance of each intrinsic modal component and dynamically adjusts its weight in the aggregation process to achieve the enhancement of important features and the suppression of minor features, thereby obtaining a more discriminative and representative full-frequency domain significant aggregation representation vector of the motion time course. Specifically, in a specific example of the present application, such as Figure 3 As shown, S4 includes: S41, determining the initial center vector of the motion time frequency domain feature clustering based on the characteristic energy distribution of the set of semantic feature vectors of the motion time frequency domain intrinsic modal component waveforms; S42, based on the spatial span of each motion time frequency domain intrinsic modal component waveform semantic feature vector in the set of motion time frequency domain intrinsic modal component waveform semantic feature vectors relative to the initial center vector of the motion time frequency domain feature clustering, performing explicit dynamic aggregation on the set of motion time frequency domain intrinsic modal component waveform semantic feature vectors to obtain the motion time full frequency domain significant aggregation representation vector.

[0036] Specifically, the S41 determines the initial center vector of the motion time frequency domain feature clustering based on the characteristic energy distribution of the set of semantic feature vectors of the motion time frequency domain intrinsic modal component waveform. Specifically, first, the static energy factor of each motion time frequency domain intrinsic modal component waveform semantic feature vector in the set of semantic feature vectors of the motion time frequency domain intrinsic modal component waveform is calculated to obtain a set of static energy factors of the motion time frequency domain intrinsic modal component; then, the motion time frequency domain intrinsic modal component waveform semantic feature vector corresponding to the maximum value in the set of static energy factors of the motion time frequency domain intrinsic modal component is selected as the initial center vector of the motion time frequency domain feature clustering. That is, the static energy factor of each motion time-domain intrinsic modal component waveform semantic feature vector is calculated based on the predefined metric function to evaluate its stability and importance in the feature space, and the motion time-domain intrinsic modal component waveform semantic feature vector corresponding to the largest static energy factor is selected as the initial center vector of motion time-domain frequency domain feature clustering to determine the cluster core of the motion time-domain frequency domain feature distribution, thereby providing a stable reference point for subsequent feature aggregation.

[0037] Among them, the specific process of calculating the static energy factor of each motion time-domain intrinsic mode component waveform semantic feature vector in the set of motion time-domain intrinsic mode component waveform semantic feature vectors includes: calculating the kurtosis of the motion time-domain intrinsic mode component waveform semantic feature vector, and inputting the kurtosis into the sigmoid activation function to obtain the static energy factor of the motion time-domain intrinsic mode component.

[0038] Specifically, S42, based on the spatial span of each motion time-course frequency domain intrinsic modal component waveform semantic feature vector in the set of motion time-course frequency domain intrinsic modal component waveform semantic feature vectors relative to the initial center vector of the motion time-course frequency domain feature clustering, performs explicit dynamic aggregation on the set of motion time-course frequency domain intrinsic modal component waveform semantic feature vectors to obtain the motion time-course full frequency domain significant aggregation representation vector. Specifically, first, based on the spatial span between each motion time frequency domain intrinsic modal component waveform semantic feature vector in the set of motion time frequency domain intrinsic modal component waveform semantic feature vectors and the initial center vector of the motion time frequency domain feature clustering, and the set of static energy factors of the motion time frequency domain intrinsic modal components, the dynamic aggregation energy factor of each motion time frequency domain intrinsic modal component waveform semantic feature vector in the set of motion time frequency domain intrinsic modal component waveform semantic feature vectors is calculated to obtain a set of dynamic aggregation energy factors of motion time frequency domain intrinsic modal components; then, based on the set of dynamic aggregation energy factors of motion time frequency domain intrinsic modal components, the set of motion time frequency domain intrinsic modal component waveform semantic feature vectors is significantly modulated and aggregated to obtain the motion time full frequency domain significant aggregation representation vector. That is, by considering the feature space proximity of each motion time-course frequency domain intrinsic mode component waveform semantic feature vector relative to the motion time-course frequency domain feature distribution cluster core and the importance difference of the intrinsic attributes, its contribution to the global frequency domain feature of the motion time-course data can be more accurately measured. Then, based on the gated mask mechanism, the obtained dynamic aggregation energy factor is nonlinearly transformed and screened to generate corresponding weights, so as to perform weighted aggregation on the sequence of the motion time-course frequency domain intrinsic mode component waveform semantic feature vectors, thereby enhancing the expression of important features and suppressing the influence of irrelevant or noise features, so as to obtain a significant aggregation representation vector of the full frequency domain of the motion time-course.

[0039] Among them, the specific process of calculating the dynamic aggregation energy factor of each motion time-domain intrinsic modal component waveform semantic feature vector in the set of motion time-domain intrinsic modal component waveform semantic feature vectors includes: taking the square value of the number of feature vectors between the motion time-domain intrinsic modal component waveform semantic feature vector and the initial center vector of the motion time-domain frequency domain feature cluster as the spatial span coefficient, and calculating the weighted ratio between the product of the static energy factor of the motion time-domain intrinsic modal component waveform semantic feature vector and the static energy factor of the initial center vector of the motion time-domain frequency domain feature cluster and the spatial span coefficient to obtain the dynamic aggregation energy factor of the motion time-domain intrinsic modal component.

[0040] More specifically, based on the set of dynamic aggregation energy factors of the frequency domain intrinsic modal components of the motion time course, the specific process of performing significant modulation aggregation on the set of waveform semantic feature vectors of the frequency domain intrinsic modal components of the motion time course includes: inputting the set of dynamic aggregation energy factors of the frequency domain intrinsic modal components of the motion time course into a gated mask unit to obtain a set of dynamic aggregation weight factors of the frequency domain intrinsic modal components of the motion time course; and calculating the weighted sum of the set of waveform semantic feature vectors of the frequency domain intrinsic modal components of the motion time course based on the set of dynamic aggregation weight factors of the frequency domain intrinsic modal components of the motion time course to obtain the significant aggregation representation vector of the entire frequency domain of the motion time course. Among them, the set of dynamic aggregation energy factors of the frequency domain intrinsic modal components of the motion time history is input into the gated mask unit to obtain the set of dynamic aggregation weight factors of the frequency domain intrinsic modal components of the motion time history, including: inputting the set of dynamic aggregation energy factors of the frequency domain intrinsic modal components of the motion time history into the sigmoid function for normalization processing to obtain the set of normalized dynamic aggregation energy factors of the frequency domain intrinsic modal components of the motion time history; based on the preset mask threshold, inactivating the set of normalized dynamic aggregation energy factors of the frequency domain intrinsic modal components of the motion time history to obtain the set of dynamic aggregation weight factors of the frequency domain intrinsic modal components of the motion time history.

[0041] In summary, in the above embodiment, the set of semantic feature vectors of the waveform of the intrinsic modal component of the motion time course frequency domain is subjected to full-frequency domain feature significant aggregation to obtain a full-frequency domain significant aggregation representation vector of the motion time course, including: processing the set of semantic feature vectors of the waveform of the intrinsic modal component of the motion time course frequency domain with the following feature dynamic aggregation formula to obtain the full-frequency domain significant aggregation representation vector of the motion time course, wherein the feature dynamic aggregation formula is:

[0042] X={x1,x2,...,x i , ..., x n}

[0043]

[0044] x c =x m

[0045]

[0046]

[0047]

[0048] w si =mask(w i )

[0049]

[0050]

[0051] Wherein, X represents the set of semantic feature vectors of the waveform of the intrinsic modal component of the motion time course in the frequency domain, x1, x2, x i 、x m and x n They respectively represent the first, second, i-th, m-th and n-th semantic feature vectors of the waveform of the intrinsic modal component of the motion time course frequency domain in the set of semantic feature vectors of the waveform of the intrinsic modal component of the motion time course frequency domain, and the value of n is the number of semantic feature vectors of the waveform of the intrinsic modal component of the motion time course frequency domain, represents the eigenvalue of the jth position in the semantic feature vector of the waveform of the eigenmodal component of the i-th motion time course frequency domain, μ i and σ i 4 denote the characteristic mean and the square of the characteristic variance of the semantic feature vector of the frequency-domain intrinsic modal component waveform of the i-th motion time course, E{·} denotes the expected value of the calculation set, sigmoid denotes the sigmoid activation function, represents the static energy factor of the semantic feature vector of the waveform of the ith motion time-domain intrinsic mode component, argmax represents the index corresponding to the maximum value, m represents the index of the largest static energy factor in the set of static energy factors of the motion time-domain intrinsic mode component, x c represents the initial center vector of the motion time frequency domain feature clustering, represents the static energy factor of the initial center vector of the motion time frequency domain feature clustering, a and b are different weight parameters, Count(x i →x m ) represents the number of feature vectors between the semantic feature vector of the ith motion time-course frequency-domain intrinsic modal component waveform and the initial center vector of the motion time-course frequency-domain feature clustering, represents the dynamic aggregation energy factor of the semantic feature vector of the waveform of the ith motion time-course frequency-domain intrinsic modal component, w i represents the i-th normalized dynamic aggregation energy factor, mask(·) represents mask processing, θ is the preset mask threshold, and w si represents the i-th dynamic aggregation weight factor, x f A significant aggregation representation vector representing the entire frequency domain of the motion time course.

[0052] In particular, the S5 calculates the cable force value of the cable based on the significant aggregation representation vector of the full frequency domain of the motion time course. In a specific example of the present application, the significant aggregation representation vector of the full frequency domain of the motion time course is input into the cable vibration frequency estimation module based on the decoder to obtain the estimated value of the cable vibration frequency; it should be understood that the decoder model can learn the mapping relationship from the significant aggregation representation vector of the full frequency domain of the motion time course to the cable vibration frequency through a large amount of training, thereby realizing the accurate estimation of the cable vibration frequency. In the technical solution of the present application, the decoder model performs feature learning and analysis on the significant aggregation representation vector of the full frequency domain of the motion time course through a multi-layer neural network structure, which can effectively capture the complex vibration mode contained therein and convert it into an estimated value of the vibration frequency. Further, based on the estimated value of the cable vibration frequency, the cable force value of the cable is calculated. That is, based on the relationship between the vibration frequency of the cable and its tension, mass, and length, according to the estimated value of the cable vibration frequency, and combined with physical parameters such as the length, mass, and tension coefficient of the bridge cable, the cable force of the cable is calculated. Specifically, the cable force calculation formula is: F = (4ml 2 f / n 2 g)-(EIπ 2 n 2 / l 2 ), where m is the linear mass per unit length, f is the nth order natural frequency of the cable, l is the vibration length of the cable, g is the acceleration of gravity, n is the vibration order, EI is the bending stiffness EI of the cable, and π is pi.

[0053] In the technical solution of the present application, each motion time frequency domain intrinsic modal component waveform semantic feature vector in the set of motion time frequency domain intrinsic modal component waveform semantic feature vectors respectively represents the waveform semantic coding features of each motion time frequency domain intrinsic modal component determined by variational modal decomposition. When it is input into the feature distribution field dynamic aggregation network based on the force field simulation, the waveform semantic confidence difference and noise ratio between each motion time frequency domain intrinsic modal component waveform semantic feature vector in the set of motion time frequency domain intrinsic modal component waveform semantic feature vectors will make the feature manifold of the motion time full frequency domain significant aggregation representation vector obtained by dynamic aggregation of feature distribution field have expression sparsity and distribution imbalance in the high-dimensional feature space, thereby affecting the accuracy of the cable vibration frequency estimation value obtained by the cable vibration frequency estimation module based on the decoder.

[0054] Based on this, in a preferred embodiment, the motion time course full frequency domain significant aggregation representation vector is input into a cable vibration frequency estimation module based on a decoder to obtain a cable vibration frequency estimation value, including:

[0055] Performing L random sampling on the motion time course full frequency domain significant aggregation representation vector to obtain L motion time course random perturbation eigenvalues;

[0056] Arrange the L motion time-history random disturbance eigenvalues ​​in order from large to small to obtain a motion time-history random disturbance ordered vector;

[0057] Calculating the inner product of the significant aggregation representation vector of the motion time course in the whole frequency domain to obtain the energy modulation characteristic value of the motion time course;

[0058] Based on the motion time energy modulation characteristic value, the motion time random perturbation ordered vector is semantically scaled to obtain a semantically enhanced motion time random perturbation ordered vector;

[0059] Calculating the product between the random perturbation ordered vector of the semantically enhanced motion time course and the transposed vector of the significant aggregation representation vector of the motion time course in the whole frequency domain to obtain the orthogonal modulation matrix of the best example prompt of the motion time course;

[0060] Calculating the matrix product between the best sample prompt orthogonal modulation matrix of the motion time course and the significant aggregation representation vector of the full frequency domain of the motion time course to obtain an optimized significant aggregation representation vector of the full frequency domain of the motion time course;

[0061] The optimized motion time-course full-frequency domain significant aggregation representation vector is input into the decoder-based cable vibration frequency estimation module to obtain the cable vibration frequency estimation value.

[0062] The above optimization process is expressed as follows:

[0063] S i =∈ i , i = 1, 2, ..., L

[0064]

[0065] u=average(V)

[0066]

[0067] V order =Order{S i}

[0068]

[0069]

[0070]

[0071] Where V is the significant aggregation representation vector of the full frequency domain of the motion time course, represents the normal distribution, ∈ i represents a random variable that follows a normal distribution, L represents the number of random sampling, u and σ 2 They represent the feature mean and feature variance of the significant aggregation representation vector of the motion time course in the full frequency domain, S i represents the L random perturbation eigenvalues ​​of the motion time course obtained by L random sampling of the significant aggregation representation vector of the motion time course in the full frequency domain, V order is the ordered vector of random disturbances in the motion history, represents the calculation of the inner product of the eigenvector, e is the eigenvalue of the motion time course energy modulation, M t is the orthogonal modulation matrix of the best example of the motion time course, ⊙ represents the point multiplication by position, The representation vectors are multiplied, and V′ is the significant aggregation representation vector of the full frequency domain of the motion time course.

[0072] Accordingly, by performing L random sampling on the motion time-course full frequency domain significant aggregation representation vector to obtain a set of new feature values ​​based on the original features but containing slight changes, randomness is introduced to explore different aspects of the feature space, and by simulating the noise or variation in the actual data, the model is better generalized to unseen data. Then, these motion time-course random perturbation feature values ​​are ordered from large to small to form a motion time-course random perturbation ordered vector to identify which perturbation directions may be the most important for the current problem. Such an ordering not only helps the algorithm focus on the feature differences that can best distinguish different categories, but also helps to reveal hidden data structures or patterns. Subsequently, the inner product of the motion time-domain significant aggregation representation vector is calculated to obtain the motion time-based energy modulation eigenvalue, which reflects the overall strength or "energy" of the original feature vector and provides a benchmark for measuring the importance of the feature. Using this motion time-based energy modulation eigenvalue as a reference point, the previously formed motion time-based random perturbation ordered vector is semantically scaled to construct a semantically enhanced motion time-based random perturbation ordered vector. In this way, those disturbances that appear to be more important in the original feature space are amplified, thereby enhancing the model's sensitivity to changes in key features, while also retaining relatively small but potentially useful disturbance information.

[0073] Then, the product of the ordered vector of random disturbance of semantically enhanced motion time course and the transpose of the vector of significant aggregation representation of motion time course in the whole frequency domain is calculated to obtain the orthogonal modulation matrix of the best example prompt of motion time course, which aims to find the interaction mode between the features under two different perspectives. The formed matrix can be regarded as a conversion tool for capturing and representing the complex linear relationship between features. Finally, the best example prompt orthogonal modulation matrix of motion time course constructed above is applied to the original vector of significant aggregation representation of motion time course in the whole frequency domain to complete the final feature structure modulation, so that the features are more in line with the requirements of the objective function, reduce redundant information, and maximize the retention of useful information, so as to improve learning efficiency and prediction performance.

[0074] In summary, the feature manifold optimization process of the motion time course full frequency domain significant aggregation representation vector is explained, which extracts the best prompt sample based on the fine-grained target retrieval of random perturbations, and then guides the adversarial robustness learning based on the best prompt sample to complete the structure of the high-dimensional manifold of the feature vector in the feature space to improve the detail and structure of the feature expression. In this way, the accuracy of the cable vibration frequency estimation value obtained by the cable vibration frequency estimation module based on the decoder is improved.

[0075] In summary, the computer vision-based automated measurement method according to the embodiment of the present application is explained, which uses computer vision technology to track the vibration video of the bridge cable containing the target point to obtain the motion time history data of the cable, and then, by performing frequency domain analysis based on variational mode decomposition on the motion time history data of the cable, the waveform semantic features of each frequency domain intrinsic mode component of the cable motion time history data are captured, and then, based on the characteristic energy distribution of each frequency domain intrinsic mode component, it is significantly modulated and dynamically aggregated to capture the full frequency domain waveform oscillation mode of the cable motion time history data, so as to realize the intelligent estimation of the cable vibration frequency and the calculation of the cable force. In this way, the limitations of the traditional contact measurement method can be effectively avoided, while reducing the dependence on the physical property information of the cable, and improving the accuracy and efficiency of the cable force measurement.

[0076] Furthermore, an automated measurement system based on computer vision is also provided.

[0077] Figure 4 FIG. 1 is a block diagram of an automated measurement system based on computer vision according to an embodiment of the present application. Figure 4 As shown, according to the embodiment of the present application, the computer vision-based automated measurement system 300 includes: a vibration video acquisition module 310, which is used to acquire a vibration video of a bridge cable containing a target point collected by a camera; a target tracking module 320, which is used to track the target point in the vibration video based on a preset tracking algorithm to obtain the motion time-history data of the cable; a frequency domain analysis module 330, which is used to perform frequency domain analysis based on modal decomposition on the motion time-history data of the cable to obtain a set of semantic feature vectors of the waveforms of the frequency domain intrinsic modal components of the motion time; a full-frequency domain feature significant aggregation module 340, which is used to perform full-frequency domain feature significant aggregation on the set of semantic feature vectors of the waveforms of the frequency domain intrinsic modal components of the motion time to obtain a full-frequency domain significant aggregation representation vector of the motion time; and a cable force value calculation module 350, which is used to calculate the cable force value of the cable based on the full-frequency domain significant aggregation representation vector of the motion time.

[0078] As described above, the computer vision-based automated measurement system 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an automated measurement algorithm based on computer vision. In a possible implementation, the computer vision-based automated measurement system 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the computer vision-based automated measurement system 300 can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the computer vision-based automated measurement system 300 can also be one of the many hardware modules of the wireless terminal.

[0079] Alternatively, in another example, the computer vision-based automated measurement system 300 and the wireless terminal may also be separate devices, and the computer vision-based automated measurement system 300 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0080] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An automated measurement method based on computer vision, characterized in that: include: Obtain a vibration video of the bridge cable including the target point collected by the camera; Based on a preset tracking algorithm, target tracking is performed on the target point in the vibration video to obtain motion time history data of the cable; Performing frequency domain analysis based on modal decomposition on the motion time history data of the cable to obtain a set of semantic feature vectors of waveforms of frequency domain intrinsic modal components of the motion time history; Performing full-frequency-domain feature significant aggregation on the set of semantic feature vectors of the frequency-domain intrinsic modal component waveform of the motion time course to obtain a full-frequency-domain significant aggregation representation vector of the motion time course; Calculating the cable force value of the cable based on the significant aggregation representation vector of the motion time course in the full frequency domain; Among them, the set of semantic feature vectors of the motion time frequency domain intrinsic modal component waveform is subjected to full frequency domain feature significant aggregation, including: determining the initial center vector of the motion time frequency domain feature clustering based on the characteristic energy distribution of the set of semantic feature vectors of the motion time frequency domain intrinsic modal component waveform; based on the spatial span of each motion time frequency domain intrinsic modal component waveform semantic feature vector in the set of semantic feature vectors of the motion time frequency domain intrinsic modal component waveform relative to the initial center vector of the motion time frequency domain feature clustering, performing explicit dynamic aggregation on the set of semantic feature vectors of the motion time frequency domain intrinsic modal component waveform to obtain the full frequency domain significant aggregation representation vector of the motion time.

2. The computer vision-based automated measurement method according to claim 1, characterized in that: The motion time history data of the cable is subjected to frequency domain analysis based on modal decomposition to obtain a set of waveform semantic feature vectors of the motion time history frequency domain intrinsic modal component, including: Performing variational modal decomposition on the motion time history data of the cable to obtain a set of frequency domain eigenmodal components of the motion time history; Each motion time-domain frequency-domain intrinsic mode component in the set of motion time-domain frequency-domain intrinsic mode components is input into an intrinsic mode component waveform semantic feature extractor based on a dilated convolutional layer to obtain a set of motion time-domain frequency-domain intrinsic mode component waveform semantic feature vectors.

3. The computer vision-based automated measurement method according to claim 2, characterized in that: Based on the characteristic energy distribution of the set of semantic feature vectors of the intrinsic modal component waveform of the motion time course frequency domain, determining the initial center vector of the motion time course frequency domain feature clustering includes: Calculating the static energy factor of each motion time-history frequency-domain intrinsic mode component waveform semantic feature vector in the set of motion time-history frequency-domain intrinsic mode component waveform semantic feature vectors to obtain a set of motion time-history frequency-domain intrinsic mode component static energy factors; The waveform semantic feature vector of the motion time-domain intrinsic mode component corresponding to the maximum value in the set of static energy factors of the motion time-domain intrinsic mode component is selected as the initial center vector of the motion time-domain feature clustering.

4. The computer vision-based automated measurement method according to claim 3, characterized in that: Calculating the static energy factor of each motion time-course frequency domain intrinsic mode component waveform semantic feature vector in the set of motion time-course frequency domain intrinsic mode component waveform semantic feature vectors to obtain a set of motion time-course frequency domain intrinsic mode component static energy factors, including: The kurtosis of the semantic feature vector of the waveform of the motion time-course frequency-domain intrinsic mode component is calculated, and the kurtosis is input into a sigmoid activation function to obtain the static energy factor of the motion time-course frequency-domain intrinsic mode component.

5. The computer vision-based automated measurement method according to claim 4, characterized in that: Based on the spatial span of each motion time frequency domain intrinsic mode component waveform semantic feature vector in the set of motion time frequency domain intrinsic mode component waveform semantic feature vectors relative to the motion time frequency domain feature clustering initial center vector, the set of motion time frequency domain intrinsic mode component waveform semantic feature vectors is dynamically aggregated to obtain the motion time full frequency domain significant aggregation representation vector, including: Based on the spatial span between each motion time-domain frequency domain intrinsic modal component waveform semantic feature vector in the set of motion time-domain frequency domain intrinsic modal component waveform semantic feature vectors and the initial center vector of the motion time-domain frequency domain feature clustering, and the set of motion time-domain frequency domain intrinsic modal component static energy factors, the dynamic aggregation energy factor of each motion time-domain frequency domain intrinsic modal component waveform semantic feature vector in the set of motion time-domain frequency domain intrinsic modal component waveform semantic feature vectors is calculated to obtain a set of motion time-domain frequency domain intrinsic modal component dynamic aggregation energy factors; Based on the set of dynamic aggregation energy factors of the frequency domain intrinsic modal components of the motion time course, the set of waveform semantic feature vectors of the frequency domain intrinsic modal components of the motion time course is significantly modulated and aggregated to obtain the significant aggregation representation vector of the full frequency domain of the motion time course.

6. The computer vision-based automated measurement method according to claim 5, characterized in that: Calculating the dynamic aggregation energy factor of each motion time-course frequency domain intrinsic mode component waveform semantic feature vector in the set of motion time-course frequency domain intrinsic mode component waveform semantic feature vectors to obtain a set of motion time-course frequency domain intrinsic mode component dynamic aggregation energy factors, including: Taking the square value of the number of eigenvectors between the semantic feature vector of the waveform of the motion time-domain frequency domain intrinsic modal component and the initial center vector of the motion time-domain frequency domain feature clustering as the spatial span coefficient, the weighted ratio between the product of the static energy factor of the waveform semantic feature vector of the motion time-domain frequency domain intrinsic modal component and the static energy factor of the initial center vector of the motion time-domain frequency domain feature clustering and the spatial span coefficient is calculated to obtain the dynamic aggregation energy factor of the motion time-domain frequency domain intrinsic modal component.

7. The computer vision-based automated measurement method according to claim 6, characterized in that: Based on the set of dynamic aggregation energy factors of the frequency domain intrinsic modal components of the motion time course, a set of waveform semantic feature vectors of the frequency domain intrinsic modal components of the motion time course is subjected to significant modulation aggregation to obtain a significant aggregation representation vector of the full frequency domain of the motion time course, including: Inputting the set of dynamic aggregation energy factors of the motion time-course frequency-domain intrinsic mode components into a gated mask unit to obtain a set of dynamic aggregation weight factors of the motion time-course frequency-domain intrinsic mode components; Based on the set of dynamic aggregation weight factors of the motion time-course frequency domain intrinsic mode components, the weighted sum of the set of waveform semantic feature vectors of the motion time-course frequency domain intrinsic mode components is calculated to obtain the significant aggregation representation vector of the motion time-course full frequency domain.

8. The computer vision-based automated measurement method according to claim 7, characterized in that: Calculating the cable force value of the cable based on the full-frequency domain significant aggregation representation vector of the motion time course includes: Inputting the motion time course full frequency domain significant aggregation representation vector into a cable vibration frequency estimation module based on a decoder to obtain a cable vibration frequency estimation value; The cable force value of the cable is calculated based on the estimated value of the cable vibration frequency.

9. An automated measurement system based on computer vision, characterized in that: include: A vibration video acquisition module is used to acquire a vibration video of a bridge cable including a target point collected by a camera; A target tracking module, used for tracking the target point in the vibration video based on a preset tracking algorithm to obtain the motion time history data of the cable; A frequency domain analysis module, used for performing frequency domain analysis on the motion time history data of the cable based on modal decomposition to obtain a set of semantic feature vectors of waveforms of intrinsic modal components in the motion time history frequency domain; A full-frequency domain feature significant aggregation module is used to perform full-frequency domain feature significant aggregation on the set of semantic feature vectors of the intrinsic modal component waveform of the motion time course frequency domain to obtain a full-frequency domain significant aggregation representation vector of the motion time course; The cable force value calculation module is used to calculate the cable force value of the cable based on the significant aggregation representation vector of the motion time course in the full frequency domain.

Images