A machine vision structure displacement measurement method and apparatus

Abstract

A kind of machine vision structure displacement measurement method and device, method includes: converting structure motion video into image sequence;With complex value steerable pyramid model, the image sequence is carried out spatial decomposition, obtains initial phase difference information;According to the initial phase difference information, phase recovery processing is carried out, and absolute phase difference information is obtained;According to the absolute phase difference information, phase unwrapping processing is carried out, and final phase difference information is obtained;According to the final phase difference information, structure displacement is determined.

Description

Technical Field

[0001] This article relates to the field of video image processing technology, and in particular to a machine vision structural displacement measurement method and device. Background Technology

[0002] Non-contact motion recognition methods based on digital camera optical measurements can solve problems such as the added mass of traditional sensors and the asynchrony of wireless sensing. However, traditional machine vision measurement methods require the application of a large number of high-contrast marks or patterns on the surface of the structure being measured, which is time-consuming; this method is also somewhat inadequate when the area to be measured is large or difficult to access. In addition, for traditional optical measurement systems, exciting the structure under test with only a small amount of energy may produce subtle vibration responses that are difficult to capture.

[0003] Some motion recognition methods based on phase motion amplification techniques can address the aforementioned problems. However, these methods rely on the assumption that small motions can be encoded as a phase shift of a pixel's color, and the spatial displacement encoded in the phase information is very limited, leading to phase step phenomena. Furthermore, phase estimation is an ill-posed problem where observation noise is amplified and propagates to both phase and amplitude estimations, both of which can cause excessive blurring. Therefore, a structural motion recognition method that is highly noise-resistant and can resolve phase step phenomena is needed. Summary of the Invention

[0004] This application provides a machine vision structural displacement measurement method and device. Taking into account the sparsity of phase propagation and the step problem, a multi-frequency phase analysis motion recognition framework is proposed. A phase expansion method based on the brightness transfer equation is used to further correct the estimation error of the complex-valued manipulable pyramid, thereby solving the step problem more deeply.

[0005] This application provides a machine vision method for measuring structural displacement, including:

[0006] Convert structural motion video into an image sequence;

[0007] The image sequence is spatially decomposed using a complex-valued manipulable pyramid model to obtain initial phase difference information;

[0008] Phase recovery processing is performed based on the initial phase difference information to obtain the absolute phase difference information;

[0009] The phase expansion process is performed based on the absolute phase difference information to obtain the final phase difference information;

[0010] The structural displacement is determined based on the final phase difference information.

[0011] In one exemplary embodiment, after converting the structural motion video into an image sequence, the method further includes:

[0012] The image sequence is converted from the RGB color space to the YIQ color space using the following formula:

[0013]

[0014] Where Y is the luminance signal, I is the color difference signal that the human eye is sensitive to, Q is the color difference signal that the human eye is not sensitive to, R is the red channel image signal, G is the green channel image signal, and B is the blue channel image signal.

[0015] In one exemplary embodiment, the step of using a complex-valued manipulable pyramid model to perform spatial domain decomposition on the image sequence to obtain initial phase difference information includes:

[0016] The video converted to the YIQ color space was spatially decomposed using a complex-valued manipulable pyramid model to obtain multiple frames of images at each frequency.

[0017] For each frequency, the phase information of each frame is subtracted from the phase information of the first frame to obtain the initial phase difference information ψ corresponding to each frame.

[0018] The complex-valued manipulable pyramid model is as follows:

[0019]

[0020] In the above model, A is the amplitude, ψ is the initial phase difference, and θ is the frequency. It's a convolution calculation. It represents the real part of the convolution kernel with frequency θ. θ represents the imaginary part of the convolution kernel with frequency θ, and I is the image.

[0021] In one exemplary embodiment, the step of performing phase recovery processing based on the initial phase difference information to obtain absolute phase difference information includes:

[0022] Based on the initial phase difference information, the absolute phase difference information is obtained using the quadratic residual penalty conditional likelihood function;

[0023] Wherein, the absolute phase difference is:

[0024]

[0025]

[0026] Where, k and k λ It is an integer. It is the absolute phase difference, u o,λ It is a multi-frequency target complex exponent. It is a wrapping phase, μλ It is a dimensionless relative frequency, and λ is the phase observation value. It refers to the absolute magnitude of the target.

[0027] In one exemplary embodiment, the step of performing phase unrolling processing based on the absolute phase difference information to obtain the final phase difference information includes:

[0028] Step 1. Determine the initial unfolding phase difference based on the absolute phase difference information;

[0029] Step 2. Calculate the initial integer estimate based on the initial unfolded phase difference;

[0030] Step 3. Determine the second expanded phase difference based on the initial integer estimate and the absolute phase difference information;

[0031] Step 4. Calculate the residual between the second expanded phase difference and the initial expanded phase difference, and calculate the corrected expanded phase difference based on the residual;

[0032] Step 5. Update the initial unfolded phase difference according to the corrected unfolded phase difference, and calculate the second integer estimate;

[0033] Step 6. If abs(g1-g2)>0, such that g1=g2, then repeat steps 1-5; where g1 is the initial integer estimate and g2 is the second integer estimate.

[0034] Otherwise, the second expanded phase difference is taken as the final phase difference.

[0035] In one exemplary embodiment, the step of calculating the corrected expanded phase difference based on the residual includes:

[0036] The corrected expanded phase difference is calculated based on this residual using the phase expansion model;

[0037] The phase expansion model is as follows:

[0038]

[0039] Where DCT is the Discrete Cosine Transform, IDCT is the Inverse Discrete Cosine Transform, ξ is the residual, and φ is the inverse discrete cosine transform. c It is the corrected unfolded phase difference, where X and Y are the lengths of the image in the x and y directions, respectively.

[0040] In one exemplary embodiment, determining the structural displacement based on the final phase difference information includes:

[0041] The structural motion response is determined based on the final phase difference information;

[0042] Based on the motion response, the ratio of the length of the measured structure to the number of pixels the measured structure spans in the image is converted into structural displacement.

[0043] In one exemplary embodiment, the structural motion response is:

[0044]

[0045] In the above formula, u is the structural motion response in the x-direction, v is the structural motion response in the y-direction, φ0 is the structural phase in the x-direction, and φ π / 2 It is the structural phase in the y-direction.

[0046] This application also provides a machine vision structural displacement measurement device, the device comprising: a memory and a processor; characterized in that the memory is used to store a program for performing machine vision structural displacement measurement, and the processor is used to read and execute the program for performing machine vision structural displacement measurement, and execute the method described in any one of the above embodiments.

[0047] This application also provides a computer-readable storage medium storing computer-executable instructions, characterized in that, when the computer-executable instructions are executed by a processor, they perform the method described in any one of the above embodiments.

[0048] Compared with related technologies, this application provides a machine vision method and apparatus for measuring structural displacement. The method includes: converting a structural motion video into an image sequence; performing spatial decomposition of the image sequence using a complex-valued manipulable pyramid model to obtain initial phase difference information; performing phase recovery processing based on the initial phase difference information to obtain absolute phase difference information; performing phase unrolling processing based on the absolute phase difference information to obtain final phase difference information; and determining the structural displacement based on the final phase difference information. Through the technical solution of this invention, considering the sparsity of phase propagation and the step problem, a multi-frequency phase analysis motion recognition framework is proposed; a phase unrolling method based on the brightness transfer equation is used to further correct the estimation error of the complex-valued manipulable pyramid, thereby solving the step problem more deeply.

[0049] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0050] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0051] Figure 1 This is a flowchart of the machine vision structural displacement measurement method according to an embodiment of this application;

[0052] Figure 2 This is a schematic diagram of a machine vision structural displacement measurement device according to an embodiment of this application;

[0053] Figure 3 These are some exemplary flowcharts of machine vision structural displacement measurement methods;

[0054] Figure 4 This is a frame from a video of a pipe being vibrated by a hammer, selected from some examples, along with a schematic diagram of the measurement point;

[0055] Figure 5 This is a frame from a video of a cantilever bridge under environmental load selected in some examples, along with a schematic diagram of two measurement points.

[0056] Figure 6 These are schematic diagrams of the displacement response corresponding to the first four modes of measurement points in some exemplary pipeline videos;

[0057] Figure 7 These are schematic diagrams of the first four mode shapes in some exemplary pipeline videos;

[0058] Figure 8 These are some examples of schematic diagrams showing the displacement response of two measurement points in the first two modes of a cantilever bridge video. Detailed Implementation

[0059] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0060] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0061] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0062] This disclosure provides a machine vision method for measuring structural displacement, such as... Figure 1 As shown, the method includes steps S100-S140, as detailed below:

[0063] S100. Convert the structural motion video into an image sequence;

[0064] S110. The image sequence is spatially decomposed using a complex-valued manipulable pyramid model to obtain initial phase difference information;

[0065] S120. Perform phase recovery processing based on the initial phase difference information to obtain absolute phase difference information;

[0066] S130. Perform phase expansion processing based on the absolute phase difference information to obtain the final phase difference information;

[0067] S140. Determine the structural displacement based on the final phase difference information.

[0068] In one exemplary embodiment, after converting the structural motion video into an image sequence, the method further includes: converting the image sequence from the RGB color space to the YIQ color space using the following formula:

[0069]

[0070] Where Y is the luminance signal, I is the color difference signal that the human eye is sensitive to, Q is the color difference signal that the human eye is not sensitive to, R is the red channel image signal, G is the green channel image signal, and B is the blue channel image signal.

[0071] In one exemplary embodiment, a complex-valued manipulable pyramid model is used to perform spatial decomposition on the image sequence to obtain initial phase difference information, including:

[0072] The video converted to the YIQ color space was spatially decomposed using a complex-valued manipulable pyramid model to obtain multiple frames of images at each frequency.

[0073] For each frequency, the phase information of each frame is subtracted from the phase information of the first frame to obtain the initial phase difference information ψ corresponding to each frame.

[0074] The complex-valued manipulable pyramid model is as follows:

[0075]

[0076] In the above model, A is the amplitude, ψ is the initial phase difference, θ is the frequency, G is the real part of the convolution kernel, and H is the imaginary part of the convolution kernel. It's a convolution calculation.

[0077] Phase difference information is obtained by subtracting the phase information of the current frame from the phase information of the first frame:

[0078] Δφ θ (t)=φ θ (t)-φ θ (1),

[0079] Where t is time, φ θ (t) represents the ideal phase information of the t-th frame image, φ θ (1) is the ideal phase information of the first frame image, Δφ θ (t) represents the ideal phase difference information under ideal conditions, i.e., it does not contain noise or phase step phenomena. In this embodiment, ψ is the initial phase difference. The absolute phase difference needs to be obtained through phase retrieval, while φ is the phase difference information obtained after the final calculation (phase retrieval and phase expansion). In other words, it is calculated from ψ through phase retrieval. Then, through phase expansion from... φ is calculated.

[0080] In one exemplary embodiment, phase recovery processing is performed based on the initial phase difference information to obtain absolute phase difference information, including:

[0081] Based on the initial phase difference information, the absolute phase difference information is obtained using the quadratic residual penalty conditional likelihood function; wherein, the absolute phase difference calculated by phase recovery... for:

[0082]

[0083]

[0084] Where, k and k λ It is an integer.

[0085] The process of obtaining absolute phase difference information by performing phase recovery processing based on the initial phase difference information is as follows:

[0086] Phase recovery is performed based on the initial phase difference information; this phase recovery can be achieved by solving the quadratic residual penalty conditional likelihood function.

[0087]

[0088] Among them, u o,λ It is a multi-frequency complex exponent, b o,λ It is the absolute amplitude. It is the absolute phase difference that needs to be obtained through phase recovery, u λ (>0) is a dimensionless relative frequency, Λ=[λ0,λ1,…,λ n-1 [This refers to the time period of a series of multi-frequency observation data.] It is the image forming operator, s is the observation number, l(z,|u| 2 ) is u o,λ The negative log-likelihood of candidate solutions. σ 2 It is the noise variance. It is the Frobenius norm, γ1, γ2>0 are weighting parameters, and the phase difference information is unknown. Due to phase shift δ λ The model is based on disturbance parameters.

[0089] Solving the formula for the likelihood function of the quadratic residual penalty condition Through the following steps:

[0090] Step 1: via u s,λ conduct Minimize;

[0091] This involves the first two terms of the quadratic residual penalty conditional likelihood function, which can be directly solved for λ and s through sparse phase amplitude inversion; after solving for λ and s through inversion, ... Minimize the computation.

[0092] Step 2: via u0,λ conduct Minimize;

[0093] This involves the first two terms of the quadratic residual penalty conditional likelihood function, which can be rewritten using a quadratic equation problem as follows:

[0094]

[0095] if It is orthogonal, that is E is the identity matrix, u 0,λ The optimal solution can be expressed as:

[0096]

[0097] In the above formula, It is an image forming operator. yes The conjugate transpose of u s,λ u is the value in the s-th calculation process. o,λ γ1 and γ2 are both weight parameters.

[0098] Step 3: Through δ λ , and b o.λ conduct Minimize

[0099] This involves the multi-frequency target complex exponent u. 0,λ The nonlinear fitting can be divided into two parts: phase synchronization and absolute phase recovery.

[0100] In phase synchronization, the phase offset δ λ The estimation requires introducing a reference channel λ′∈Λ for the λ channel, which can make δ λ The calculation is converted to δ on the reference channel. λ′ The calculation.

[0101] Therefore, the solution for phase synchronization can be expressed as the following formula:

[0102]

[0103] Here, ψ is the value that needs to be solved through phase retrieval. The initial phase information, It is δ λ′ Candidate solutions.

[0104] In absolute phase recovery, it is necessary to assume b beforehand. o,λ =|u 0,λ | and Then the absolute phase can be calculated as:

[0105]

[0106] By further introducing the phase period, the absolute phase difference information can be finally obtained;

[0107] Wherein, the absolute phase difference is:

[0108]

[0109]

[0110] Where, k and k λ It is an integer.

[0111] In one exemplary embodiment, phase expansion processing is performed based on the absolute phase difference information to obtain the final phase difference information. The implementation process is as follows:

[0112] Step 1. Based on absolute phase difference information Determine the initial expansion phase difference φ1;

[0113] Step 2. Calculate the initial integer estimate g1 based on the initial unfolded phase difference φ1;

[0114] In this step, using Calculate the initial integer estimate, where round indicates rounding.

[0115] Step 3. Based on the initial integer estimate g1 and the absolute phase difference information Determine the second expansion phase difference φ2;

[0116] In this step, the second expanded phase difference φ2 is calculated using the following formula:

[0117]

[0118] Step 4. Calculate the residual ξ between the second expanded phase difference φ2 and the initial expanded phase difference φ1, and calculate the corrected expanded phase difference φ based on the residual ξ. c ;

[0119] Step 5. Based on the corrected expansion phase difference φ c Update the initial unfolded phase difference φ1 and calculate the second integer estimate g2;

[0120] In this step, based on the corrected unfolded phase difference φ c The initial expansion phase difference φ1 is updated as follows:

[0121] Update the initial expansion phase difference φ1' = φ1 + φ c ;

[0122] The second integer estimate g2 is calculated as follows:

[0123]

[0124] Update the second expanded phase difference φ2 based on the calculated second integer estimate g2:

[0125] Update the second phase difference

[0126] Step 6. If abs(g1-g2)>0, such that g1=g2, then repeat steps 1-5; where g1 is the initial integer estimate and g2 is the second integer estimate.

[0127] Otherwise, the second expanded phase difference is taken as the final phase difference.

[0128] In one exemplary embodiment, the corrected expanded phase difference φ is obtained based on the residual ξ. c ,include:

[0129] Based on this residual ξ, the corrected expanded phase difference φ is calculated using the phase expansion model. c ;

[0130] The phase expansion model is as follows:

[0131]

[0132] DCT stands for Discrete Cosine Transform, and IDCT stands for Inverse Discrete Cosine Transform.

[0133] In one exemplary embodiment, determining the structural displacement based on the final phase difference information includes: determining the structural motion response based on the final phase difference information; and converting the motion response and the ratio of the length of the structure to be measured to the number of pixels the structure to be measured traverses in the image into structural displacement.

[0134] In one exemplary embodiment, the structural motion response is:

[0135]

[0136] In the above formula, u is the structural motion response in the x-direction, v is the structural motion response in the y-direction, φ0 is the structural phase in the x-direction, and φ π / 2 It is the structural phase in the y-direction.

[0137] This disclosure also provides a machine vision structural displacement measurement device, such as... Figure 2 As shown, the device includes a memory 210 and a processor 220; the memory 210 is used to store a program for performing machine vision structural displacement measurement, and the processor 220 is used to read and execute the program for performing machine vision structural displacement measurement, and execute any of the machine vision structural displacement measurement methods in the above embodiments.

[0138] This disclosure also provides a computer-readable storage medium storing computer-executable instructions, characterized in that the computer-executable instructions, when executed by a processor, perform any machine vision structural displacement measurement method.

[0139] Example 1

[0140] A structural motion recognition method based on an adaptive phase analysis framework, such as Figure 3 As shown, the specific implementation process is as follows:

[0141] 1. Perform color space conversion:

[0142] After the video is read, each image in the sequence is converted from the RGB color space to the YIQ color space using the following formula:

[0143]

[0144] Where Y is the luminance signal, I is the color difference signal that the human eye is sensitive to, Q is the color difference signal that the human eye is not sensitive to, R is the red channel image signal, G is the green channel image signal, and B is the blue channel image signal.

[0145] 2. Using a complex-valued manipulable pyramid model, the image sequence is spatially decomposed. The phase information of each frame is subtracted from that of the first frame to obtain the phase difference.

[0146] The complex-valued manipulable pyramid model is as follows:

[0147]

[0148] Where A is the amplitude, φ is the phase, θ is the frequency, G is the real part of the convolution kernel, and H is the imaginary part of the convolution kernel. It's convolution.

[0149] The phase difference operation formula is as follows: Δφ θ (t) represents the phase difference between each frame and the first frame.

[0150] Δφ θ (t):=φ θ (t)-φ θ (1),

[0151] Where t is time.

[0152] 3. Perform phase recovery to resolve phase ill-conditioning issues.

[0153] To perform multi-frequency phase recovery on the phase difference information obtained in step 2, a multi-frequency complex-valued model must first be established:

[0154]

[0155] Among them, u o,λ It is a multi-frequency target complex exponent, b o,λ and These are the absolute magnitude and phase of the object, u λ (>0) is a dimensionless relative frequency, and Λ is the absolute phase, where Λ = [λ0,λ1,…,λ]. n-1 [ ] is the time period of a series of multi-frequency observation data.

[0156] u λ It can be calculated in the second step; but b o,λ and Need to get from the noise signal z o,λ and noiseless signal y o,λ Recovery in the middle; the formula is as follows:

[0157]

[0158] in, It is a random noise observation operator. It is the image forming operator, where s is the observation number and S is the total number of observations.

[0159] Due to the nonlinearity of the likelihood function relative to the observation model, from z s,λ Direct recovery of u o,λ It's relatively difficult. For random noise models with independent samples, a maximum likelihood estimate can be established, as shown in the following formula:

[0160]

[0161] Where, l(z,|u| 2 ) is u o,λ The negative log-likelihood of candidate solutions. σ 2 It is the noise variance.

[0162] The above solution requires the use of the following quadratic residual penalty conditional likelihood function:

[0163]

[0164] in, The Frobenius norm is used, γ1, γ2 > 0 are weighting parameters, and these unknown phases are determined by the phase shift δ. λ The model is based on disturbance parameters.

[0165] Step 3 involves the following sub-steps:

[0166] Step 3.1: via u s,λ conduct Minimize

[0167] This involves the first two terms of the quadratic residual penalty conditional likelihood function, which can be solved by solving λ and s respectively, or directly by sparse phase amplitude inversion.

[0168] Step 3.2: via u 0,λ conduct Minimize

[0169] This involves the first two terms of the quadratic residual penalty conditional likelihood function, which can be rewritten using a quadratic equation problem as follows:

[0170]

[0171] if It is orthogonal, that is E is the identity matrix, u 0,λ The optimal solution can be expressed as:

[0172]

[0173] Step 3.3: Through δ λ , and b o.λ conduct Minimize

[0174] This involves the multi-frequency target complex exponent u. 0,λ Nonlinear fitting can be divided into phase synchronization and absolute phase recovery.

[0175] In phase synchronization, the phase offset δ λ The estimation requires introducing a reference channel λ′∈Λ for the λ channel, which can make δ λ The calculation is converted to δ on the reference channel. λ′ The calculation of phase synchronization can be expressed as follows:

[0176]

[0177] Where ψ represents the initial phase difference information. It is δ λ′ Candidate solutions.

[0178] In absolute phase recovery, it is necessary to assume b beforehand. o,λ =|u 0,λ | and Then the absolute phase can be calculated as:

[0179]

[0180] By further introducing the phase period, the final solution expression is:

[0181]

[0182]

[0183] Where k is an integer.

[0184] 4. Perform phase expansion

[0185] The phase difference information obtained in step 3 is subjected to phase expansion. First, a phase expansion model is established:

[0186] Where φ is the expanded phase and g is the integer to be solved.

[0187] Solving the phase expansion problem requires introducing the Discrete Cosine Transform (DCT) and the Inverse Discrete Cosine Transform (IDCT), as shown in the following equation:

[0188] Step 4 includes the following sub-steps:

[0189] Step 4.1: Calculate the result in Step 3. Substituting into the DCT, the initial unfolded phase difference φ1 is calculated, and then... Obtain an initial integer estimate, where round indicates rounding.

[0190] Step 4.2: Calculate the expanded phase difference Then calculate the residuals ξ of φ2 and φ1, and substitute the residuals into the formula to obtain the corrected expanded phase difference φ. c .

[0191] Step 4.3: Update the expanded phase difference φ1 = φ1 + φ c Calculate integers Thus update

[0192] Step 4.4: If abs(g1-g2)>0, make g1=g2, then repeat the above sub-step; otherwise, output the final phase difference φ2.

[0193] 5. Motion signal output

[0194] The final phase difference φ calculated in step 4 can be converted into a structural motion response, which can be expressed as:

[0195]

[0196] Where u and v are the motion responses of the structure in the x and y directions, respectively, and φ0 and φ π / 2 It is the local phase of the structure in the x and y directions. Without considering image distortion, the motion response can be converted into structural displacement by the ratio of the structure's length to the number of pixels it spans in the image.

[0197] Machine vision-based structural displacement measurement methods can be evaluated from both motion and modal recognition perspectives. In this analysis, a pipe (e.g.,...) was selected as an example. Figure 4 (as shown) and cantilever bridges (such as Figure 5 The vibration videos shown are all sourced from open-source videos from MIT labs. To verify the stability of the algorithm, the method in this embodiment is compared with traditional phase-based methods. To increase the difficulty of the analysis, Gaussian noise with a variance of 0.014 was artificially added to all analyzed video signals beforehand.

[0198] In the pipeline test, the external force comes from the impact of a hammer, and the measurement point is located at... Figure 4 The white dot marks the location. Figure 6 (a)-6(d) represent the displacement time histories of measuring point 1 corresponding to the first four modes, with reference data derived from theoretical results provided by MIT. Although the displacements obtained using the phase-based method partially match the reference data, they are susceptible to ill-conditioned estimation and phase step problems, resulting in significant deviations when large displacements (peaks) occur. In contrast, the method proposed in this invention outperforms the phase-based method because the displacement response calculated by it essentially matches the reference data. The estimation results of the first four pipe mode shapes are as follows: Figure 7 As shown in (a)-7(b), it can be seen that the mode shapes calculated by the method proposed in this embodiment are quite close to the reference mode shapes. The phase-based method, due to a lack of statistical inference, results in mode shapes that differ significantly from the reference data.

[0199] In the cantilever bridge test, the external forces come from environmental loads such as waves and wind, and the measurement points are located at... Figure 4 The white dots are marked in the middle, with the white dot above being measurement point 1 and the white dot below being measurement point 2. Figure 8 (a)-8(b) and Figure 8 (c)-8(d) correspond to the displacement time histories at measurement points 1 and 2 for the first and second modes, respectively, with reference data obtained from accelerometers placed at these two points. Phase-based methods are highly sensitive to noise, easily producing ambiguous results and making it difficult to obtain accurate displacement time histories. In contrast, the method proposed in this embodiment can obtain displacement time histories that closely match the reference data.

[0200] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. A machine vision method for measuring structural displacement, characterized in that, include: Convert structural motion video into an image sequence; The transformation will be performed using a complex-valued manipulable pyramid model. The video in the color space is spatially decomposed to obtain multiple frames of images at each frequency. For each frequency, the phase information of each frame is subtracted from the phase information of the first frame to obtain the initial phase difference information corresponding to each frame. The complex-valued manipulable pyramid model is as follows: In the above model, It's the amplitude. It is the initial phase difference information. It's frequency. It's a convolution calculation. This indicates that the frequency is The real part of the convolution kernel, This indicates that the frequency is The imaginary part of the convolution kernel, It is an image; Based on the initial phase difference information, the absolute phase difference information is obtained using the quadratic residual penalty conditional likelihood function; Wherein, the quadratic residual penalty conditional likelihood function is: in, It is the absolute amplitude. It is the time period of a series of multi-frequency observation data. It is an image forming operator. It is the observation number sequence. yes The negative log-likelihood of candidate solutions. , It is the noise variance. These are weight parameters. It is a phase shift; The absolute phase difference is: in, It is an integer. It is the absolute phase difference. It is a multi-frequency target complex exponent. It is a wrap-around phase. It is a dimensionless relative frequency. These are phase observations. It is the absolute magnitude of the target. It is an integer; The phase expansion process is performed based on the absolute phase difference information to obtain the final phase difference information; The structural displacement is determined based on the final phase difference information.

2. The machine vision structural displacement measurement method according to claim 1, characterized in that, After converting the structural motion video into an image sequence, the method further includes: The image sequence from Color space conversion to Color space, using the following formula: in, It is a brightness signal. It is a color difference signal that the human eye is sensitive to. These are color difference signals that the human eye is not sensitive to. R is the red channel image signal, G is the green channel image signal, and B is the blue channel image signal.

3. The machine vision structural displacement measurement method according to claim 2, characterized in that, The step of performing phase unrolling processing based on the absolute phase difference information to obtain the final phase difference information includes: Step 1. Determine the initial unfolding phase difference based on the absolute phase difference information; Step 2. Calculate the initial integer estimate based on the initial unfolded phase difference; Step 3. Determine the second expanded phase difference based on the initial integer estimate and the absolute phase difference information; Step 4. Calculate the residual between the second expanded phase difference and the initial expanded phase difference, and calculate the corrected expanded phase difference based on the residual; Step 5. Update the initial unfolded phase difference according to the corrected unfolded phase difference, and calculate the second integer estimate; Step 6. If , making Then repeat steps 1-5; Otherwise, the second expanded phase difference is taken as the final phase difference; where, It is the initial integer estimate. It is the second integer estimate.

4. The machine vision structural displacement measurement method according to claim 3, characterized in that, The corrected expanded phase difference calculated based on the residual includes: The corrected expanded phase difference is calculated based on this residual using the phase expansion model; The phase expansion model is as follows: in, It is the discrete cosine transform. It is the inverse discrete cosine transform. It is a residual. It is the corrected expanded phase difference. and The images are in and Length in the direction.

5. The machine vision structural displacement measurement method according to claim 4, characterized in that, Determining the structural displacement based on the final phase difference information includes: The structural motion response is determined based on the final phase difference information; Based on the motion response, the ratio of the length of the measured structure to the number of pixels the measured structure spans in the image is converted into structural displacement.

6. The machine vision structural displacement measurement method according to claim 3, characterized in that, The structural motion response is: , In the above formula, Is Structural motion response in the direction, Is Structural motion response in the direction, Is Structural phase in direction, Is Structural phase in the direction.

7. A machine vision structural displacement measurement device, characterized in that, The apparatus comprises: a memory and a processor; characterized in that the memory is used to store a program for performing a machine vision structural displacement measurement method, and the processor is used to read and execute the program for performing a machine vision structural displacement measurement method, and to perform the method according to any one of claims 1-6.

8. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by the processor, the machine vision structural displacement measurement method as described in any one of claims 1-6 is performed.

Images