Machine vision intelligent displacement monitoring method and device based on digital image correlation and measurement point cross correlation fusion

Abstract

This invention relates to a machine vision-based intelligent displacement monitoring method and device based on the fusion of digital image correlation and measurement point cross-correlation, belonging to the field of displacement monitoring technology. This invention solves problems such as abnormal data interference, insufficient accuracy due to temperature changes, and poor stability in long-distance monitoring in traditional machine vision monitoring. The method involves high-frequency image acquisition, virtual benchmark measurement point screening and prediction, fusion of multiple algorithms to solve for actual displacement, data anomaly removal and serialization, and polynomial fitting to abstract and update benchmark points. The device executes the aforementioned methods and incorporates a built-in temperature sensor for error compensation. This invention effectively removes abnormal data, compensates for temperature interference, improves monitoring robustness, stability, and accuracy, and ensures reliable monitoring results under different environments.

Description

Technical Field

[0001] This invention belongs to the field of displacement monitoring technology, and relates to a machine vision intelligent displacement monitoring method and device based on the fusion of digital image correlation and measurement point cross-correlation. Background Technology

[0002] Long-distance bridges, as key nodes and core infrastructure of transportation networks, are crucial engineering projects ensuring the efficient passage of trunk highways, railways, and urban expressways. Their structural safety and service status directly affect public safety, traffic stability, and the normal operation of the national economy. During long-term operation, long-distance bridges continuously endure multiple loads, including vehicle reciprocating loads, wind loads, temperature stress, concrete shrinkage and creep, and uneven foundation settlement. Damage such as structural deformation, pier displacement, and main beam deflection gradually accumulates and develops, becoming key indicators that can induce bridge ultimate state failure and trigger major safety accidents. Therefore, all-weather, high-precision, real-time monitoring of key parameters such as structural deformation, pier displacement, and main beam alignment of long-distance bridges, and timely identification of structural anomalies and early damage, are core technical means to ensure safe bridge operation, extend service life, and prevent safety risks.

[0003] In recent years, my country's transportation infrastructure construction has gradually shifted from large-scale new construction to a "construction and maintenance" approach. A large number of long bridges have entered their middle and old service years. Coupled with the real pressures of continuously increasing traffic loads, rising proportion of heavy vehicles, and frequent extreme weather events, traditional maintenance methods such as manual inspections and periodic testing have prominent shortcomings, such as low efficiency, strong subjectivity, inability to capture dynamic responses in real time, and difficulty in covering hidden parts. They can no longer meet the realistic needs of long bridges for high safety levels, long service life, and intelligent maintenance. The industry's demand for automated, intelligent, and non-contact structural health monitoring technologies is becoming increasingly urgent.

[0004] Machine vision monitoring technology, with its advantages of being non-contact, flexible in deployment, moderate in cost, and able to intuitively acquire information on structural surface deformation and displacement, has gradually replaced traditional contact or manual measurement methods such as strain gauges, displacement gauges, and levels. It has been widely used in the deflection and displacement monitoring of small and medium-span bridges and conventional structures. For short-distance monitoring scenarios within 100 meters, technologies such as image acquisition, feature recognition, displacement calculation, and error correction are relatively mature and can meet the monitoring accuracy and stability requirements of general engineering scenarios.

[0005] However, existing machine vision monitoring technologies face significant application bottlenecks and technical limitations due to the typical characteristics of long-span bridges, such as large spans and long monitoring distances (usually exceeding 100m). Under long-distance imaging conditions, monitoring systems are susceptible to interference from complex outdoor environmental factors such as atmospheric temperature gradients, humidity changes, uneven lighting intensity, backlighting / low light environments, air disturbances, fog, and dust. This leads to problems such as decreased image and video clarity, blurred feature points, geometric distortion, reduced signal-to-noise ratio, and inter-frame drift, severely affecting the accuracy of structural feature extraction and displacement calculation. Furthermore, the increased pixel equivalent physical size caused by long-distance imaging, the difficulty in compensating for random errors introduced by environmental disturbances, and the insufficient robustness of traditional visual algorithms further result in poor monitoring data stability, excessive errors, and significant long-term drift, failing to meet the requirements for high-precision, all-weather, and long-term monitoring of structural deformation and pier displacement in long-span bridges.

[0006] In summary, while existing machine vision technology is mature in short-distance and small-to-medium-sized bridge monitoring, it is difficult to adapt to the monitoring needs of long-distance, highly interference-prone, and highly reliable long-distance bridges. Its application effect and engineering applicability in long-distance structural health monitoring of long-distance bridges have long been limited, becoming a key technical problem restricting the promotion and implementation of intelligent bridge monitoring technology. The industry urgently needs an intelligent monitoring method for structural deformation of long-distance bridges that can resist complex environmental interference, is suitable for long-distance imaging, and has both high precision and high stability. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a machine vision intelligent displacement monitoring method and device based on the fusion of digital image correlation and measurement point cross-correlation.

[0008] To achieve the above objectives, the present invention provides the following technical solution: On the one hand, a machine vision-based intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation is first provided. This method includes: S1. Use interrupt trigger mode to perform high-frequency image acquisition, control the acquisition time error within a preset range, and store the acquired image in the acquisition buffer. S2. Select the measurement points that meet the conditions as the original points of the virtual reference measurement points, calculate the cumulative discrete difference sequence of the absolute position change of the measurement points, and obtain the predicted final value through the least squares method. S3. The position of the measurement point in the image is solved by integrating the Jacobian matrix, Hessian matrix and inverse Gaussian algorithm. The confidence level is quantified by the standardized zero mean square difference function. After iterative optimization by IC-GN algorithm, the actual spatial displacement of the measurement point is obtained by coordinate transformation. S4. Determine the virtual reference measuring point based on the actual displacement results of the measuring point, and store its position data and corresponding time in the preset absolute position sequence; S5. After removing anomalies from the data in the absolute position sequence, the data is serialized by time. Iterative differential filtering and short-time Fourier algorithm are used for analysis to supplement missing measurement points and obtain standardized serialized data. S6. Solve for the polynomial coefficients by matrix determinant, fit the serialized measurement point data to obtain the abstract reference point position, and periodically update the polynomial coefficients and abstract reference point data, using them as the reference quantity for the displacement of other measurement points.

[0009] Furthermore, in step S1, the image acquisition time interval is calculated as follows: ,and

[0010] In the formula, Indicates the data collection time period. For the sampling frequency, Indicates the allowable error percentage; Interrupt-triggered mode refers to using CPU timer interrupts to acquire images in an orderly manner at a fixed frequency; The acquired images are stored in the acquisition buffer for asynchronous calculation of image measurement points in subsequent processing.

[0011] Furthermore, in step S2, the conditions for the original point of the virtual reference measurement point are determined based on the adjacent distance, displacement amplitude and frequency, and relevant measurement points with a value less than the preset adjacent distance, preset displacement amplitude and preset frequency threshold are selected as the original point of the virtual reference measurement point. Within a given time period, samples are taken at fixed time intervals. The displacement at each moment is calculated based on the images, resulting in a time-equivalent ordered result of the absolute position change of each measuring point. Assume a time series value of length N. Calculate the new sequence of its cumulative discrete differences:

[0012] in The average value of the actual sequence is calculated using the following formula: ; The final predicted value is obtained using the least squares method, as shown in the formula:

[0013] In the formula, This represents the output value of the nth model. The predicted final value is used as a comparison value of the displacement / deflection change of the measurement point as a backup virtual reference point. It is used to select the measurement point with the smallest displacement from multiple candidate measurement points as the relative virtual reference point.

[0014] Furthermore, in step S3, the parameters of the first-order shape function used to represent the changes in image position and image center point are defined by the Jacobian matrix, and the specific expression is as follows:

[0015] In the formula, , They represent the center point at , Displacement in the direction, Corresponding to , exist Displacement gradient in the direction, , Corresponding to , exist Displacement gradient in the direction; Then, the midline point P of the detected image is selected, and its alignment confidence is calculated. The alignment confidence of point P is quantified using the standardized zero-mean squared error correlation function.

[0016] In the formula, Jacobi matrix parameters The change This represents the grayscale value of the referenced subset. The grayscale value represents the target subset. This represents the average gray value of the referenced subset. This represents the average gray value of the target subset; The homogeneous coordinate form representing the coordinates of the center point of a subset; This refers to the warp function. This represents the ZNCC-normalized subpixel convergence value of the reference subset. Represents the ZNCC-normalized subpixel convergence value of the target subset; This represents the coordinate difference between any point within the subset and the center point. Next, the IC-GN algorithm is executed iteratively to solve for the changes in the parameters of the Jacobian matrix through iterative optimization. ; Finally, combining the previously obtained image location data of the measuring points, the Jacobian matrix parameters, and the relevant data of the virtual reference points, the actual displacement of the measuring points is solved using the coordinate transformation formula.

[0017] Furthermore, iterative optimization is used to solve for the changes in the parameters of the Jacobian matrix. The iterative form is

[0018]

[0019]

[0020] .

[0021] Furthermore, in step S4, based on the calculated current displacement result of the measuring point, it is determined whether this measuring point is used as a virtual reference measuring point by other measuring points. The product of the actual distance between the measuring point and the candidate virtual reference point and the displacement change of the candidate virtual reference point is used as the judgment index, and the virtual reference point with the smallest product is taken as the virtual reference point. If the measuring point is a virtual reference measuring point, its current calculated position data is stored in the absolute position sequence, and the current time is recorded. The absolute position sequence is defined as a first-in-first-out measuring point position data cache space, which is used to store the measuring point position data within a preset time period.

[0022] Further, in step S5, the data in the absolute position sequence is filtered, and the filtered data is arranged according to time intervals and stored in the serialized sequence. The serialized sequence serves as a first-in-first-out data buffer, storing the serialized and processed measurement point position data. The data is processed using an iterative differential filtering formula, expressed as:

[0023] in, Forward coefficients, For feedback coefficients, The original value, For calculated values; The filtered data is analyzed using the short-time Fourier transform algorithm. The short-time Fourier transform algorithm analysis process is as follows:

[0024] in The original signal, It is a sliding window function; After analyzing the Fourier equation, the monitoring points that were filtered out by the filtering function are added back to ensure that the serialized points are all standard time-equal measurement points and maintain the original trend of change.

[0025] Furthermore, in step S6, the polynomial coefficients of the serialized data are differentiated to fit the position data of multiple measurement points within a preset time period; the polynomial coefficients are solved using matrix determinant expansion for the first time, and thereafter the coefficients are adjusted appropriately according to position changes, and the polynomial coefficients are recalculated every preset time period; wherein, Define the polynomial formula as follows:

[0026] Define matrix: ; parameter vector ; Observation vector ; Solving the matrix equation: The matrix form of the equation is The formula for solving the coefficients is derived. ; The serialized position data is fitted using the above polynomial formula to obtain the current position data of the measuring point, which is used as the position data of the abstract reference point at the current moment. The position data of the measuring point as the abstract reference point is stored and replaced. The abstract reference point data shows the long-term displacement change of the measuring point, displays the displacement history of the measuring point, and serves as the displacement reference quantity for the changes of other measuring points.

[0027] On the other hand, an apparatus is also provided for performing the aforementioned machine vision intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation. The apparatus includes a housing, a camera component, a power supply component, a communication component, and a core processing component. The camera component acquires monitoring images, the power supply component provides energy, the communication component is responsible for monitoring data transmission, and the core processing component is used to perform the machine vision intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation, and calculates the monitoring displacement based on the acquired monitoring images.

[0028] Furthermore, the device also has a built-in temperature sensor to compensate for grayscale value drift errors caused by temperature changes. The temperature compensation algorithm includes a lookup table method and a mathematical model method. The lookup table method refers to pre-calibrating the pixel offset at different temperature points in the constant temperature chamber, generating a compensation coefficient table, and then using the real-time temperature value as an index to look up the table and directly output the corrected displacement data during actual operation. The mathematical modeling method establishes a mathematical model between temperature and measurement error, obtains measurement deviation data at different temperatures through experiments or data analysis, and then corrects the measurement results using a compensation algorithm based on the actual temperature value during measurement.

[0029] The beneficial effects of this invention are as follows: This invention employs a fusion algorithm of digital image correlation and measurement point cross-correlation, making full use of the spatial correlation characteristics of multi-measurement point data. Through cross-correlation operations, it performs cross-validation and consistency analysis on visual displacement data, effectively identifying and eliminating abnormal data points caused by factors such as image blurring, feature point mismatch, and local environmental disturbances, thus significantly improving the robustness of displacement calculation results.

[0030] The fusion algorithm of this invention effectively solves the measurement deviation problem caused by temperature changes in traditional monitoring. Through mutual constraints and data verification mechanisms among multiple measuring points, it can suppress systematic errors caused by factors such as structural thermal expansion and contraction, optical lens thermal drift, and image distortion due to diurnal temperature differences and seasonal temperature changes, thus achieving effective error correction. Even when image clarity fluctuates due to environmental influences and there are interference factors such as temperature changes in long-distance monitoring, this fusion algorithm can still function stably, suppressing the propagation and amplification of random errors and temperature-related systematic errors, ensuring the stability and reliability of displacement monitoring results, and achieving consistent monitoring accuracy under different environmental temperature conditions. This effectively solves the problem of measurement deviation caused by temperature interference in the monitoring of long bridges using traditional machine vision technology.

[0031] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0032] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the overall process of the machine vision intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the process of calculating the current displacement based on the current measuring point and the virtual reference measuring point according to an embodiment of the present invention; Figure 3 Displacement measurement images obtained using traditional digital image correlation algorithms; Figure 4 This is a short-time displacement measurement map obtained using the digital image correlation and measurement point cross-correlation fusion algorithm of the present invention; Figure 5 This illustrates a long-time displacement measurement image obtained using the digital image correlation and measurement point cross-correlation fusion algorithm of the present invention. Figure 6 This is a schematic diagram of the structure of a machine vision intelligent displacement monitoring device based on the fusion of digital image correlation and measurement point cross-correlation according to an embodiment of the present invention. Detailed Implementation

[0033] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0034] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0035] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0036] Please see Figures 1-3 This invention relates to a machine vision-based intelligent displacement monitoring method and device that integrates digital image correlation and measurement point cross-correlation.

[0037] Example 1 This embodiment first provides a detailed implementation process of a machine vision intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation, which includes at least the following steps: Step S1, Data Acquisition: S11. High-frequency image acquisition: Ensure consistent image acquisition time intervals, with time errors controlled within 10% of the interval. The acquisition time interval is calculated using the following formula: ,and

[0038] In the formula, Indicates the data collection time period. The acquisition frequency is set to the interrupt-triggered mode for image acquisition. Interrupt-triggered mode refers to using the CPU's timer interrupt to acquire images sequentially at a fixed frequency. If sufficient memory is available, the acquired images are stored in the acquisition buffer, and subsequent image measurement points are calculated asynchronously.

[0039] S2: Calculations and processing related to the forward trend algorithm: S21. Selection of the original point of the virtual reference measurement point: The situation where a monitoring device simultaneously monitors two or more displacement deflection points, each point corresponding to an infrared light source, a displacement deflection point, and a measurement point is regarded as a multi-measurement point environment. In the multi-measurement point environment, based on the position and displacement amplitude of each measurement point, the relevant measurement points with close adjacent distances and small displacement amplitudes and frequencies are selected as the original points of the virtual reference measurement points. In this algorithm, for a certain measurement point, theoretically, all other measurement points except itself can be used as measurement points of the virtual reference point, and the measurement point selected according to the algorithm is the original point of the virtual reference point. S22. Obtaining Time-Equivalent Ordered Results: Within a given time period, obtain the time-equivalent ordered results of the absolute position change at each measuring point. Specifically, use the fixed time interval of S1 to sample images, such as acquiring images at 10Hz. Calculate the displacement results at each moment based on the images to obtain high-frequency time-equivalent ordered data. Then, extract points at fixed intervals, such as taking one data point every 1000 points in the 10Hz data, thus transforming it into 0.01Hz time-equivalent ordered data; assuming a time series value of length N... Calculate the new sequence of its cumulative discrete differences using the following formula:

[0040] in The average value of the actual sequence is calculated using the following formula: .

[0041] S23. Obtaining the Predicted Final Value: The predicted final value is obtained using the least squares method, with the following formula:

[0042] The predicted final value is used as a comparison value for the displacement / deflection change of the measuring point to select as the backup virtual reference point, so as to select the measuring point with the smallest displacement as the relative virtual reference point from among multiple candidate measuring points.

[0043] S3 and DIC Algorithm Related Calculations and Processing: By combining the Jacobian matrix, the Hessian matrix, and the inverse Gaussian algorithm, the current image position of the measurement point is calculated. The parameters of the first-order shape function are defined by the Jacobian matrix, and their specific expression is as follows:

[0044] In the formula, , They represent the center point at , Displacement in the direction, Corresponding to , exist Displacement gradient in the direction, , Corresponding to , exist The displacement gradient in the direction. A first-order shape function refers to a function that only involves changes in the image position and the image center point, and only requires a one-dimensional matrix as shown below; a second-order shape function, because it involves changes in the shape, requires a two-dimensional matrix to describe the image changes.

[0045] Let any point within the image region center point P For, after its deformation The first-order form function is:

[0046] Secondly, the alignment confidence of point P is calculated. Each measurement point is bounded to a detected shape (i.e., a rectangle), and point P is the center point of that shape. Within the IC-GN (Incremental Gaussian-Newton) algorithm framework, the ZNSSD (Standardized Zero Mean Squared Difference) correlation function is used to quantify the alignment confidence of point P, thereby determining the accuracy of measurement point matching in the image and providing a basis for subsequent iterative optimization. The specific calculation formula is as follows:

[0047] In the formula, Jacobi matrix parameters The change This represents the grayscale values ​​of the referenced subset (i.e., the selected image frame). The grayscale value represents the target subset. This represents the average gray value of the referenced subset. This represents the average gray value of the target subset; The homogeneous coordinate form representing the coordinates of the center point of a subset; This refers to the warp function. This represents the ZNCC-normalized subpixel convergence value of the reference subset. Represents the ZNCC-normalized subpixel convergence value of the target subset; This represents the coordinate difference between any point in the subset and the center point.

[0048] Next, the IC-GN algorithm is used for iterative calculations. The process is as follows: Step 1: Initialize parameters: Set the following parameters according to the initial monitoring state (no displacement of the measuring point): Jacobian matrix parameters (Displacement gradient term is 0), parameter change Initialize the number of iterations. and maximum number of iterations .

[0049] Step 2: Calculate the deformed coordinates and ZNSSD confidence scores: based on the current... Combined with the above first-order shape function calculation Substitute into the ZNSSD formula to calculate the comparison confidence level of point P. Quantify the accuracy of measurement point matching.

[0050] Step 3: Calculate the first-order gradient and the approximate Hessian matrix: Substitute the current... After deformation, the coordinates and grayscale parameters are used to calculate the first-order gradient using the following formula. and approximate Hessian matrix :

[0051]

[0052] Step 4: Solve for parameter changes and update parameters Substituting the first-order gradient and the approximate Hessian matrix above, we can solve for the current... Update parameters Number of iterations .

[0053] Step 5: Determine the convergence condition verify If the condition is met, the iteration terminates and the output is given. The optimal position of the corresponding measurement point is determined; if the optimal position is not met, return to step 2 and proceed to the next iteration.

[0054] Finally, the current displacement of the measuring point is calculated. Combining the previously obtained measuring point image position data, Jacobian matrix parameters, and relevant data of the virtual reference point, the actual displacement of the measuring point is solved using coordinate transformation formulas.

[0055] S4. Virtual reference point determination and data storage: Virtual reference point determination: Based on the current displacement result of the measuring point calculated in step three, determine whether this measuring point is used as a virtual reference measuring point by other measuring points. Theoretically, each measuring point can be used as a virtual reference measuring point for other measuring points besides itself. The actual distance between the measuring point and the candidate virtual reference point is multiplied by the displacement change of the candidate virtual reference point, and the candidate point with the minimum value is taken as the virtual reference point. Absolute position sequence storage: If the measuring point is a virtual reference measuring point, its current calculated position data is stored in the "absolute position sequence" and the current time is recorded; Absolute position sequence definition: A first-in-first-out cache space for measuring point position data, used to store measuring point position data within 24 hours.

[0056] S5. Data frequency domain serialization and filtering: Frequency domain time serialization: The data in the "absolute position sequence" is filtered to remove data with excessive fluctuations. The remaining calculated data is arranged by time intervals and stored in the "serialized sequence". An iterative differential filtering formula is used to process the data, which is expressed as:

[0057] in, Forward coefficients, For feedback coefficients, The original value, For calculated values; A short-time serialization method is adopted (e.g., when sampling at 10Hz, the data is serialized into 12 seconds and 120 data points). The time sliding window is recursively calculated in seconds. The analysis focuses on the Short-Time Fourier Transform (STFT) algorithm. The STFT formula is as follows:

[0058] in The original signal, It is a sliding window function; After analyzing the Fourier equation, the monitoring points that were filtered out by the filtering function are added to ensure that the serialized points are all standard time-interval measurement points and maintain the original trend of change. Define a serialized sequence as a first-in-first-out data buffer to store the serialized and organized measurement point location data, preparing for subsequent fitting calculations.

[0059] S6. Polynomial Fitting and Determination of Abstract Benchmark Points: The serialized data was differentiated using polynomial coefficients, and the location data of 200 measurement points within 20 minutes were fitted. The polynomial coefficients were solved by matrix determinant expansion for the first time, and then the coefficients were adjusted appropriately according to the changes in location. The polynomial coefficients were recalculated every 2 hours. Define the polynomial formula as follows:

[0060] Define matrix: ; parameter vector ; Observation vector ; Solving the matrix equation: The matrix form of the equation is The formula for solving the coefficients is derived. ; The serialized position data is fitted using the above polynomial formula to obtain the current position data of the measurement point, which is then used as the position data of the abstract reference point at the current moment. Store and replace the position data of the measuring point as an abstract reference point. The abstract reference point data shows the long-term displacement change of the measuring point, displays the displacement history of the measuring point, and serves as the displacement reference quantity for the changes of other measuring points.

[0061] By continuously acquiring and processing images to obtain the displacement time history of multiple points, and repeating the above steps, the two algorithms can continuously interact and collaborate to ensure the fusion effect and stably handle displacement and image distortion problems on the device side.

[0062] Figure 3 The displacement measurement image obtained by a traditional digital image correlation algorithm is shown; Figure 4 This shows a short-time displacement measurement image obtained using the digital image correlation and measurement point cross-correlation fusion algorithm of the present invention; Figure 5 The diagram shows a long-term displacement measurement image obtained using the digital image correlation and measurement point cross-correlation fusion algorithm of the present invention.

[0063] Example 2 This embodiment provides a machine vision intelligent displacement monitoring device for implementing the machine vision intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation proposed in Embodiment 1. The device includes a housing, a camera component, a power supply component, a communication component, and a core processing component. The camera component acquires monitoring images, the power supply component provides energy, the communication component is responsible for monitoring data transmission, and the core processing component executes the machine vision intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation, calculating the monitored displacement based on the acquired monitoring images.

[0064] Example 3 This embodiment, based on Embodiments 1 and 2, also enables temperature compensation.

[0065] Temperature compensation refers to the thermal expansion and contraction of lens materials due to temperature changes, with the focal length shift conforming to a quadratic function model. For example, the coefficient of thermal expansion of aluminum lens barrels. Much higher than glass Temperature difference Can cause changes in focal length Thermal deformation of the camera's internal support structure causes CMOS displacement, and the contraction and expansion of the alloy can lead to sensor translation, resulting in an overall image shift. Uncompensated AD converters experience grayscale drift with temperature changes, causing edge detection and localization errors.

[0066] Temperature compensation algorithms include lookup table method and mathematical model method.

[0067] The lookup table method involves pre-calibrating pixel offsets at different temperature points (e.g., -40℃ / 25℃ / 85℃) in a constant temperature chamber and generating a compensation coefficient table. During actual operation, the real-time temperature value is used as an index to look up the table and directly output the corrected displacement data.

[0068] The mathematical modeling method involves establishing a mathematical model between temperature and measurement error, obtaining measurement deviation data at different temperatures through experiments or data analysis, and then correcting the measurement results using a compensation algorithm based on the actual temperature value during measurement. Specifically, the nonlinear relationship between temperature and pixel deviation can be fitted using the least squares method, for example, by constructing a quadratic polynomial. (T is temperature, a is fitting coefficient), input the temperature value in real time to calculate the compensation amount.

[0069] Based on the above solution, a temperature sensor needs to be embedded in the machine vision intelligent displacement monitoring device to measure the real-time temperature of the device.

[0070] Specifically, the equipment adopts a modular design, such as Figure 6 As shown, the device is divided into five main modules: a core processing unit, a front-end sensing unit, a communication interface unit, a power management unit, and an auxiliary function unit. It uses a 4XARM multi-core processor as its control and computing core, and works collaboratively with various sensors and communication interfaces to achieve machine vision intelligent displacement monitoring based on the fusion of digital image correlation and measurement point cross-correlation. Each unit is connected to a data bus via a standard interface, enabling closed-loop control of the entire process of data acquisition, processing, storage, transmission, and power supply.

[0071] The core processing unit, a 4XARM multi-core processor, serves as the control center and computing core of the entire terminal system, establishing bidirectional data interaction channels with all other units. The core processing unit coordinates the working timing of the front-end sensing unit, communication interface unit, power management unit, and auxiliary function units, achieving synchronous control and collaborative operation of multiple modules. It executes the machine vision intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation, receiving bridge images, vibration, tilt angle, and other data collected by the front-end sensing unit. It calculates the bridge displacement using a digital image correlation algorithm and combines it with a measurement point cross-correlation algorithm to complete data fusion and error correction, obtaining high-precision bridge deformation / displacement monitoring results. The core processing unit issues control commands to each unit, receives status feedback and collected data from each unit, outputs processed monitoring data to the communication interface unit, and simultaneously receives configuration commands from the host computer.

[0072] The front-end sensing unit communicates bidirectionally with the 4XARM core processing unit, responsible for the multi-dimensional data acquisition required for bridge deformation monitoring. It serves as the data source for displacement monitoring methods, specifically including: Visual CMOS camera component: The camera component is the core image acquisition module for machine vision displacement monitoring. It is used to acquire monitoring image sequences of the target area of ​​the bridge, provide raw image data for digital image correlation algorithms, and realize non-contact bridge displacement and deformation monitoring. MEMS Accelerometer Module: By collecting vibration acceleration data of bridge structures through MEMS accelerometer sensors, it can assist in the analysis of bridge dynamic response characteristics and perform cross-correlation verification with visual displacement data to improve the anti-interference capability and reliability of displacement monitoring. MEMS tilt module: It collects tilt change data of bridge structure through MEMS tilt sensor, provides supplementary data for the analysis of deformation characteristics such as bridge deflection and rotation, and integrates with visual displacement data to improve the multi-dimensional characterization of bridge deformation. Temperature and humidity module: Collects ambient temperature and humidity data at the bridge monitoring site. On the one hand, it can be used to analyze the impact of environmental factors on the deformation of the bridge structure. On the other hand, it can be used for environmental adaptability compensation of the equipment itself to avoid interference from changes in ambient temperature and humidity on the accuracy of sensor acquisition.

[0073] The communication interface unit serves as the device's external data interaction channel, communicating bidirectionally with the 4XARM core processing unit. It supports multi-mode data transmission to meet the monitoring data upload and device configuration requirements in different scenarios, specifically including: 4G communication module: Enables wireless remote transmission of monitoring data through cellular mobile communication network, suitable for bridge monitoring scenarios without wired network coverage, and can upload displacement monitoring results and equipment status data to remote monitoring platform in real time; WIFI communication module: Supports short-range wireless local area network communication, enabling on-site debugging, configuration and data export of equipment, and can also serve as a supplement to wired communication to realize the local or local area network transmission of monitoring data; Ethernet interface: Provides wired network communication capabilities, which can be connected to a local area network or fiber optic network via network cable to achieve high-bandwidth and high-reliability monitoring data transmission, suitable for bridge monitoring scenarios with wired network conditions; USB 3.0 interface: Provides a high-speed data transfer interface, which can be used for exporting locally stored data, upgrading device firmware, external debugging and other operations, and supports fast reading, writing and transmission of large image data.

[0074] The power management unit provides a stable and reliable energy supply to all components of the equipment, enabling power distribution, management, and protection, ensuring long-term stable operation of the equipment at the bridge site. Specifically, it includes: Lithium battery + supercapacitor energy storage module: As the energy carrier of the device, the lithium battery provides continuous basic power supply, while the supercapacitor can provide peak current support when the device starts up and the communication module operates at high power for a short time. At the same time, it can provide backup power during short-term power outages to avoid data loss caused by unexpected power failures. Power management module: It realizes the charging and discharging management of energy storage module, overcharge and over-discharge protection, voltage conversion and voltage regulation control, provides suitable operating voltage for different units (core processing unit, sensing unit, communication unit, etc.), and has the power status monitoring function, which can feed back battery power and power supply status to the core processing unit. Point-of-load power module: As the terminal distribution unit for power output, it distributes the stable voltage output by the power management module to each load unit of the device (such as sensors, processors, communication modules, etc.) as needed, so as to achieve precise power supply and load isolation and avoid power interference between different modules.

[0075] The auxiliary function unit communicates bidirectionally with the 4XARM core processing unit, providing basic guarantees and auxiliary support for the operation of the device, specifically including: Time synchronization unit: Provides high-precision time synchronization services for equipment, which can be achieved through satellite time synchronization or network time synchronization, ensuring the consistency of timestamps for image acquisition, data processing, and displacement calculation, and providing a time reference for the time series analysis of bridge deformation and the synchronous fusion of multi-point data; Storage unit: Provides local data storage capability for the device, which can store raw image data, sensor data, processed displacement monitoring results, device operation logs and other data, realize local data backup and avoid data loss due to network interruption; Indicator Unit: Through indicator lights, displays, and other devices, the operating status of the equipment (such as power supply status, communication status, data acquisition status, fault status, etc.) is displayed intuitively, making it convenient for on-site commissioning and maintenance personnel to quickly determine the working status of the equipment.

[0076] In summary, this invention solves the industry problems of low image clarity, poor stability, and insufficient displacement measurement accuracy caused by temperature and humidity changes, light interference, and atmospheric disturbances in the monitoring of long bridges with spans >100m by introducing a cross-correlation algorithm for measurement points and a temperature compensation algorithm. It realizes all-weather, high-precision automated monitoring of long-distance deflection and displacement of long bridges, filling the application gap of existing machine vision technology in the field of health monitoring of long bridges.

[0077] The temperature compensation algorithm of this invention can effectively eliminate the influence of environmental temperature changes on bridge structural deformation and optical imaging system. The cross-correlation algorithm of measurement points realizes cross-verification and fusion correction of visual displacement data and multi-source sensor data, which greatly reduces the measurement error caused by environmental interference. It solves the defects of traditional technology in unstable effect and low data reliability in long-distance monitoring, and provides reliable technical support for real-time and accurate monitoring of key limit state indicators such as structural deformation and pier displacement of long bridges.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A machine vision intelligent displacement monitoring method based on digital image correlation and measurement point cross-correlation fusion, characterized in that: The method includes: S1. Use interrupt trigger mode to perform high-frequency image acquisition, control the acquisition time error within a preset range, and store the acquired image in the acquisition buffer. S2. Select the measurement points that meet the conditions as the original points of the virtual reference measurement points, calculate the cumulative discrete difference sequence of the absolute position change of the measurement points, and obtain the predicted final value through the least squares method. S3. The position of the measurement point in the image is solved by integrating the Jacobian matrix, Hessian matrix and inverse Gaussian algorithm. The confidence level is quantified by the standardized zero mean square difference function. After iterative optimization by IC-GN algorithm, the actual spatial displacement of the measurement point is obtained by coordinate transformation. S4. Determine the virtual reference measuring point based on the actual displacement results of the measuring point, and store its position data and corresponding time in the preset absolute position sequence; S5. After removing anomalies from the data in the absolute position sequence, the data is serialized by time. Iterative differential filtering and short-time Fourier algorithm are used for analysis to supplement missing measurement points and obtain standardized serialized data. S6. Solve for the polynomial coefficients by matrix determinant, fit the serialized measurement point data to obtain the abstract reference point position, and periodically update the polynomial coefficients and abstract reference point data, using them as the reference quantity for the displacement of other measurement points.

2. The machine vision intelligent displacement monitoring method based on digital image correlation and fusion of measurement point cross-correlation according to claim 1, characterized in that: In step S1, the image acquisition time interval is calculated as follows: , and wherein denotes the acquisition time period, is the acquisition frequency, denotes the allowed error proportion; Interrupt-triggered mode refers to using CPU timer interrupts to acquire images in an orderly manner at a fixed frequency; The acquired images are stored in the acquisition buffer for asynchronous calculation of image measurement points in subsequent processing.

3. The machine vision intelligent displacement monitoring method based on digital image correlation and fusion with measuring point cross-correlation according to claim 1, characterized in that: In step S2, the conditions for the original point of the virtual reference measurement point are determined based on the adjacent distance, displacement amplitude and frequency. Relevant measurement points with a value less than the preset adjacent distance, preset displacement amplitude and preset frequency threshold are selected as the original points of the virtual reference measurement point. Within a given time period, samples are taken at fixed time intervals, and the displacement results at each moment are calculated based on the images to obtain the time-equivalent ordered results of the absolute position change of each measuring point. Assume a time series of values of length N Compute a new series of cumulative running differences: in The average value of the actual sequence is calculated using the following formula: ; The final predicted value is obtained using the least squares method, as shown in the formula: In the formula, This represents the output value of the nth model. The predicted final value is used as a comparison value of the displacement / deflection change of the measurement point as a backup virtual reference point. It is used to select the measurement point with the smallest displacement from multiple candidate measurement points as the relative virtual reference point.

4. The machine vision intelligent displacement monitoring method based on digital image correlation and fusion of measurement points cross-correlation according to claim 1, characterized in that: In step S3, the parameters of the first-order shape function used to represent the changes in image position and image center point are defined by the Jacobian matrix, and the specific expression is as follows: In the formula, , They represent the center point at , Displacement in the direction, Corresponding to , exist Displacement gradient in the direction, , Corresponding to , exist Displacement gradient in the direction; Then, the midline point P of the detected image is selected, and its alignment confidence is calculated. The alignment confidence of point P is quantified using the standardized zero-mean squared error correlation function. In the formula, Jacobi matrix parameters The change This represents the grayscale value of the referenced subset. The grayscale value represents the target subset. This represents the average gray value of the referenced subset. This represents the average gray value of the target subset; The homogeneous coordinate form representing the coordinates of the center point of a subset; This refers to the warp function. This represents the ZNCC-normalized subpixel convergence value of the reference subset. Represents the ZNCC-normalized subpixel convergence value of the target subset; This represents the coordinate difference between any point within the subset and the center point. Next, the IC-GN algorithm is executed iteratively to solve for the changes in the parameters of the Jacobian matrix through iterative optimization. ; Finally, combining the previously obtained image location data of the measuring points, the Jacobian matrix parameters, and the relevant data of the virtual reference points, the actual displacement of the measuring points is solved using the coordinate transformation formula.

5. The machine vision intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation according to claim 4, characterized in that: Iterative optimization to solve variation of jacobian matrix parameters An iterative form of 。 6. The machine vision intelligent displacement monitoring method based on digital image correlation and fusion with measuring point cross-correlation according to claim 1, characterized in that: In step S4, based on the calculated current displacement result of the measuring point, it is determined whether this measuring point is used as a virtual reference measuring point by other measuring points. The product of the actual distance between the measuring point and the candidate virtual reference point and the displacement change of the candidate virtual reference point is used as the judgment index, and the virtual reference point with the smallest product is taken as the virtual reference point. If the measuring point is a virtual reference measuring point, its current calculated position data is stored in the absolute position sequence, and the current time is recorded. The absolute position sequence is defined as a first-in-first-out measuring point position data cache space, which is used to store the measuring point position data within a preset time period.

7. The machine vision intelligent displacement monitoring method based on digital image correlation and fusion with measuring point cross-correlation according to claim 1, characterized in that: In step S5, the data in the absolute position sequence is filtered, and the filtered data is arranged by time interval and stored in the serialized sequence. The serialized sequence serves as a first-in-first-out data buffer, storing the serialized and processed measurement point position data. The data is processed using an iterative differential filtering formula, expressed as follows: wherein, is a forward coefficient, is a feedback coefficient, is an original value, is a calculated value; The filtered data is analyzed using the short-time Fourier transform algorithm. The short-time Fourier transform algorithm analysis process is as follows: wherein is the original signal, is a sliding window function; After analyzing the Fourier equation, the monitoring points that were filtered out by the filtering function are added back to ensure that the serialized points are all standard time-equal measurement points and maintain the original trend of change.

8. The machine vision intelligent displacement monitoring method based on digital image correlation and measurement point cross-correlation fusion according to claim 1, characterized in that: In step S6, the polynomial coefficients of the serialized data are differentiated to fit the position data of multiple measurement points within a preset time period; The first step was to use matrix determinant expansion to solve for the polynomial coefficients. Subsequently, the coefficients were adjusted appropriately based on changes in position, and the polynomial coefficients were recalculated every preset time interval. Define the polynomial formula as follows: Define matrix: ； Parameter vector ; Observation vector ; Matrix equation solution: the matrix form of the equation is , and the coefficient solution formula is derived ; The serialized position data is fitted using the above polynomial formula to obtain the current position data of the measuring point, which is used as the position data of the abstract reference point at the current moment. The position data of the measuring point as the abstract reference point is stored and replaced. The abstract reference point data shows the long-term displacement change of the measuring point, displays the displacement history of the measuring point, and serves as the displacement reference quantity for the changes of other measuring points.

9. A device for performing the machine vision intelligent displacement monitoring method based on digital image correlation and measurement point cross-correlation fusion according to any one of the preceding claims 1-8, characterized in that: The device includes a housing, a camera component, a power supply component, a communication component, and a core processing component. The camera component acquires monitoring images, the power supply component provides energy, the communication component is responsible for monitoring data transmission, and the core processing component is used to execute a machine vision intelligent displacement monitoring method based on the fusion of digital image correlation and measurement point cross-correlation, and calculates the monitored displacement based on the acquired monitoring images.

10. The machine vision intelligent displacement monitoring device based on the fusion of digital image correlation and measurement point cross-correlation according to claim 9, characterized in that: The device also has a built-in temperature sensor to compensate for grayscale value drift errors caused by temperature changes. The temperature compensation algorithm includes a lookup table method and a mathematical model method. The lookup table method refers to pre-calibrating the pixel offset at different temperature points in the constant temperature chamber, generating a compensation coefficient table, and then using the real-time temperature value as an index to look up the table and directly output the corrected displacement data during actual operation. The mathematical modeling method establishes a mathematical model between temperature and measurement error, obtains measurement deviation data at different temperatures through experiments or data analysis, and then corrects the measurement results using a compensation algorithm based on the actual temperature value during measurement.

Images