A temperature-varying pixel drift correction method and system for visual displacement measurement

Abstract

The application provides a temperature change pixel drift correction method and system for visual displacement measurement, the method comprising: acquiring an image sequence of a measurement target under different temperature conditions and temperature data corresponding to each frame of the image sequence; performing displacement calculation on the measurement target based on the image sequence to obtain displacement data; performing modal decomposition on the displacement data to obtain a dominant spatial mode and a corresponding modal coefficient; establishing a mapping relationship model between the modal coefficient and the temperature data; and determining the corresponding modal coefficient according to an actual measured temperature value based on the mapping relationship model, and determining a temperature change pixel drift amount in combination with the dominant spatial mode to correct the actual measured displacement data. The application can effectively correct the temperature change pixel drift in visual displacement measurement and improve the precision of displacement measurement.

Description

Technical Field

[0001] This application relates to the field of image processing and visual displacement measurement, specifically to a method and system for correcting temperature-varying pixel drift for visual displacement measurement. Background Technology

[0002] Visual displacement measurement technology, with its significant advantages such as non-contact operation, high precision, and full-field coverage, has been widely used in many fields, including aerospace, automotive manufacturing, and civil engineering. As the core image acquisition hardware of this technology, cameras and lenses are crucial for accurate imaging of real-world scenes. However, in practical applications, image acquisition faces complex working environments, and environmental factors inevitably affect its performance. In particular, ambient temperature varies significantly with the alternation of day and night and seasonal changes. Combined with the heat generated by the camera itself, these combined temperature fluctuations affect optical components such as cameras and lenses. On the one hand, temperature changes cause changes in the refractive index of optical materials, leading to alterations in imaging geometry parameters; on the other hand, temperature changes can also induce thermal deformation of the internal structure of the camera and lens, resulting in changes in pixel positions and imaging relationships during the imaging process. In visual displacement measurement scenarios with high precision requirements, such as bridge health monitoring and precision component manufacturing, this temperature-induced drift has become a significant source of error.

[0003] To mitigate the impact of temperature-induced pixel drift on visual displacement measurements, existing technologies typically address this through temperature control, reference compensation, and model correction. One approach involves using heating or cooling devices to ensure the image acquisition device operates under relatively stable temperature conditions, thus reducing the impact of temperature fluctuations. However, this method usually requires additional hardware, increasing system cost and complexity. Another approach compensates for drift by synchronously setting a reference region or target during measurement, but obtaining a long-term stable reference object is difficult in practical applications, limiting its applicability. A third approach corrects measurement errors caused by temperature variations by establishing a correlation between imaging parameter changes and temperature. This approach is usually based on an imaging geometry model, attributing imaging errors caused by temperature changes to variations in relevant imaging parameters, assuming these parameters are deterministic functions of temperature. However, due to the large number of parameters involved and the coupling relationships between different parameters, the model complexity is often high.

[0004] A literature search of existing technologies revealed Chinese Patent CN120778005A, which proposes a general thermal drift suppression method and visual measurement device for visual displacement measurement. This method eliminates accumulated errors caused by environmental changes due to the separation of calibration and measurement in traditional methods through real-time common-field correction. Furthermore, it achieves sub-pixel level thermal drift correction accuracy through block homography transformation, thereby improving measurement accuracy in complex real-world environments. While this method corrects drift by simultaneously setting a reference calibration plate during measurement, obtaining a long-term stable reference object is difficult in practical applications.

[0005] Therefore, there is a need for a temperature-dependent pixel drift correction method and apparatus that can directly characterize the relationship between temperature change and image drift without relying on imaging parameter modeling. Summary of the Invention

[0006] In view of the deficiencies in the prior art, the purpose of this application is to provide a method and system for correcting temperature-varying pixel drift for visual displacement measurement.

[0007] According to a first aspect of this application, a method for correcting temperature-varying pixel drift for visual displacement measurement is provided, comprising: Acquire image sequences of the target under different temperature conditions and temperature data corresponding to each frame of the image sequence; Displacement data is obtained by calculating the displacement of the measurement target based on the image sequence. Modal decomposition is performed on the displacement data to obtain the dominant spatial modes and their corresponding modal coefficients; Establish a mapping model between the modal coefficients and temperature data; Based on the mapping relationship model, the corresponding modal coefficients are determined according to the actual measured temperature values, and the temperature-varying pixel drift is determined in combination with the dominant spatial mode to correct the measured displacement data.

[0008] Optionally, the step of calculating the displacement of the measurement target based on the image sequence to obtain displacement data includes: Select any frame in the image sequence as a reference frame, and use the remaining frames as target frames; Several calculation points are determined in the reference frame according to a preset interval; Based on the image matching algorithm, the calculation points are matched and tracked in each of the target frames to obtain the displacement data of each calculation point; The displacement data of each calculation point in each target frame are combined to obtain the displacement data of each target frame. The displacement data includes the horizontal displacement component and the vertical displacement component of each calculation point at different times.

[0009] Optionally, the step of performing modal decomposition on the displacement data to obtain the dominant spatial modes and corresponding modal coefficients includes: The displacement data of each frame of the image are arranged in chronological order, and the displacement data of each frame of the image is expanded in the spatial dimension to construct a spatiotemporal data matrix. The spatiotemporal data matrix is ​​subjected to singular value decomposition to obtain a time basis matrix, a spatial basis matrix, and a singular value diagonal matrix. The time basis matrix includes several time basis vectors, the spatial basis matrix includes several spatial basis vectors, and the singular value diagonal matrix is ​​a diagonal matrix composed of several singular values. Select the largest value from the singular value diagonal matrix r A number of singular values ​​are selected, and the spatial basis vectors corresponding to the selected singular values ​​are used as the dominant spatial modes. The modal coefficients corresponding to the dominant spatial modes are obtained by processing the time basis vectors using the following formula: in, Indicates the first k The dominant spatial mode in the first i Frame corresponding temperature value Modal coefficients under; Indicates the first k Singular values ​​corresponding to each dominant spatial mode Indicates the first k The time basis vectors corresponding to the dominant spatial modes In the i Components at the frame; Indicates the first i The temperature value corresponding to the frame.

[0010] Optionally, establishing the mapping model between the modal coefficients and temperature data includes: Obtain the temperature data corresponding to the modal coefficients over time; The modal coefficients of each dominant spatial mode are fitted with the corresponding temperature data. After the fitting is completed, a mapping relationship model between the modal coefficients and the temperature is obtained. The mapping relationship model includes the mapping function between the modal coefficients of each dominant spatial mode and the temperature.

[0011] Optionally, the mapping function is established based on the Gaussian process regression algorithm, using temperature as input and the modal coefficients of the corresponding dominant spatial mode as output, and is used to characterize the nonlinear mapping relationship between the modal coefficients of the dominant spatial mode and the corresponding temperature.

[0012] Optionally, the step of determining the corresponding modal coefficients based on the actual measured temperature values ​​according to the mapping relationship model, and determining the temperature-varying pixel drift amount in conjunction with the dominant spatial mode, to correct the measured displacement data includes: Based on the mapping relationship model, the corresponding modal coefficients and dominant spatial modes are determined according to the actual measured temperature values. Based on the modal coefficients corresponding to each dominant spatial mode, the dominant spatial modes are superimposed to obtain the temperature-varying pixel drift amount; The measured displacement data is subtracted from the temperature-dependent pixel drift to eliminate the measurement deviation caused by temperature changes, thus obtaining the corrected displacement data.

[0013] According to a second aspect of this application, a temperature-varying pixel drift correction system for visual displacement measurement is provided, comprising: an image acquisition device, a temperature control device, a displacement calculation module, a mode decomposition module, a mapping establishment module, and a drift correction module, wherein... The image acquisition device is used to acquire image sequences of the measurement target under different temperature conditions; The temperature control device is used to control the cooling or heating of the image acquisition device and to acquire temperature data corresponding to each frame of the image sequence. The displacement calculation module is used to perform displacement calculation on the measurement target based on the image sequence to obtain displacement data; The mode decomposition module is used to perform mode decomposition on the displacement data to obtain the dominant spatial modes and corresponding mode coefficients; The mapping establishment module is used to establish a mapping relationship model between the modal coefficients and temperature data; The drift correction module is used to determine the corresponding modal coefficients based on the actual measured temperature values ​​according to the mapping relationship model, and to determine the temperature-varying pixel drift amount in combination with the dominant spatial mode, so as to correct the measured displacement data.

[0014] Optionally, the image acquisition device includes a camera and a lens, the lens being connected to the camera, and the camera and lens being used to acquire image sequences of the measurement target; the temperature control device includes a cooling unit, a heating unit, a temperature sensor, a control unit, and an insulation box; the image acquisition device is placed inside the insulation box, the temperature sensor is arranged on the camera and / or lens for acquiring temperature data, the cooling unit is located inside the insulation box, the heating unit is located inside the insulation box, and the cooling unit, the heating unit, and the temperature sensor are all connected to the control unit.

[0015] Optionally, the temperature-varying pixel drift correction system further includes a computer, and the displacement calculation module, the mode decomposition module, the mapping establishment module and the drift correction module are all stored in the computer.

[0016] According to a third aspect of this application, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of a temperature-varying pixel drift correction method for visual displacement measurement as provided in the first aspect of this application.

[0017] This application provides a temperature-varying pixel drift correction method for visual displacement measurement. By performing mode decomposition on displacement data, the dominant spatial modes and their corresponding modal coefficients are obtained. This reduces modeling complexity and improves computational efficiency while preserving key feature information. Based on mode decomposition, a mapping relationship between each dominant modal coefficient and temperature is established, enabling quantitative calculation of temperature-varying pixel drift. This improves the ability to characterize complex drift behavior and thus enhances the accuracy of temperature-varying pixel drift correction. This application can effectively correct temperature-varying pixel drift in visual displacement measurement, improving the accuracy of displacement measurement.

[0018] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0019] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of a temperature-dependent pixel drift correction method in one embodiment of this application; Figure 2 This is a schematic diagram of the structure of a temperature-varying pixel drift correction system in one embodiment of this application; Figure 3 This is a schematic diagram of a temperature-varying pixel drift correction system in one embodiment of this application; Figure 4 This is a comparison diagram of the average absolute error before and after temperature-varying pixel drift correction in one embodiment of this application; wherein, (a) is a comparison diagram of the average absolute error of horizontal displacement before and after temperature-varying pixel drift correction, and (b) is a comparison diagram of the average absolute error of vertical displacement before and after temperature-varying pixel drift correction. Figure 5 This is a comparison of displacement amplitude cloud maps before and after temperature-induced pixel drift correction in one embodiment of this application.

[0020] In the diagram: 1-Camera, 2-Lens, 3-Cooling unit, 4-Heating unit, 5-Temperature sensor, 6-Control unit, 7-Insulated box, 8-Observation window, 9-Supporting structure, 10-Computer, 11-Measurement target, 12-Optical platform. Detailed Implementation

[0021] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all fall within the protection scope of the present application. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0022] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0023] Visual displacement measurement technology is widely used in many fields. As the core image acquisition hardware of this technology, cameras and lenses are crucial for accurate imaging of real-world scenes. However, temperature fluctuations can affect optical components such as cameras and lenses, and this temperature-induced drift has become a significant source of error. Existing technologies typically address this through temperature control, reference compensation, and model correction, but these methods have the following drawbacks: temperature control usually requires additional hardware configuration, increasing system cost and complexity; reference compensation requires setting a reference area or target, but obtaining a long-term stable reference object is difficult in practical applications; and model correction involves a large number of parameters, and the coupling relationships between different parameters can easily lead to high model complexity. Based on these problems, this application provides a temperature-varying pixel drift correction method for visual displacement measurement to solve the aforementioned issues.

[0024] Reference Figure 1 As shown, this application provides a method for correcting temperature-dependent pixel drift in visual displacement measurement, including: S1. Obtain the image sequence of the target under different temperature conditions and the temperature data corresponding to each frame of the image sequence; S2. Calculate the displacement of the target based on the image sequence to obtain displacement data; S3. Perform modal decomposition on the displacement data to obtain the dominant spatial modes and their corresponding modal coefficients; S4. Establish a mapping model between modal coefficients and temperature data; S5. Based on the mapping relationship model, determine the corresponding modal coefficients according to the actual measured temperature values, and combine the dominant spatial modes to determine the temperature-varying pixel drift, and correct the measured displacement data.

[0025] For example, the temperature-varying pixel drift correction method employs a temperature-varying pixel drift correction system, which includes an image acquisition device, a temperature control device, and a computer 10. The image acquisition device includes a camera 1 and a lens 2, and the temperature control device includes a cooling unit 3, a heating unit 4, a temperature sensor 5, a control unit 6, and a heat preservation box 7. The image sequence is acquired by the camera 1 and the lens 2, and the temperature data is acquired by the temperature sensor 5 installed in the temperature-varying pixel drift correction system and is obtained synchronously with the image sequence. Specifically, the temperature data is synchronously acquired by the temperature sensor 5 arranged on the camera 1 and the lens 2. The image acquisition device is arranged facing the measurement target 11 and continuously images the measurement target 11 to obtain an image sequence arranged in chronological order. The temperature sensor 5 is used to acquire the temperature information of key parts of the image acquisition device in real time and to perform time correspondence with the image sequence. The temperature sensor 5 can be arranged on the camera housing, the outer surface of the lens, or its adjacent position. In this embodiment, several temperature sensors 5 are respectively arranged on the camera 1 and the lens 2. By averaging the temperature values ​​acquired by multiple temperature sensors 5, comprehensive temperature data characterizing the overall temperature state of the image acquisition device is obtained.

[0026] In some implementations, the image acquisition frequency and temperature acquisition frequency can be the same or different. Preferably, the temperature acquisition frequency is higher than the image acquisition frequency, and the correspondence between the two is achieved through time matching. The image sequence and temperature data are aligned using timestamps. For each frame of image, the temperature data corresponding to its time point is selected first. When the sampling times are inconsistent, the temperature data within the nearest time window is selected and averaged. When there is no directly matching temperature data for a corresponding time point, time interpolation can be used to supplement it, thereby ensuring the consistency between the image data and the temperature data. In this embodiment, the image acquisition frequency is 1 fps, and the temperature acquisition frequency is 2 Hz. For each frame of image, the temperature data corresponding to its timestamp is selected, and the temperature sampling values ​​within the corresponding time interval are averaged to determine the temperature value corresponding to that frame.

[0027] In this embodiment, the image acquisition device is arranged inside the temperature control device to keep the image acquisition device in a controlled temperature environment; the image acquisition device and the measurement target 11 are fixedly installed on the optical platform 12 to reduce the influence of external vibration on the measurement process, so that the measurement result mainly reflects the drift caused by temperature change; the measurement target 11 is a pattern with obvious texture features. In this embodiment, a speckle pattern is preferred. The speckle pattern has the characteristics of random distribution and rich gray-scale variation, which is conducive to achieving stable matching and obtaining the full-field displacement of the measurement area.

[0028] The embodiments described above in this application, by performing modal decomposition on displacement data, obtain the dominant spatial modes and their corresponding modal coefficients. This reduces modeling complexity and improves computational efficiency while preserving key feature information. Based on modal decomposition, a mapping relationship between each dominant modal coefficient and temperature is established, enabling quantitative calculation of temperature-dependent pixel drift. This improves the ability to characterize complex drift behavior and thus enhances the accuracy of temperature-dependent pixel drift correction. This application can effectively correct temperature-dependent pixel drift in visual displacement measurement, improving the accuracy of displacement measurement.

[0029] In some specific embodiments of this application, calculating the displacement of the target based on an image sequence to obtain displacement data may further include: S21. Select any frame in the image sequence as the reference frame, and use the remaining frames as the target frames. S22. Determine several calculation points in the reference frame according to a preset spacing; S23. Based on the image matching algorithm, the calculation points are matched and tracked in each target frame to obtain the displacement data of each calculation point. S24. Combine the displacement data of each calculation point in each target frame to obtain the displacement data of each target frame. The displacement data includes the horizontal displacement component and the vertical displacement component of each calculation point at different times.

[0030] For example, the above displacement calculation is implemented based on machine vision methods (such as image matching algorithms, digital image correlation methods, etc.) to obtain full-field displacement data of the measurement area where the target is located.

[0031] Specifically, in this embodiment, the displacement calculation is preferably implemented using a digital image correlation method. A frame in the image sequence is selected as a reference frame, and several calculation points are selected in the reference frame at preset intervals. A corresponding sub-region is constructed with the calculation points as the center. The horizontal and vertical displacement components of the calculation points in the reference frame are both 0. In some embodiments, the sub-region is preferably a square region with a side length that is an odd number in the range of 15 to 51 pixels, and the spacing between calculation points can be 3 to 10 pixels. In this embodiment, the side length of the sub-region is 41 pixels, and the spacing between calculation points is 5 pixels.

[0032] In subsequent frames, using sub-regions as templates, matching searches are performed within the corresponding regions. The image grayscale distribution or texture features of the sub-regions between the reference frame and the target frame are evaluated based on similarity metrics to determine the initial matching positions of each calculation point at different times. On this basis, a displacement shape function is introduced to describe the local deformation of the sub-regions, and a sub-pixel matching method is used to refine the matching positions to obtain sub-pixel level displacement results. Furthermore, based on the displacement results of each calculation point, full-field displacement data of the measurement area is constructed, forming a displacement sequence that changes over time. The displacement data includes the horizontal and vertical displacement components corresponding to each calculation point at different times.

[0033] In some specific embodiments of this application, modal decomposition of displacement data to obtain the dominant spatial modes and corresponding modal coefficients may further include: S31. Arrange the displacement data of each frame image in chronological order, and expand the displacement data of each frame image in the spatial dimension to construct a spatiotemporal data matrix. S32. Perform singular value decomposition on the spatiotemporal data matrix to obtain the time basis matrix, the spatial basis matrix, and the singular value diagonal matrix. The time basis matrix includes several time basis vectors, the spatial basis matrix includes several spatial basis vectors, and the singular value diagonal matrix is ​​a diagonal matrix composed of several singular values. S33. Select the largest value from the singular value diagonal matrix. r A number of singular values ​​are selected, and the spatial basis vectors corresponding to the selected singular values ​​are used as the dominant spatial modes. S34. Using the time basis vector, the modal coefficients corresponding to the dominant spatial modes are obtained according to the following formula: in, Indicates the first k The dominant spatial mode in the first i Frame corresponding temperature value Modal coefficients under; Indicates the first k Singular values ​​corresponding to each dominant spatial mode Indicates the firstk The time basis vectors corresponding to the dominant spatial modes In the i Components at the frame; Indicates the first i The temperature value corresponding to the frame.

[0034] For example, step S3 in this embodiment includes: organizing displacement data under different temperature conditions in chronological order, expanding the displacement data of each frame in the spatial dimension, and constructing a spatiotemporal data matrix, as shown in the following formula: in, Represents a spatiotemporal data matrix. N This indicates the number of image frames corresponding to the displacement data. M The number of calculation points is indicated; each row of the spatiotemporal data matrix represents the displacement data distribution of the corresponding frame, and each column represents the change of the displacement component of the corresponding calculation point in the horizontal or vertical direction over time.

[0035] Then, singular value decomposition is performed on the spatiotemporal data matrix to obtain the spatial modes and their corresponding modal coefficients, as shown in the following formula: in, The time basis matrix, Let be the spatial basis matrix. Let be a singular value diagonal matrix, where, , This is a function that takes the minimum value.

[0036] Specifically, singular value decomposition is performed on the spatiotemporal data matrix to extract its... A singular value, and obtain the same as Each singular value corresponds to a unique value. time basis vectors and There are 10 spatial basis vectors, and each spatial basis vector is used as a spatial mode. The singular values ​​are arranged in descending order on the main diagonal of the singular value diagonal matrix.

[0037] Next, select the first [singular value] based on its magnitude. r The selected singular value is used as the basis vector of the selected singular value as the dominant spatial mode, and the corresponding singular value is the basis vector of the selected singular value. k The dominant spatial modes are The corresponding number k The time basis vectors are The corresponding modal coefficients for: ; in, Represents the time basis vector In the i Components at the frame; r This indicates the number of dominant spatial modes selected, and r To meet Integers. r The value is based on the previous r The characterization effect of each spatial mode on the main response features of temperature-varying pixel drift is determined in this embodiment. .

[0038] Based on this, a low-dimensional modal representation of the displacement data is obtained, and the previous... r The dominant spatial modes are represented as follows: The corresponding modal coefficient matrix is ​​represented as Among them, the modal coefficient sequence .

[0039] This application achieves a low-dimensional modal representation of temperature-varying pixel drift response by performing modal decomposition on displacement data, which reduces modeling complexity and improves computational efficiency while preserving key feature information.

[0040] It should be noted that the displacement data obtained in step S2 and the displacement data for modal decomposition in step S3 are both measured displacement data used to establish a mapping relationship model. The measured displacement data includes displacement data under different temperature conditions, which is used to train or calibrate the mapping relationship model. The measured displacement data in step S5 is the displacement data to be corrected obtained through visual displacement measurement in the actual application stage.

[0041] In some specific embodiments of this application, establishing a mapping model between modal coefficients and temperature data may further include: S41. Obtain the temperature data corresponding to the modal coefficients over time; S42. Fit the modal coefficients of each dominant spatial mode to the corresponding temperature data. After fitting, obtain the mapping relationship model between the modal coefficients and the temperature. The mapping relationship model includes the mapping function between the modal coefficients of each dominant spatial mode and the temperature.

[0042] For example, in step S4 of this application embodiment, the relationship between the modal coefficients and temperature data of each dominant spatial mode is fitted to establish a mapping relationship model between the modal coefficients and temperature. Specifically, from the modal coefficient matrix... The modal coefficients corresponding to each dominant spatial mode are extracted, and the corresponding temperature data sequences are obtained based on the temperature data. : in, Indicates the firsti The temperature value corresponding to the frame.

[0043] Then, for each dominant spatial mode, a mapping relationship between the modal coefficients and temperature is established, and the first... k Each modal coefficient is expressed as a function of temperature: in, Indicates the first k The mapping function between the modal coefficients of a dominant spatial mode and temperature.

[0044] In this embodiment, the mapping function The model is built using Gaussian process regression. Specifically, the temperature sequence T is used as the input variable, and the corresponding modal coefficient sequence is used as the input variable. As the output variable, a regression model is established between temperature and modal coefficients. The Gaussian process regression assumption is that the mapping function satisfies the Gaussian process prior: in, C It is a covariance function used to describe different temperatures. The degree of correlation between the corresponding modal coefficients; In this embodiment, the covariance function is preferably a radial basis function kernel, which has the following form: in, This represents the signal variance, used to control the amplitude of the function fluctuation. This is a length scale parameter used to control the smoothness of function changes.

[0045] Furthermore, a covariance matrix is ​​constructed based on the covariance function. Its elements are: in, The first element in the covariance matrix i Line 1 j The element values ​​of the column; and They represent the first i Frame and the j The temperature value corresponding to the frame.

[0046] In the actual modeling process, to account for the influence of measurement errors, a noise term is introduced into the covariance matrix, resulting in a covariance matrix with the noise term. : in, Indicates the noise variance. It is an identity matrix.

[0047] Furthermore, the hyperparameters of the Gaussian process regression model are determined by maximizing the logarithmic marginal likelihood, thereby establishing a mapping relationship model between the modal coefficients and temperature. The logarithmic marginal likelihood function is expressed as: The above method enables the establishment of a mapping model between modal coefficients and temperature, which can then be used for subsequent modal coefficient prediction.

[0048] In some specific embodiments of this application, the mapping function is established based on the Gaussian process regression algorithm, with temperature as input and the modal coefficients of the corresponding dominant spatial mode as output, and is used to characterize the nonlinear mapping relationship between the modal coefficients of the dominant spatial mode and the corresponding temperature.

[0049] For example, the mapping function is established using the Gaussian process regression method.

[0050] Based on modal decomposition, this application establishes a mapping relationship between temperature and each dominant modal coefficient, thereby realizing the decomposition modeling of temperature-varying pixel drift response, which improves the ability to characterize complex drift behavior and thus enhances the accuracy of temperature-varying pixel drift correction.

[0051] In some specific embodiments of this application, based on a mapping relationship model, the corresponding modal coefficients are determined according to the actual measured temperature values, and the temperature-varying pixel drift is determined in conjunction with the dominant spatial mode to correct the measured displacement data. This may further include: S51. Based on the mapping relationship model, determine the corresponding modal coefficients and dominant spatial modes according to the actual measured temperature values; S52. Based on the modal coefficients corresponding to each dominant spatial mode, the dominant spatial modes are superimposed to obtain the temperature-varying pixel drift amount; S53. Subtract the temperature-induced pixel drift from the measured displacement data to eliminate the measurement deviation caused by temperature changes, and obtain the corrected displacement data.

[0052] For example, in step S5, based on the mapping relationship model, the modal coefficients corresponding to each dominant spatial mode are determined according to the temperature measurement values; and the dominant spatial modes are modally superimposed according to the modal coefficients corresponding to each dominant spatial mode to obtain the temperature-varying pixel drift amount, which is then used to correct the displacement data to be measured; specifically, for a given temperature input Construct the covariance vector between it and the training samples. : And predict the corresponding modal coefficients based on the mapping relationship model: in, Indicates the first k The dominant spatial mode at temperature Predicted modal coefficient values ​​under the following conditions; Indicates temperature input The covariance vector between the temperature values ​​in the training temperature sequence; K This represents the covariance matrix after introducing a noise term; Indicates the first k The sequence of modal coefficients corresponding to each dominant spatial mode.

[0053] Based on this, the modal coefficients are combined with the corresponding spatial modes, and the spatial modes are superimposed according to the contribution of each dominant spatial mode to obtain the temperature-varying pixel drift under the corresponding temperature conditions. It is represented as: in, For the first k One dominant spatial mode.

[0054] The above method enables the prediction of modal coefficients and reconstruction of drift response based on temperature input, thereby obtaining the temperature-varying pixel drift amount under the corresponding temperature conditions and correcting the measured displacement data to be corrected.

[0055] This application directly establishes a relationship model between temperature and drift based on image drift response, avoiding dependence on imaging parameter modeling and improving the applicability of the method in this application.

[0056] Reference Figure 2 and Figure 3 As shown, based on the same inventive concept, another embodiment of this application provides a temperature-varying pixel drift correction system for visual displacement measurement. The temperature-varying pixel drift correction system 100 includes an image acquisition device, a temperature control device, a displacement calculation module, a mode decomposition module, a mapping establishment module, and a drift correction module, wherein... Image acquisition device 110 is used to acquire image sequences of the measurement target under different temperature conditions; Temperature control device 120 is used to control the cooling or heating of the image acquisition device and to acquire temperature data corresponding to each frame of the image sequence. The displacement calculation module 130 is used to perform displacement calculation on the measurement target based on the image sequence to obtain displacement data; The mode decomposition module 140 is used to perform mode decomposition on the displacement data to obtain the dominant spatial modes and the corresponding mode coefficients; The mapping establishment module 150 is used to establish a mapping relationship model between modal coefficients and temperature data; The drift correction module 160 is used to determine the corresponding modal coefficients based on the actual measured temperature value according to the mapping relationship model, and to determine the temperature-varying pixel drift amount in combination with the dominant spatial mode, so as to correct the measured displacement data.

[0057] It should be noted that the modules in the temperature-varying pixel drift correction system for visual displacement measurement provided in the above embodiments of this application correspond to the steps of the temperature-varying pixel drift correction method for visual displacement measurement in any of the above embodiments. Those skilled in the art can refer to the step features of the temperature-varying pixel drift correction method for visual displacement measurement to implement the corresponding modules in the temperature-varying pixel drift correction system for visual displacement measurement, which will not be repeated here.

[0058] In some specific embodiments of this application, the image acquisition device includes a camera 1 and a lens 2, with the lens 2 connected to the camera 1. The camera 1 and the lens 2 are used to acquire image sequences of the measurement target 11. The temperature control device includes a cooling unit 3, a heating unit 4, a temperature sensor 5, a control unit 6, and an insulation box 7. The image acquisition device is placed inside the insulation box 7. The temperature sensor 5 is arranged on the camera 1 and / or the lens 2 to acquire temperature data. The cooling unit 3 is located inside the insulation box 7, and the heating unit 4 is located inside the insulation box 7. The cooling unit 3, the heating unit 4, and the temperature sensor 5 are all connected to the control unit 6.

[0059] For example, the image acquisition device is used to continuously image a measurement target to obtain an image sequence arranged in chronological order. In some embodiments, the camera 1, lens 2, and measurement target of the image acquisition device can be mounted on the optical platform 12 via the support structure 9 to improve the stability of the measurement process and reduce the influence of external vibrations on the measurement process. The measurement target is a pattern with obvious texture features; in this embodiment, a speckle pattern is preferred. The insulated box 7 is used to form a relatively closed imaging environment with heat insulation properties to reduce the influence of external temperature fluctuations on the measurement process and to provide a stable thermal environment for the image acquisition device. The insulated box can accommodate the image acquisition device. In some embodiments, an observation window 8 is provided on the side of the insulated box near the measurement target 11 for the image acquisition device to image the external measurement target. The observation window 8 is made of a transparent material, preferably glass. The temperature sensor 5 is used to acquire temperature data of the image acquisition device and the imaging environment. In this embodiment, there can be multiple heating and cooling units, which can be arranged in pairs at different positions inside the insulated box to achieve uniform adjustment of the imaging environment; the control unit uses PID control method to adjust the heating and cooling units to achieve stable control of the imaging environment temperature.

[0060] In some specific embodiments of this application, the temperature-varying pixel drift correction system also includes a computer 10, in which the displacement calculation module, mode decomposition module, mapping establishment module and drift correction module are all stored.

[0061] For example, the computer includes a processor and a memory. The memory stores a computer program. When the processor executes the computer program, it performs displacement calculations on the image sequence to obtain displacement data of the measurement area. It then performs mode decomposition on the displacement data to obtain spatial modes and corresponding mode coefficients. Based on the mode coefficients and temperature data, it establishes a mapping model and determines the corresponding mode coefficients according to the temperature measurement value. Based on this, it combines the spatial modes to perform mode superposition, obtaining the temperature-varying pixel drift, thereby achieving temperature-varying pixel drift correction of the measured displacement data. The control unit is connected to the computer 10 and is used to control the heating and cooling units according to the acquired imaging environment temperature data to adjust the imaging environment temperature.

[0062] In one specific embodiment, an image acquisition device is used to continuously acquire images of the measurement target, and a temperature control device is used to adjust the imaging environment within a certain temperature range, while simultaneously acquiring corresponding temperature data; displacement calculation is performed based on the acquired image sequence, and temperature-induced pixel drift correction processing is performed according to the method described above. Figure 4 (a) and Figure 4 As shown in (b), the error index curves corresponding to the horizontal and vertical displacements are given respectively. After correction, the mean absolute error (MAE) in both directions is at a low level, indicating that the temperature-induced pixel drift has effectively corrected the measurement results in both directions; Figure 5 As shown, the comparison results of the displacement amplitude cloud map of the last frame are given. After correction, the displacement amplitude is significantly reduced and the overall distribution tends to be more uniform, thereby improving the accuracy and precision of the displacement measurement results.

[0063] In another embodiment of this application, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described temperature-varying pixel drift correction method for visual displacement measurement.

[0064] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules that implement the above methods), computer instructions, etc., and the aforementioned computer programs and computer instructions can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0065] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0066] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0067] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0068] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0070] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0072] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.

[0073] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A method for correcting temperature-varying pixel drift in visual displacement measurement, characterized in that, include: Acquire image sequences of the target under different temperature conditions and temperature data corresponding to each frame of the image sequence; Displacement data is obtained by calculating the displacement of the measurement target based on the image sequence. Modal decomposition is performed on the displacement data to obtain the dominant spatial modes and their corresponding modal coefficients; Establish a mapping model between the modal coefficients and temperature data; Based on the mapping relationship model, the corresponding modal coefficients are determined according to the actual measured temperature values, and the temperature-varying pixel drift is determined in combination with the dominant spatial mode to correct the measured displacement data. The step of performing modal decomposition on the displacement data to obtain the dominant spatial modes and corresponding modal coefficients includes: The displacement data of each frame of the image are arranged in chronological order, and the displacement data of each frame of the image is expanded in the spatial dimension to construct a spatiotemporal data matrix. The spatiotemporal data matrix is ​​subjected to singular value decomposition to obtain a time basis matrix, a spatial basis matrix, and a singular value diagonal matrix. The time basis matrix includes several time basis vectors, the spatial basis matrix includes several spatial basis vectors, and the singular value diagonal matrix is ​​a diagonal matrix composed of several singular values. Select the largest value from the singular value diagonal matrix r A number of singular values ​​are selected, and the spatial basis vectors corresponding to the selected singular values ​​are used as the dominant spatial modes. The modal coefficients corresponding to the dominant spatial modes are obtained by processing the time basis vectors using the following formula: ； in, Indicates the first k The dominant spatial mode in the first i Frame corresponding temperature value Modal coefficients under; Indicates the first k Singular values ​​corresponding to each dominant spatial mode Indicates the first k The time basis vectors corresponding to the dominant spatial modes In the i Components at the frame; Indicates the first i The temperature value corresponding to the frame.

2. The temperature-varying pixel drift correction method for visual displacement measurement according to claim 1, characterized in that, The step of calculating the displacement of the measurement target based on the image sequence to obtain displacement data includes: Select any frame in the image sequence as a reference frame, and use the remaining frames as target frames; Several calculation points are determined in the reference frame according to a preset interval; Based on the image matching algorithm, the calculation points are matched and tracked in each of the target frames to obtain the displacement data of each calculation point; The displacement data of each calculation point in each target frame are combined to obtain the displacement data of each target frame. The displacement data includes the horizontal displacement component and the vertical displacement component of each calculation point at different times.

3. The temperature-varying pixel drift correction method for visual displacement measurement according to claim 1, characterized in that, The process of establishing the mapping model between the modal coefficients and temperature data includes: Obtain the temperature data corresponding to the modal coefficients over time; The modal coefficients of each dominant spatial mode are fitted with the corresponding temperature data. After the fitting is completed, a mapping relationship model between the modal coefficients and the temperature is obtained. The mapping relationship model includes the mapping function between the modal coefficients of each dominant spatial mode and the temperature.

4. The temperature-varying pixel drift correction method for visual displacement measurement according to claim 3, characterized in that, The mapping function is established based on the Gaussian process regression algorithm, using temperature as input and the modal coefficients of the corresponding dominant spatial mode as output through regression analysis. It is used to characterize the nonlinear mapping relationship between the modal coefficients of the dominant spatial mode and the corresponding temperature.

5. The temperature-varying pixel drift correction method for visual displacement measurement according to claim 1, characterized in that, The process involves determining the corresponding modal coefficients based on the actual measured temperature values ​​according to the mapping relationship model, and combining this with the dominant spatial mode to determine the temperature-varying pixel drift, thereby correcting the measured displacement data. This includes: Based on the mapping relationship model, the corresponding modal coefficients and dominant spatial modes are determined according to the actual measured temperature values. Based on the modal coefficients corresponding to each dominant spatial mode, the dominant spatial modes are superimposed to obtain the temperature-varying pixel drift amount; The measured displacement data is subtracted from the temperature-dependent pixel drift to eliminate the measurement deviation caused by temperature changes, thus obtaining the corrected displacement data.

6. A temperature-varying pixel drift correction system for implementing the method of any one of claims 1-5 for visual displacement measurement, characterized in that, include: The system comprises an image acquisition device, a temperature control device, a displacement calculation module, a mode decomposition module, a mapping establishment module, and a drift correction module. The image acquisition device is used to acquire image sequences of the measurement target under different temperature conditions; The temperature control device is used to control the cooling or heating of the image acquisition device and to acquire temperature data corresponding to each frame of the image sequence. The displacement calculation module is used to perform displacement calculation on the measurement target based on the image sequence to obtain displacement data; The mode decomposition module is used to perform mode decomposition on the displacement data to obtain the dominant spatial modes and corresponding mode coefficients; The mapping establishment module is used to establish a mapping relationship model between the modal coefficients and temperature data; The drift correction module is used to determine the corresponding modal coefficients based on the actual measured temperature values ​​according to the mapping relationship model, and to determine the temperature-varying pixel drift amount in combination with the dominant spatial mode, so as to correct the measured displacement data.

7. A temperature-varying pixel drift correction system for visual displacement measurement according to claim 6, characterized in that, The image acquisition device includes a camera and a lens, the lens being connected to the camera. The camera and lens are used to acquire image sequences of the measurement target. The temperature control device includes a cooling unit, a heating unit, a temperature sensor, a control unit, and an insulation box. The image acquisition device is placed inside the insulation box. The temperature sensor is arranged on the camera and / or lens to acquire temperature data. The cooling unit and the heating unit are both located inside the insulation box. The cooling unit, the heating unit, and the temperature sensor are all connected to the control unit.

8. A temperature-varying pixel drift correction system for visual displacement measurement according to claim 6, characterized in that, The temperature-varying pixel drift correction system also includes a computer, and the displacement calculation module, the mode decomposition module, the mapping establishment module and the drift correction module are all stored in the computer.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-5.

Images