A method and device for measuring displacement distance based on machine vision

Abstract

The present invention relates to the technical field of bridge measurement, and specifically to a displacement distance measurement method and device based on machine vision, comprising: S1, acquiring image data of a bridge structure, constructing the image data into three-dimensional point cloud data, and obtaining coordinate values ​​of corresponding positions on the bridge structure according to the three-dimensional point cloud data; S2, dividing the obtained coordinate values ​​into a plurality of observation surfaces, and calculating the displacement index of the bridge structure on each observation surface; S3, acquiring a target position between each observation surface, verifying the angle, vertical direction, horizontal direction and image overlapping deflection point between each observation surface, calculating the matching coefficient between each deflection point, combining the matching coefficient with the actual distribution position of the bridge structure, introducing the displacement index, and obtaining the bridge displacement coefficient on each observation surface; thereby improving the accuracy and efficiency of identifying abnormal bridge displacement.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge measurement, and in particular to a displacement distance measurement method and device based on machine vision. Background Art

[0002] With the rapid development of my country's national economy, the traffic volume has increased significantly, the traffic density and vehicle load have increased, the burden on bridges has become increasingly serious, and the safety has been declining. Many bridges have already experienced excessive deflection and significant changes in bridge line shape before accidents. Therefore, it is very important to monitor the structural response and behavior of bridges under various environmental conditions through a bridge health monitoring system. It obtains various information reflecting the structural status and environmental factors, thereby analyzing the structural health status and evaluating the reliability of the structure, providing a scientific basis for bridge management and maintenance decisions.

[0003] For example, Chinese patent publication number CN116734744A discloses a lightweight measurement method and system for online camera displacement based on infrared targets. It innovatively uses infrared thermal imaging cameras to form a temperature difference between the hollow elliptical feature target and the background, and enhances the infrared emissivity of the graphene spray paint target surface, and then uses image processing technology to measure the target displacement. A lightweight measurement system for engineering structure displacement used in low visibility is designed. In view of the problems of poor quality of thermal imaging original images, high noise, and large solution errors caused by environmental occlusion of the target, a sub-pixel ellipse center positioning algorithm based on thermal imaging image enhancement and purification optimization is proposed to effectively improve the solution accuracy and robustness of the results.

[0004] For example, Chinese patent publication number CN116105674A discloses a method and device for measuring the curve of vertical deformation of a bridge with temperature change. The method includes: obtaining temperature data output by an ambient temperature sensor in real time, obtaining coordinate data output by a GNSS receiver installed at a bridge monitoring point in real time, obtaining the elevation of the bridge monitoring point based on the elevation coordinates, translating the temperature data along the time axis, the translation amount being the time that the bridge deformation lags behind the temperature change, fitting the relationship curve between the bridge monitoring point elevation and the temperature based on a sample data pair consisting of the translated temperature data and the bridge elevation data, and obtaining the relationship curve between the bridge monitoring point deformation and the temperature change based on the relationship curve.

[0005] When measuring the displacement of the bridge itself, the prior art uses image recognition methods to identify the displacement generated at this time, and identifies the corresponding situation of the bridge based on these data. However, this measurement actually measures the specific displacement of the bridge, and it is easy to ignore the impact of different displacement values ​​generated on the bridge, as well as the relative situation between the corresponding displacements when the displacement occurs on the bridge. As a result, the final measured displacement cannot be compensated according to the structure of the bridge, and it is difficult to identify abnormal displacement on the bridge. Summary of the invention

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a displacement distance measurement method based on machine vision, including: S1, obtaining image data of the bridge structure, constructing the image data into three-dimensional point cloud data, and obtaining the coordinate value of the corresponding position on the bridge structure according to the three-dimensional point cloud data.

[0007] S2, dividing the obtained coordinate values ​​into multiple observation planes, and calculating the displacement index of the bridge structure on each observation plane.

[0008] S3, obtain the target position between each observation surface, verify the angle between each observation surface, the vertical direction, the horizontal direction and the deflection point of the image overlap, calculate the matching coefficient between each deflection point, combine the matching coefficient with the actual distribution position of the bridge structure, introduce the displacement index, and obtain the bridge displacement coefficient on each observation surface.

[0009] S4, measuring the displacement of each bridge structure when multiple observation surfaces are combined, extracting the displacement mutation on each bridge structure, and determining the relative compensation coefficient between the bridge structures.

[0010] S5, based on the obtained relative compensation coefficient and bridge displacement coefficient, the values ​​of the bridge structure at different observation planes and angles are comprehensively summed to determine the direction and magnitude of the comprehensive displacement generated by the current bridge structure.

[0011] A displacement distance measuring device based on machine vision uses the above-mentioned displacement distance measuring method based on machine vision, including: an image processing module for acquiring image data of a bridge structure and converting the image data into three-dimensional point cloud data, and using image processing technology to extract coordinate values ​​of corresponding positions on the bridge structure.

[0012] The displacement index calculation module is used to divide the obtained coordinate values ​​into multiple observation planes and calculate the displacement index of the bridge structure on each observation plane.

[0013] The bridge displacement coefficient calculation module is used to obtain the target position between each observation surface, verify the angle, vertical direction, horizontal direction and the deflection point of image overlap, calculate the matching coefficient, and obtain the bridge displacement coefficient in combination with the displacement index.

[0014] The compensation coefficient calculation module is used to measure the displacement of the bridge structure when multiple observation surfaces are combined, extract the displacement mutation, and determine the relative compensation coefficient.

[0015] The comprehensive displacement calculation module is used to comprehensively sum the values ​​of the bridge structure at different observation surfaces and angles based on the relative compensation coefficient and the bridge displacement coefficient to determine the direction and size of the comprehensive displacement.

[0016] The beneficial effects of the present invention are as follows: 1. The present invention can more accurately reflect the displacement of the bridge structure on different observation surfaces by constructing three-dimensional point cloud data and calculating the displacement index; at the same time, the accuracy of displacement measurement is further improved by matching the deflection points and calculating the bridge displacement coefficient.

[0017] 2. The present invention uses machine vision technology to perform displacement measurement, avoiding the tedious manual operation and data processing process in traditional methods and improving measurement efficiency; it can be applied to different types of bridge structures and complex displacement situations, and obtains the final displacement measurement results through a comprehensive displacement calculation module, providing comprehensive data support for the monitoring and maintenance of bridge structures.

[0018] 3. The present invention can timely discover abnormal displacement of bridge structure through displacement mutation extraction and compensation coefficient calculation module, and correct it through compensation coefficient to ensure the reliability of displacement measurement; this is of great significance for preventing bridge safety accidents and ensuring traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0020] Figure 1 The present invention is a flowchart of a displacement distance measurement method based on machine vision.

[0021] Figure 2 The present invention is a flow chart of step S2 of a displacement distance measurement method based on machine vision.

[0022] Figure 3 The present invention is a flow chart of step S3 of a displacement distance measurement method based on machine vision.

[0023] Figure 4 The present invention is a flow chart of step S32 of a displacement distance measurement method based on machine vision.

[0024] Figure 5 The present invention is a flow chart of step S5 of a displacement distance measurement method based on machine vision.

[0025] Figure 6It is a system framework diagram of a displacement distance measurement device based on machine vision. DETAILED DESCRIPTION

[0026] The embodiments of the present invention are described in detail below. The embodiments described below are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. If no specific techniques or conditions are specified in the embodiments, the techniques or conditions described in the literature in the art or the product specifications are used.

[0027] See also Figure 1 , a displacement distance measurement method based on machine vision, comprising: S1, acquiring image data of a bridge structure, constructing the image data into three-dimensional point cloud data, and obtaining coordinate values ​​of corresponding positions on the bridge structure according to the three-dimensional point cloud data.

[0028] S2, dividing the obtained coordinate values ​​into multiple observation planes, and calculating the displacement index of the bridge structure on each observation plane.

[0029] S3, obtain the target position between each observation surface, verify the angle between each observation surface, the vertical direction, the horizontal direction and the deflection point of the image overlap, calculate the matching coefficient between each deflection point, combine the matching coefficient with the actual distribution position of the bridge structure, introduce the displacement index, and obtain the bridge displacement coefficient on each observation surface.

[0030] S4, measuring the displacement of each bridge structure when multiple observation surfaces are combined, extracting the displacement mutation on each bridge structure, and determining the relative compensation coefficient between the bridge structures.

[0031] S5, based on the obtained relative compensation coefficient and bridge displacement coefficient, the values ​​of the bridge structure at different observation planes and angles are comprehensively summed to determine the direction and magnitude of the comprehensive displacement generated by the current bridge structure.

[0032] In step S1, the three-dimensional point cloud data can be collected by setting it at the key position of the bridge structure to obtain the image of the current bridge and the corresponding spatial coordinate set; for the observation surface, the image of the current bridge is collected from multiple positions and angles, and these bridge images are converted into coordinate values.

[0033] At the same time, multiple image data will be obtained and overlapped to obtain multiple coordinate values ​​for a position. For these coordinate values, it is necessary to verify the change rules of these coordinate values ​​under different situations, and whether the coordinate values ​​are consistent in a relatively stable state, to judge the coordinate changes of the current bridge structure at different times. These change values ​​represent the displacement of the current bridge.

[0034] In one embodiment of the present invention, in step S2, the obtained coordinate values ​​need to be divided into a plurality of observation planes, and the displacement index of the bridge structure on each observation plane is calculated based on the values ​​obtained on these observation planes.

[0035] The division of coordinate values ​​into observation surfaces is an important concept in bridge displacement monitoring, which helps to analyze the deformation of a specific area in a more detailed manner; the observation surface can be a physically well-defined plane or a virtually defined reference surface; in this way, it is possible to focus on the behavior of certain critical parts of the bridge structure and measure and analyze them in detail.

[0036] For the observation surfaces set up at this time, a baseline reference observation surface, a local deformation observation surface, a multi-dimensional observation surface and a time series-related observation surface will be set; the points on these observation surfaces will change over time, and the main purpose is to verify the displacement-related data measured on the corresponding observation surfaces as the displacement index used at this time.

[0037] The datum reference observation surface is set up for reference of the overall coordinate value displacement; the local deformation observation surface is used to identify whether there is a corresponding bending or deformation angle on the coordinate corresponding to the bridge structure; the multi-dimensional observation surface divides the three-dimensional data into two-dimensional content to obtain multiple coordinate values; the time series related observation surface is to observe the periodic displacement of the corresponding coordinates on each observation surface or the path of displacement change.

[0038] These observation surfaces will divide a single bridge into multiple observation surfaces at corresponding positions according to the different bridge structures that need to be identified at present, and read the values ​​of these observation surfaces on the corresponding bridges. At the same time, the changes in the displacement of the current bridge will be identified according to the different settings of the observation surfaces.

[0039] like Figure 2 As shown, the implementation process of step S2 also includes: S21, constructing a time series of the current observation surface, extracting feature points from the current observation surface, using feature descriptors to match feature points obtained at different times, and establishing a corresponding relationship between feature points; the corresponding relationship between feature points is to identify the displacement of the current observation surface caused by changes in time in the corresponding time series, and to identify the correspondence between corresponding feature points when the bridge structure changes over time, so as to obtain multiple groups of corresponding point pairs to identify the displacement value on the current feature point; the feature descriptor can represent the mark value set for the current feature point, which is mainly used to identify the source of the feature point.

[0040] When establishing the correspondence between feature points, the detection points of the bridge structure on each observation surface can be obtained, and the coordinate values ​​corresponding to the monitoring points can be identified. These monitoring points can be regarded as the feature points currently being processed. According to the change of the coordinate values ​​of the monitoring points over time, an environmental change curve can be set, and the displacement generated by the current monitoring point in the environmental change curve can be used as the displacement value identified at this time.

[0041] S22, calculating the displacement value of the feature point on the current observation surface according to the correspondence relationship of the feature points.

[0042] S23, searching for common feature points between adjacent observation surfaces and connecting them, to determine the relative position relationship between adjacent observation surfaces; the relative position relationship at this time is represented as a displacement vector between the common feature points between adjacent observation surfaces, and the displacement vector represents the specific displacement that can be generated by a certain point on the bridge after a period of observation after using multiple observation surfaces to analyze the current bridge, and this value will be represented in the form of a vector.

[0043] S24, calculating the displacement influence coefficient and the displacement correlation coefficient of the displacement value between adjacent observation surfaces according to the relative position relationship between adjacent observation surfaces.

[0044] S25, obtaining the displacement index on each observation surface according to the displacement influence coefficient and the displacement correlation coefficient.

[0045] In the current step, firstly, a single observation surface is analyzed to identify the abnormal mode of each observation surface, and at the same time, it is determined whether there are common feature points between the observation surfaces under the corresponding abnormal modes, or whether there are common feature points between the observation surfaces under different abnormal modes. These common feature points are used to identify the relative position relationship between the observation surfaces, and at the same time, the displacement influence coefficient and the displacement correlation coefficient of the displacement value under the relative position relationship are calculated. The displacement influence coefficient can be calculated according to the total difference ratio of the displacement values ​​between adjacent observation surfaces at this time, and the displacement correlation coefficient can be calculated according to the difference between adjacent observation surfaces, as well as the maximum and minimum values ​​of the difference.

[0046] The displacement correlation coefficient can be expressed as follows: obtaining the difference in displacement values ​​between adjacent observation surfaces, constructing a difference sequence, and calculating the displacement correlation coefficient according to the difference sequence.

[0047] ;in, represents the displacement correlation coefficient, represents the maximum value of the difference sequence, represents the minimum value of the difference sequence, Represents the difference in displacement value corresponding to the i-th element in the difference sequence, It indicates the adjustment coefficient, and its value is 0.001. Indicates the length of the difference sequence, which is from 1 to n.

[0048] The displacement influence coefficient is expressed as: ;in, represents the displacement influence coefficient, represents the weight of the i-th element in the difference sequence, It represents the sum of the displacement value differences in the difference sequence; at this time, the displacement influence coefficient mainly determines the ratio of the difference in the generated displacement value to the total value to obtain the corresponding comprehensive difference.

[0049] After the displacement influence coefficient and the displacement correlation coefficient are obtained, the weighted average of the displacement influence coefficient and the displacement correlation coefficient is taken as the displacement index at this time.

[0050] ;in, represents the displacement index, represents the displacement correlation coefficient, represents the displacement influence coefficient, represents the weight coefficient of the displacement correlation coefficient, Represents the weight coefficient of the displacement influence coefficient; the weight coefficients of the displacement correlation coefficient and the displacement influence coefficient can be 0.5 and 0.5 respectively to comprehensively consider the influence of these two coefficients.

[0051] Through the above steps, not only can the displacement index of the bridge structure on each observation surface be accurately calculated, but also each observation surface can be effectively identified and connected to form a comprehensive understanding and monitoring of the entire bridge structure behavior. This method not only improves the monitoring accuracy, but also provides strong support for the long-term tracking of the bridge health status.

[0052] In this step, when calculating the displacement value and the difference of the displacement value, the points on the observation surface selected at this time are one-to-one corresponding. What is mainly calculated is the displacement value of these points after mapping on different observation surfaces, and the relationship between different observation surfaces is identified based on this displacement value. When the calculation is performed at this time, the displacement value difference generated under multiple time points will be compared, and these points will be constructed into a difference sequence to complete the identification of the displacement index. This displacement index will represent the relative displacement on each observation surface.

[0053] In one embodiment of the present invention, in step S3, the bridge displacement coefficient is obtained by calculating the relative values ​​between the observation surfaces and comprehensively processing the points with corresponding relationships.

[0054] In this step, the set observation surface will generate images of the current bridge structure at multiple angles according to the form of collected data, and the image here will contain the target position that needs to be identified, and the target position can represent a combination of multiple points; and determine whether there are deflection points between the current observation surfaces in different horizontal and vertical directions. This point indicates that a point different from the normal displacement direction has appeared at this position, or a point that exceeds the normal displacement value. These points will be considered as deflection points at this time. At the same time, the image overlap and angle setting are used to analyze the bridge structure from multiple angles, and overlap the observation surfaces at multiple angles after the angle change, identify the overlapping points, and split out the non-overlapping points at this time to identify whether there are related problems.

[0055] For example, the currently identified bridge structure is divided into multiple window images, and each image can be identified as an observation surface. At this time, the identification angle is to determine whether the initial angle of the current bridge structure at the corresponding position has changed. The identification of the horizontal and vertical directions is also to identify whether the bridge has been displaced at the corresponding position. Image superposition is to identify the degree of change.

[0056] like Figure 3 As shown, the implementation method of step S3 includes: S31, setting a target position on each observation surface, and calculating the angle between each observation surface according to the target position; the target position here is used to indicate the area on the bridge structure that currently needs to be identified, and the angle between each observation surface is calculated to determine the relative position between the currently identified areas, as well as the relative situation between the multiple observation planes currently divided for the bridge.

[0057] S32, based on the angles between the observation surfaces, identify the relative position relationship between the observation surfaces, and obtain the first deflection point in the horizontal and vertical directions; at this time, the relative position relationship represents the corresponding angles between the observation surfaces, in order to understand the distribution of the current observation surfaces.

[0058] S33, performing an overlapping process on each observation surface, and identifying a second deflection point after the image overlaps.

[0059] S34, identify the first deflection point and the second deflection point in turn, and calculate the matching coefficient between the deflection points; the matching coefficient at this time is to compare the first deflection point and the second deflection point in the horizontal direction, vertical direction, angle, and the number of overlapping pixels, calculate multiple similarity values ​​for these values, and use the comprehensive value of these similarity values ​​as the matching coefficient used at this time.

[0060] When acquiring the first deflection point, it is necessary to identify the contour information corresponding to the first deflection point; perform edge detection and classification on the points around the first deflection point to determine the contour information corresponding to these first deflection points, and whether these deflection points indicate that the current bridge structure has large or abnormal deflection points in the horizontal or vertical direction.

[0061] When identifying the second deflection point, the current overlapped image is compared with the normal image to identify the points where displacement occurs and the points where the bridge structure shows obvious movement. The bridge structure where these points were originally located is verified, and the pixel overlap ratio between each observation surface during overlap is calculated for these bridge structures to represent the number of these displaced points.

[0062] like Figure 4 As shown, when obtaining the first deflection point in the horizontal direction and the vertical direction, the specific implementation method of step S32 can be, S321, obtaining the position points in the horizontal relative position and the vertical relative position on each observation surface, and verifying the distance between each position point and the supporting point of the adjacent bridge structure.

[0063] S322, according to the distance between each position point and the supporting point of the adjacent bridge structure, obtain the relative displacement of each position point relative to the supporting point at different times, and obtain the displacement distribution interval and displacement distribution probability corresponding to the relative displacement.

[0064] S323, obtaining the distance between each position point and the center point of the support point, and calculating the joint probability value of each position point with respect to the distance between the position point and the center point of the support point and the relative displacement ratio, wherein the joint probability value represents the probability value of the distance value between the current position point and the center point of the support point and the relative displacement ratio of the current position point occurring together.

[0065] S324, after normalizing the relative displacement, the displacement distribution probability and the joint probability value of the position point, the distribution coefficient of the position point is calculated, and the position point whose distribution coefficient is greater than a preset threshold is used as the first deflection point to be output.

[0066] The above displacement distribution interval represents the interval corresponding to the maximum and minimum values ​​of the relative displacement, and the displacement distribution probability represents the probability of the relative displacement. At this time, the relative displacement is obtained by calculating the distance between the position point and the support point in different time periods, and the relative displacement ratio is the ratio of the relative displacement to the original distance between the position point and the support point.

[0067] The distribution coefficient can be expressed as: ;in, represents the distribution coefficient, It represents the index value of the relative displacement of the position point at time t. At this time, the index value of the relative displacement is the value after normalization according to the displacement distribution interval of the relative displacement, that is, the relative displacement minus the minimum value of the displacement distribution interval divided by the difference between the maximum value and the minimum value of the displacement distribution interval. The final ratio is the index value of the relative displacement at this time; represents the displacement distribution probability of the position point at time t, Represents the joint probability value of the position point at time t; Represents the number of moments, and the value of t ranges from 1 to T; The average value of the index value representing the relative displacement, The standard deviation of the index value representing the relative displacement, represents the average value of the displacement distribution probability, represents the standard deviation of the displacement distribution probability, represents the average of the joint probability values, represents the standard deviation of the joint probability value, represents the weight coefficient of relative displacement, represents the weight coefficient of the displacement distribution probability, Represents the weight coefficient of the joint probability value; the weight coefficient is set to 0.3, 0.4, and 0.3 in the order of relative displacement, displacement distribution probability, and joint probability value.

[0068] For the preset threshold, the average value of the distribution coefficient calculated on the relevant bridge structure in the historical data can be selected as the preset threshold set at this time. The output of these position points can reflect the main changes in the displacement at these position points, and reflect the obvious points of the current bridge structure when displacement occurs. These points can intuitively reflect the displacement of the current bridge, and the displacement measured by these points can accurately express the current status of the bridge.

[0069] The second deflection point can be obtained by obtaining the number of overlaps of the position points of each observation surface, clustering the current position points according to the number of overlaps, using the relative displacement ratio between the position points and the support points as the index value when clustering each position point, and using the position point corresponding to the largest cluster center after clustering as the output second deflection point.

[0070] The second deflection point is relative to the first deflection point. The second deflection point is the point where the bridge structure is most frequently displaced and output. These points overlap many times under multiple observation surfaces, and the relative displacement ratio changes little. These points indicate the points where the bridge is most likely to be displaced. If the values ​​at these points change significantly, the overall structure of the bridge will be affected.

[0071] S35, calculating the cosine similarity of the first deflection point and the second deflection point, and obtaining a matching coefficient according to the cosine similarity of the first deflection point and the second deflection point; at this time, calculating the cosine similarity value is to convert the obtained first deflection point and the second deflection point into a vector form, and calculating the cosine similarity in the form of vector dot product and modulus length.

[0072] After obtaining the matching coefficient, it is necessary to compare the current matching coefficient with the actual distribution position of the bridge structure to determine at which positions of the bridge structure the displacement values ​​will be similar, and combine the previously calculated displacement index with the matching coefficients at these positions to obtain the current bridge displacement coefficient.

[0073] The way to combine the matching coefficient with the actual distribution position of the bridge structure is to collect the matching coefficient with the corresponding position on the bridge structure, that is, to identify the first deflection point and the second deflection point corresponding to the current matching coefficient, set the matching threshold according to the position of the bridge structure, verify the points where the matching coefficient is greater than the matching threshold, and calculate the average value of the matching coefficient at this time to quantify the relationship between these positions. When the matching coefficient is greater than the matching threshold, the matching coefficient is adjusted to obtain the adjusted matching coefficient, and the adjusted matching coefficient is output.

[0074] For each matching coefficient, it can be expressed as, ;in, represents the matching coefficient between the jth first deflection point and the kth second deflection point, represents the adjusted matching coefficient of the jth first deflection point and the kth second deflection point, represents the exponential decay function of the jth first deflection point and the kth second deflection point, ,in represents the exponential constant, Represents the attenuation coefficient, which can be 0.1 or 0.2. represents the distance between the jth first deflection point and the kth second deflection point. This distance represents the calculated corresponding distance value to indicate the influence of the distance between the first deflection point and the second deflection point on the current matching coefficient. Indicates the matching threshold, which is set by the average value of the matching coefficient in the historical data; represents the maximum value of the matching coefficient, Indicates the minimum value of the matching coefficient.

[0075] The current implementation method is to adjust the matching coefficients that are greater than the matching threshold to reduce the differences between these matching coefficients and realize the specific value of the current matching coefficient.

[0076] Finally, the bridge displacement coefficient can be expressed as: ;in, represents the bridge displacement coefficient, represents the displacement index of the jth first offset point, represents the number of first deflection points, represents the number of the second deflection points, j ranges from 1 to m, and k ranges from 1 to K; The weight coefficient of the displacement index of the j-th first offset point is represented. The value of the weight coefficient of the displacement index can be set to the average value of the historical data set at the corresponding first deflection point; The weight coefficient of the matching coefficient of the j-th first offset point is represented. The weight coefficient of the matching coefficient set at this time can be set to the ratio of the matching coefficient value related to the corresponding first offset point to the overall matching coefficient value.

[0077] After obtaining the bridge displacement coefficient, the displacement measurement of the current bridge can be known, so that the approximate displacement of the bridge can be known based on the current analysis, which is convenient for analyzing related issues of the bridge.

[0078] In one embodiment of the present invention, step S4 is to identify and process multiple observation surfaces to obtain a relative compensation coefficient. In this step, the identified offset refers to the value of the matching coefficient of the currently measured bridge structure, and the moving average model is used for analysis to obtain a relative compensation coefficient; at the same time, the displacement mutation represents the value that does not conform to the trend change in the current calculation. This value needs to be processed using the corresponding compensation factor or adjustment parameter to reduce the predicted displacement error at this time.

[0079] The relative compensation coefficient can be expressed as, ;in, represents the relative compensation coefficient, represents the basic compensation factor, Represents the adjustment parameter, and the value range of the adjustment parameter is 0 to 1; It represents the index value of the matching coefficient at time t, that is, the average value of all matching coefficient values ​​of the first deflection point and the second deflection point at time t.

[0080] The basic compensation factor must also meet the following requirements: , represents the moving average of the matching coefficient at time t, It represents the moving average value of the matching coefficient at time t-1. The window size for calculating the moving average value at this time can be set according to the current frequency of bridge monitoring. The window size corresponding to the frequency related to the current bridge monitoring in the historical data is used as the window size for calculating the moving average value at this time.

[0081] When the basic compensation factor meets the corresponding conditions, the calculated relative compensation coefficient is output to obtain the compensation value required for displacement measurement in the current scene.

[0082] In one embodiment of the present invention, step S5 comprehensively sums the values ​​of the bridge structure at different observation planes and angles, and expresses these values ​​in the form of vectors to identify the magnitude and corresponding direction of the current displacement.

[0083] In this step, the relative compensation coefficient and the bridge displacement coefficient are comprehensively judged to determine the comprehensive situation of the current bridge structure when the displacement occurs. The comprehensive situation of the current displacement is judged based on the size of these two values, and the direction of these displacement changes is extracted as the corresponding displacement identified at this time.

[0084] like Figure 5 As shown, the processing method of step S5 includes: S51, first obtaining the original displacement value of the bridge structure, the relative displacement of the bridge structure in the current observation surface relative to other observation surfaces, the relative compensation coefficient and the bridge displacement coefficient; and normalizing these data.

[0085] S52, adjusting the displacement value of the current observation surface according to the original displacement value of the bridge structure, the relative displacement of the bridge structure in the current observation surface relative to other observation surfaces and the relative compensation coefficient, and setting the displacement value of the observation surface.

[0086] S53, combining the observed displacement value with the bridge displacement coefficient to obtain a comprehensive displacement vector, and using the magnitude and direction of the comprehensive displacement vector as the direction and magnitude of the comprehensive displacement generated by the current bridge structure.

[0087] The observed displacement value can be expressed as: ;in, represents the observed displacement value of the observation surface a, represents the original displacement value of the bridge structure at observation surface a; represents the relative displacement of the bridge structure between observation surface a and observation surface b, represents the average value of the relative displacement of the bridge structure in the observation plane, Represents the relative compensation coefficient corresponding to observation surface a and observation surface b.

[0088] The comprehensive displacement vector can be expressed as ;in, represents the integrated displacement vector, represents the bridge displacement coefficient of observation surface a, Represents the number of observation surfaces. The value range of a is 1 to A. The value range of b is the same as that of a. Observation surface b always represents other observation surfaces except observation surface a.

[0089] After obtaining the comprehensive displacement vector, the modulus of this comprehensive displacement vector is the size of the displacement currently being evaluated and measured, and the direction of the vector is the direction of the current measured displacement, which is used to evaluate the overall related displacement. At this time, the measured displacement is based on the actual measured value, and these actual values ​​are further processed to amplify the impact and significance of this displacement value, so as to achieve overall analysis and identification of the displacement value, so that an image detection device can be set at the corresponding position to complete the measurement and processing of the current bridge structure.

[0090] like Figure 6 As shown, in one embodiment of the present invention, a displacement distance measuring device based on machine vision is also provided, including: an image processing module, a displacement index calculation module, a bridge displacement coefficient calculation module, a compensation coefficient calculation module, and a comprehensive displacement calculation module.

[0091] The image processing module uses an image acquisition device to obtain the currently required image data, and obtains the corresponding coordinate values ​​after processing these image data, and inputs these coordinate values ​​into the displacement index calculation module; the displacement index calculation module calculates the obtained coordinate values ​​to amplify the corresponding influence of the current displacement, and outputs the data to the bridge displacement coefficient calculation module; the bridge displacement coefficient calculation module calculates the conditions on different observation surfaces, and outputs the data to the compensation coefficient calculation module; the compensation coefficient calculation module calculates the relative compensation coefficient that needs to be set at this time, and outputs the data to the comprehensive displacement calculation module; the comprehensive displacement calculation module completes the identification of the overall displacement situation, and outputs the corresponding results to the external terminal.

[0092] The image processing module is used to obtain the image data of the bridge structure and convert the image data into three-dimensional point cloud data. The image processing technology is used to extract the coordinate values ​​of the corresponding positions on the bridge structure; basic data support is provided to ensure the accuracy of subsequent displacement calculations.

[0093] The displacement index calculation module is used to divide the obtained coordinate values ​​into multiple observation surfaces and calculate the displacement index of the bridge structure on each observation surface; quantify the displacement of the bridge structure on different observation surfaces and provide basic data for subsequent comprehensive displacement calculations.

[0094] The bridge displacement coefficient calculation module is used to obtain the target position between each observation surface, verify the angle, vertical direction, horizontal direction and the deflection point of image overlap, calculate the matching coefficient, and obtain the bridge displacement coefficient in combination with the displacement index; identify the relative displacement of the bridge structure between different observation surfaces and improve the accuracy of displacement measurement.

[0095] The compensation coefficient calculation module is used to measure the displacement of the bridge structure when multiple observation surfaces are combined, extract the displacement mutation, and determine the relative compensation coefficient; identify the abnormal displacement of the bridge structure, and correct it through the compensation coefficient to improve the reliability of displacement measurement.

[0096] The comprehensive displacement calculation module is used to comprehensively sum the values ​​of the bridge structure at different observation surfaces and angles according to the relative compensation coefficient and the bridge displacement coefficient, determine the direction and size of the comprehensive displacement, and obtain the final displacement measurement results, providing an important basis for the monitoring and maintenance of the bridge structure.

[0097] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention and they are still covered by the protection scope of the present invention.

Claims

1. A displacement distance measurement method based on machine vision, characterized in that: include: S1, acquiring image data of a bridge structure, constructing the image data into three-dimensional point cloud data, and obtaining coordinate values ​​of corresponding positions on the bridge structure according to the three-dimensional point cloud data; S2, dividing the obtained coordinate values ​​into multiple observation planes, and calculating the displacement index of the bridge structure on each observation plane; S3, obtaining the target position between each observation surface, verifying the angle between each observation surface, the vertical direction, the horizontal direction and the deflection point of the image overlap, calculating the matching coefficient between each deflection point, combining the matching coefficient with the actual distribution position of the bridge structure, introducing the displacement index, and obtaining the bridge displacement coefficient on each observation surface; S4, measuring the displacement of each bridge structure when the bridge structure is combined on multiple observation surfaces, extracting the displacement mutation on each bridge structure, and determining the relative compensation coefficient between the bridge structures; S5, based on the obtained relative compensation coefficient and bridge displacement coefficient, the values ​​of the bridge structure at different observation planes and angles are comprehensively summed to determine the direction and magnitude of the comprehensive displacement generated by the current bridge structure.

2. The method for measuring displacement distance based on machine vision according to claim 1, characterized in that: Step S2 includes: S21, constructing a time series of the current observation surface, extracting feature points from the current observation surface, matching the feature points obtained at different times using feature descriptors, and establishing a corresponding relationship between the feature points; S22, calculating the displacement value of the feature point on the current observation surface according to the corresponding relationship of the feature points; S23, searching for common feature points between adjacent observation surfaces to connect them, and determining the relative position relationship between the adjacent observation surfaces; S24, calculating the displacement influence coefficient and the displacement correlation coefficient of the displacement value between the adjacent observation surfaces according to the relative position relationship between the adjacent observation surfaces; S25, obtaining the displacement index on each observation surface according to the displacement influence coefficient and the displacement correlation coefficient.

3. The method for measuring displacement distance based on machine vision according to claim 2, characterized in that: The displacement correlation coefficient can be expressed as follows: obtaining the difference in displacement values ​​between adjacent observation surfaces, constructing a difference sequence, and calculating the displacement correlation coefficient according to the difference sequence; ； in, represents the displacement correlation coefficient, represents the maximum value of the difference sequence, represents the minimum value of the difference sequence, Represents the difference in displacement value corresponding to the i-th element in the difference sequence, represents the adjustment factor, Indicates the length of the difference sequence, the length of the difference sequence is 1 to n; The displacement influence coefficient is expressed as: ;in, represents the displacement influence coefficient, represents the weight of the i-th element in the difference sequence, Represents the sum of the displacement value differences in the difference sequence.

4. The method for measuring displacement distance based on machine vision according to claim 1, characterized in that: Step S3 includes: S31, setting a target position on each observation surface, and calculating the angle between each observation surface according to the target position; S32, identifying the relative position relationship between the observation planes according to the angles between the observation planes, and obtaining the first deflection points existing in the horizontal direction and the vertical direction; S33, performing an overlapping process on each observation surface, and identifying a second deflection point after the image overlaps; S34, identifying the first deflection point and the second deflection point in sequence, and calculating the matching coefficient between the deflection points; S35, calculating the cosine similarity of the first deflection point and the second deflection point, and obtaining a matching coefficient according to the cosine similarity of the first deflection point and the second deflection point.

5. The method for measuring displacement distance based on machine vision according to claim 4, characterized in that: Step S32 includes: S321, obtaining position points at horizontal relative positions and vertical relative positions on each observation plane, and verifying the distance between each position point and the supporting point of the adjacent bridge structure; S322, according to the distance between each position point and the supporting point of the adjacent bridge structure, obtain the relative displacement of each position point relative to the supporting point at different times, and obtain the displacement distribution interval and displacement distribution probability corresponding to the relative displacement; S323, obtaining the distance between each position point and the center point of the support point, and calculating the joint probability value of each position point with respect to the distance between the position point and the center point of the support point and the relative displacement ratio; S324, after normalizing the relative displacement, the displacement distribution probability and the joint probability value of the position point, the distribution coefficient of the position point is calculated, and the position point whose distribution coefficient is greater than a preset threshold is used as the first deflection point to be output.

6. The method for measuring displacement distance based on machine vision according to claim 5, characterized in that: The second deflection point is obtained by obtaining the number of overlaps of the position points of each observation surface, clustering the current position points according to the number of overlaps, using the relative displacement ratio between the position point and the support point as the index value when clustering each position point, and using the position point corresponding to the largest cluster center after clustering as the output second deflection point.

7. The method for measuring displacement distance based on machine vision according to claim 4, characterized in that: The matching coefficient can be expressed as follows: the first deflection point and the second deflection point corresponding to the current matching coefficient are identified, and a matching threshold is set according to the position of the bridge structure. When the matching coefficient is greater than the matching threshold, the matching coefficient is adjusted to obtain the adjusted matching coefficient, and the adjusted matching coefficient is output: ； in, represents the matching coefficient between the jth first deflection point and the kth second deflection point, represents the adjusted matching coefficient of the jth first deflection point and the kth second deflection point, represents the exponential decay function of the jth first deflection point and the kth second deflection point, represents the matching threshold, represents the maximum value of the matching coefficient, Indicates the minimum value of the matching coefficient.

8. The method for measuring displacement distance based on machine vision according to claim 7, characterized in that: The bridge displacement coefficient is expressed as: ； in, represents the bridge displacement coefficient, represents the displacement index of the jth first offset point, represents the number of first deflection points, represents the number of the second deflection points, j ranges from 1 to m, and k ranges from 1 to K; represents the weight coefficient of the displacement index of the jth first offset point, The weight coefficient representing the matching coefficient of the j-th first offset point.

9. The method for measuring displacement distance based on machine vision according to claim 1, characterized in that: Step S5 includes: S51, obtaining the original displacement value of the bridge structure, the relative displacement of the bridge structure in the current observation plane relative to other observation planes, the relative compensation coefficient and the bridge displacement coefficient; S52, adjusting the displacement value of the current observation surface according to the original displacement value of the bridge structure, the relative displacement of the bridge structure in the current observation surface relative to other observation surfaces, and the relative compensation coefficient, and setting the displacement value of the observation surface; S53, combining the observed displacement value with the bridge displacement coefficient to obtain a comprehensive displacement vector, and using the magnitude and direction of the comprehensive displacement vector as the direction and magnitude of the comprehensive displacement generated by the current bridge structure.

10. A displacement distance measuring device based on machine vision, using a displacement distance measuring method based on machine vision as claimed in any one of claims 1 to 9, characterized in that: include: An image processing module is used to obtain image data of the bridge structure, convert the image data into three-dimensional point cloud data, and extract the coordinate values ​​of corresponding positions on the bridge structure using image processing technology; A displacement index calculation module is used to divide the obtained coordinate values ​​into multiple observation planes and calculate the displacement index of the bridge structure on each observation plane; The bridge displacement coefficient calculation module is used to obtain the target position between each observation surface, verify the angle, vertical direction, horizontal direction and the deflection point of image overlap, calculate the matching coefficient, and obtain the bridge displacement coefficient in combination with the displacement index; The compensation coefficient calculation module is used to measure the displacement of the bridge structure when multiple observation surfaces are combined, extract the displacement mutation, and determine the relative compensation coefficient; The comprehensive displacement calculation module is used to comprehensively sum the values ​​of the bridge structure at different observation surfaces and angles based on the relative compensation coefficient and the bridge displacement coefficient to determine the direction and size of the comprehensive displacement.

Images