System and method for measuring relative altitude difference based on machine vision and spherical target

Abstract

The invention discloses a system and a method for measuring a relative height difference based on machine vision and a spherical target, relates to the technical field of erection construction of a main cable of a suspension bridge, and solves the problems that an existing measurement technical means is greatly influenced by environmental conditions, time and labor are consumed in specific implementation, the precision is difficult to guarantee, and real-time data acquisition is difficult. The system comprises a spherical target, an image acquisition module, a measurement service platform and a data storage system, the spherical target is used for positioning the position of a measured cable strand, the image acquisition module transmits a cable strand spherical target image in real time in a wireless or wired mode, the measurement service platform carries out image processing on the cable strand spherical target image, and the data storage system stores the processed cable strand spherical target image. Calculating the space coordinates of the spherical target, and calculating the relative sag of the cable strand; the data storage system is used for storing the acquired measurement data and various parameters; according to the invention, the working intensity of field constructors is reduced, the personal error of manual measurement is eliminated, and the efficiency and precision of cable strand adjustment and measurement are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of main cable erection construction of suspension bridges, in particular to a system and method for measuring relative height difference based on machine vision and spherical target. BACKGROUND

[0002] The construction precision of the main cable of a suspension bridge has a decisive influence on the structural linear and stress of the whole bridge, and the traditional main cable strand erection of a suspension bridge adopts the "sag adjustment method". The sag adjustment is generally carried out at night when the temperature is stable, the wind speed is small, and it is 3 hours after the rain, which requires a large amount of manpower and time cost. In addition, for a long-span suspension bridge under complex environmental and climate conditions, the entire space-time temperature field and wind field conditions are difficult to meet, and due to the large span, even if the wind speed meets the requirement of 12 m / s, the swing displacement of the strand is also large, and the existing measuring equipment is difficult to meet the requirements.

[0003] Using the existing measurement technology to measure the linearity in the process of main cable erection construction of a suspension bridge has strict requirements on environmental conditions, and there are problems of huge cost of manpower and time, and inevitable errors in manual measurement, and it is difficult to realize real-time acquisition of strand linear data.

[0004] Based on the above problems, the existing technology proposes new measurement means based on laser array, light target and machine vision, but it still cannot solve the technical problem of poor measurement effect caused by the center offset of the strand when the strand rotates and swings.

[0005] For example, Chinese patent "Suspension bridge main cable strand relative height difference measurement method" (Patent application number: CN202410912323.1, publication number: CN118999472A). This patent combines light target positioning with image processing to avoid the problem of huge cost of manpower and time in manual measurement, but this method still has the following defects: the light target positioning can only be positioned on the surface of the strand, and cannot realize the true center positioning of the strand, which leads to the inability to effectively solve the eccentricity problem, and only involves the position calculation of one point of the light target center, and the calculation accuracy cannot be guaranteed; and because the image acquisition accuracy greatly affects the measurement effect, and the particularity of the light target requires to obtain the front image of the light target, therefore the installation position and the installation quantity of the image acquisition equipment may be complex, leading to high application cost, which is not conducive to popularization and use. SUMMARY

[0006] In order to solve the problems existing in the above-mentioned prior art, the present application provides a system and method for measuring relative height difference based on machine vision and spherical target, which solves the technical problems that the existing measurement technology cannot truly solve the eccentricity problem in the strand measurement process, resulting in low measurement precision and high measurement cost.

[0007] The system for measuring relative height difference based on machine vision and spherical target comprises a spherical target, an image acquisition module, a measurement service platform and a data storage system. The spherical target is used for positioning the center point of the longitudinal section of the measured cable. The image acquisition module acquires images of the spherical target corresponding to the measured cable in real time and transmits the images. The measurement service platform processes the images of the spherical target of the cable and calculates the spatial coordinates of the center of the spherical target as the spatial coordinates of the center point of the longitudinal section of the measured cable to calculate the relative sag of the measured cable. The data storage system stores the collected measurement data, parameters and calculation results.

[0008] Further, the center of the spherical target is the center of the cable, and the spherical surface of the spherical target has obvious black and white interlaced textures in the form of square or equilateral triangle.

[0009] The method for measuring relative height difference based on machine vision and spherical target comprises the following steps.

[0010] Step 1: A spherical target is installed on the measurement section of at least one measured cable, so that the center of the spherical target coincides with the center point of the longitudinal section of the measured cable.

[0011] Step 2: An image acquisition module acquires at least one image of the spherical target and transmits the image to a measurement service platform.

[0012] Step 3: The measurement service platform extracts and matches features in the same image of the spherical target to obtain the image plane coordinates of the surface feature points of the spherical target. The surface feature points of the spherical target are at least four.

[0013] Step 4: The three-dimensional coordinate information of the surface feature points of the spherical target is obtained by using the image plane coordinates of the surface feature points of the spherical target, combining the position information and parameter information of the image acquisition module and performing space intersection calculation.

[0014] Step 5: The center coordinates of the spherical target are fitted based on the three-dimensional coordinate information of the surface feature points of the spherical target, the center coordinates of the measured cable corresponding to the spherical target are obtained, and the relative height difference of the measured cable is obtained based on the center coordinates of the measured cable.

[0015] Further, step 3 comprises extracting features of multiple images of the spherical target by using a feature detection algorithm to obtain a plurality of surface feature points of the spherical target, and then matching the plurality of surface feature points by using a feature matching algorithm to obtain the image plane coordinates of the surface feature points of the spherical target.

[0016] Further, the position information of the image acquisition module in step 4 is obtained by placing a reference mark with known spatial coordinates at the end of the spherical target and performing space resection calculation based on the image plane coordinates and the spatial coordinates of the reference mark.

[0017] Further, the space intersection calculation in step 4 is implemented by using bundle adjustment.

[0018] Further, the fitting of the spherical center coordinates of the spherical center target in step 5 includes:

[0019] Step 5.1: data preparation, given the three-dimensional coordinates of n spherical target surface feature points: , wherein ;

[0020] Step 5.2: construct a linear equation set, each three-dimensional coordinate satisfies the spherical surface equation:

[0021]

[0022] Expand and arrange to get linear form:

[0023]

[0024] Where (a, b, c) is the spherical center coordinates of the spherical center target, r is the radius, , D is an intermediate variable;

[0025] Step 5.3: construct a matrix and a vector, including: construct a coefficient matrix , each row ; construct the right side vector , each row ;

[0026] Step 5.4: solve the linear least squares problem, solve the equation set , wherein , using the normal equation:

[0027]

[0028] Solve by numerical method to get ;

[0029] Step 5.5: calculate the radius of the spherical target, the expression is as follows:

[0030]

[0031] Need to verify To ensure real solution.

[0032] Further, the measured cable in step 5 includes general cable and reference cable, and the reference sag of the general cable is obtained based on the spherical center coordinates of the general cable and the reference cable. The relative sag between the general cables can be obtained by joint calculation of the reference sags of the plurality of general cables.

[0033] Further, the surface feature points include points where the black and white textures of the spherical target surface meet.

[0034] The beneficial effects of this invention include:

[0035] To better overcome the eccentricity problem, this invention places a spherical target with a specific texture on the strand being tested, so that the center of the spherical target coincides with the center point of the longitudinal section of the strand being tested, thus avoiding the deviation between the calculated center position and the actual center position of the strand, and avoiding the need to assist in solving the eccentricity problem from the positioning level.

[0036] From a specific computational perspective, this invention collects several surface feature points on the surface of a spherical target, and combines machine vision to perform spatial calculations to obtain the spatial coordinates of multiple surface feature points. Then, it fits the coordinates of the center of the spherical target as the spatial coordinates of the strand center position. The least squares method is used to fit and calculate the spatial coordinates of multiple surface feature points to ensure the accuracy of the fitted spatial coordinates of the strand center position. Compared with calculating only one point on the optical target, the accuracy is higher, and the accuracy can be theoretically evaluated, making the measurement accuracy more controllable.

[0037] Finally, when performing surface feature point detection and extraction on a spherical target, this invention does not restrict the specific location of the extracted surface feature points. It only needs to satisfy the number of feature points required for fitting the sphere's center coordinates. Therefore, the image acquisition module of this invention only needs to ensure that it acquires an image of the spherical target of the tested cable strand. Unlike acquiring an optical target image, which requires capturing the front of the target to ensure the subsequent calculation of the target's center point, this invention has lower image acquisition requirements and does not strictly limit the installation location or number of image acquisition modules, making the overall method easier to use and promote. Attached Figure Description

[0038] Figure 1 This is an architecture diagram of a system based on machine vision and spherical target for measuring relative height difference, which is involved in an embodiment of this application.

[0039] Figure 2 This is a schematic diagram of the spherical target involved in the embodiments of this application.

[0040] Figure 3 This is a flowchart of a method for measuring relative height difference based on machine vision and a spherical target, which is involved in an embodiment of this application.

[0041] Figure 4 This is a schematic diagram of the extracted spherical target surface feature points involved in the embodiments of this application.

[0042] Figure 5 This is a schematic diagram of the installation of the camera acquisition module involved in the embodiments of this application, wherein (a) is a schematic diagram corresponding to the installation on the top of the gantry, and (b) is a schematic diagram corresponding to the installation on the side column of the gantry. Detailed Implementation

[0043] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0044] Embodiment 1

[0045] A system for measuring relative height difference based on machine vision and spherical target, as shown in Figure 1 , comprises a spherical target, an image acquisition module, a measurement service platform and a data storage system. The spherical target is used for positioning the center point of the longitudinal section of the measured strand. The image acquisition module acquires the image of the spherical target corresponding to the measured strand in real time and transmits it. The measurement service platform processes the image of the spherical target of the strand, calculates the spatial coordinates of the center of the spherical target as the spatial coordinates of the center point of the longitudinal section of the measured strand, and calculates the relative sag of the measured strand. The data storage system saves the measured data, parameters and calculation results acquired.

[0046] The image acquisition module comprises a binocular or multi-camera array camera module, which transmits the image stream of the spherical target of the strand back to the measurement service platform in a wireless or wired manner. The specific installation position is shown in Figure 5 , wherein (a) corresponds to the installation diagram of the top of the gantry, (b) corresponds to the installation diagram of the side column of the gantry. The measured spherical target and the reference spherical target can be installed on the top of the gantry or the side column of the gantry, as long as they can be kept in the image.

[0047] The measurement service platform and the data storage system have fixed IP addresses and are connected to the network in a limited or wireless manner.

[0048] The entire system adopts B / S mode layering for data transmission, control and analysis calculation in a web site manner. A set of web service website can dynamically expand or manage multiple front-end projects.

[0049] As shown in Figure 2 , the center of the spherical target is the center of the strand, and the spherical surface of the spherical target has obvious square or equilateral triangle black and white texture.

[0050] Embodiment 2

[0051] A method for measuring relative height difference based on machine vision and spherical target, as shown in Figure 3 , comprises:

[0052] Step 1: Install a spherical target on at least one measuring section of the strand being measured, such that the center of the spherical target coincides with the center point of the longitudinal section of the strand being measured;

[0053] Step 2: The camera acquisition module acquires at least one image of the spherical target and transmits it to the measurement service platform;

[0054] Step 3: The measurement service platform extracts and matches features from the same spherical target image to obtain the image plane coordinates of the surface feature points of the spherical target. There are at least 4 surface feature points of the spherical target.

[0055] Step 4: Using the image plane coordinates of the feature points on the surface of the spherical target, combined with the position and parameter information of the camera acquisition module, spatial intersection calculation is performed to obtain the three-dimensional coordinate information of the feature points on the surface of the spherical target;

[0056] Step 5: Fit the center coordinates of the spherical target based on the three-dimensional coordinate information of the feature points on the surface of the spherical target to obtain the center coordinates of the tested strand corresponding to the spherical target, and obtain the relative height difference of the tested strand based on the center coordinates of the tested strand.

[0057] The tested strands include general strands and reference strands. The reference sag of the general strand is calculated based on the sphere center coordinates of the general strand and the reference strand. The relative sag between the general strands can be obtained by jointly calculating the reference sag of multiple general strands.

[0058] In another embodiment, step 3 includes: extracting features from multiple spherical target images using a feature detection algorithm to obtain several surface feature points of the spherical target. Specifically, these surface feature points include points where the black and white textures of the spherical target surface meet, such as... Figure 4 As shown, the image plane coordinates of the surface feature points of the spherical target are then obtained by matching several surface feature points using a feature matching algorithm.

[0059] In another embodiment, step 4 includes:

[0060] The collinearity equation, combined with adjustment methods, is used to calculate parameters and correlate image feature coordinates with object-space 3D coordinates. The collinearity equation describes the geometric relationship of collinearity between object-space points, principal image points, and image feature points, as follows:

[0061]

[0062] In the formula, Image point coordinates (unit: pixels or millimeters); Like the principal point coordinates (internal orientation elements); Camera focal length (interior orientation element); : Three-dimensional coordinates of the object point; : The coordinates of the photography center in the object coordinate system (exterior orientation line element); The elements of the rotation matrix R (exterior orientation elements, usually consisting of three rotation angles) (Calculated).

[0063] Object point refers to a surface feature point on a sphere. Principal point refers to the projection point of the camera focus onto the image plane (i.e., the image), which is generally the center point of the image. Image feature point is the image point (i.e., image point) projected onto the image from the spherical feature point.

[0064] The expression for the rotation matrix R is as follows:

[0065]

[0066] In the formula, The expression for the rotation matrix about the X-axis. The expression for the rotation matrix about the Y-axis. This is the expression for the rotation matrix around the Z-axis.

[0067] The spatial intersection calculation includes spatial forward intersection: after obtaining the image plane coordinates of corresponding points in each image, and in conjunction with the relative position of the camera, the three-dimensional spatial coordinates (X, Y, Z) of the feature points on the surface of the spherical target are obtained by least squares adjustment using the linearized collinearity equations; for stereo image pairs, two collinearity equations are provided for each pair of image points, and the three-dimensional coordinates are solved simultaneously.

[0068]

[0069] Solve for (X,Y,Z) using linearization or direct least squares.

[0070] In another embodiment, the spatial intersection calculation includes spatial back intersection: by placing a reference marker at the end of a spherical target, it is not necessary to determine the spatial relative relationships between the cameras in the camera array in advance. The position information of the camera acquisition module can be directly calculated from the reference point. The calculation target is: given the coordinates of the object control point and the image point, calculate the exterior orientation elements of the camera. The solution steps are as follows: given the coordinates of the control points and the corresponding image point coordinates Construct error equations, providing two equations for each control point, and use the least squares method to iteratively solve for the corrections to the exterior orientation elements. :

[0071]

[0072] Update the exterior orientation elements and repeat the iteration until convergence.

[0073] In another embodiment, bundle adjustment involves minimizing reprojection error and jointly optimizing the interior and exterior orientation elements of all cameras and the 3D coordinates of the object point to achieve globally optimal 3D reconstruction and camera parameter estimation. The following is a detailed analysis of its computational model:

[0074] 1. Problem Definition

[0075] 1.1 Input: Image point observations from multiple images , indicating the first The three-dimensional point at the th t Projected coordinates on the image; initially estimated camera parameters (internal parameters) External reference , ) and 3D point coordinates .

[0076] 1.2 Output: Optimized camera parameters and 3D point coordinates, minimizing reprojection error.

[0077] 2. Mathematical Model

[0078] 2.1 Objective Function

[0079] The goal of bundle adjustment is to minimize the sum of squared residuals between all observed image points and their projected values:

[0080]

[0081] In the formula, For projection functions, project 3D points Projected to the Zhang's image; This is a robust kernel function used to suppress the effects of outliers.

[0082] 2.2 Projection Function The unfolding

[0083] Based on the collinearity equation, the projection process is as follows:

[0084]

[0085] After normalization, we get:

[0086]

[0087] in Rotation matrix elements, Translation vector The amount.

[0088] 2.3. Nonlinear Least Squares Problem

[0089] Transform the objective function into a nonlinear least squares form:

[0090]

[0091] For all parameters to be optimized (camera parameters + 3D point coordinates); For the first The residuals of each observation, i.e. .

[0092] In another embodiment, step 5, fitting the center coordinates of the target sphere, includes:

[0093] Step 5.1: Data preparation, given the three-dimensional coordinates of n feature points on the surface of a spherical target: ,in ;

[0094] Step 5.2: Construct a system of linear equations, where each three-dimensional point satisfies the equation of a sphere:

[0095]

[0096] Expanding and rearranging, we obtain the linear form:

[0097]

[0098] in D is an intermediate variable;

[0099] Step 5.3: Construct matrices and vectors, including: constructing the coefficient matrix. Each line Construct the right-hand vector Each line ;

[0100] Step 5.4: Solve the linear least squares problem by solving the system of equations. ,in Using the normal equation:

[0101]

[0102] The solution was obtained through numerical methods. ;

[0103] Step 5.5: Calculate the radius of the spherical target to verify the accuracy of the sphere center coordinate calculation. The expression is as follows:

[0104]

[0105] Verification required To ensure a real solution.

[0106] To verify the accuracy of the relative height difference measurement of the cable strands proposed in this embodiment, nonlinear optimization methods were used for verification. In the simulation calculation, the camera focal length was set to 2000 pixels, the pixel size to 4 micrometers, and the radial distortion coefficients k1, k2, and k3 were set to (-0.15, 0.03, 0.001), and the tangential distortion coefficients p1 and p2 were set to (0.001, -0.001), respectively, representing the simulated camera's internal orientation parameters. The baseline was set to 50 cm, the shooting distance to 1-3 m, and the image plane observation error level to 0.3 pixels. Two-view or multi-view cameras were used. At the beginning of the calculation, the actual 3D coordinate points (such as the origin) were first projected to obtain the image plane coordinates (u, v) of different cameras. An observation error with a noise level of 0.3 pixels was then added to these coordinates to obtain image point observation data with errors. Then, the three-dimensional coordinates of the measured points after the final iterative convergence optimization were calculated using bundle adjustment, while simultaneously calculating the reprojection error. Simulation calculations show that the maximum error occurs in the depth direction, achieving an accuracy better than 2mm, while the lateral accuracy is better than 0.5mm.

[0107] To verify the error in fitting the center of the three-dimensional sphere, a simulation calculation was also performed using a nonlinear optimization method. The radius of the three-dimensional sphere was set to 7cm, the center of the sphere was (0.0,0.0,0.0), the error level of the measuring points on the sphere was 2mm, and 10 measuring points were randomly generated for nonlinear fitting calculation. The maximum error of the center coordinates was better than 1.5mm, and the maximum error of the radius of the sphere was better than 0.15mm.

[0108] Simulation calculations demonstrate that, in practical applications, the accuracy of target point acquisition can be further improved by increasing the baseline length, the number of cameras, and the camera focal length, thereby achieving a sphere center coordinate fitting accuracy better than 1mm.

[0109] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A system for measuring relative height difference using machine vision and a spherical target, characterized in that, include: The system comprises a spherical target, an image acquisition module, a measurement service platform, and a data storage system. The spherical target is used to locate the center point of the longitudinal section of the cable strand being measured. The image acquisition module acquires and transmits images of the cable strand corresponding to the spherical target in real time. The measurement service platform performs image processing on the spherical target images of the cable strand, calculates the spatial coordinates of the center of the spherical target as the spatial coordinates of the center point of the longitudinal section of the cable strand being measured, and calculates the relative sag of the cable strand being measured. The data storage system saves the acquired measurement data, parameters, and calculation results.

2. The system for measuring relative height difference based on machine vision and a spherical target according to claim 1, characterized in that, The center of the spherical target is the center of the strand, and the surface of the spherical target has obvious black and white textures of squares or equilateral triangles.

3. A method for measuring relative height difference based on machine vision and a spherical target, characterized in that, The system, based on machine vision and spherical target measurement of relative height difference, as described in any one of claims 1-2, includes: Step 1: Install a spherical target on at least one measuring section of the strand being measured, such that the center of the spherical target coincides with the center point of the longitudinal section of the strand being measured; Step 2: The camera acquisition module acquires at least one image of the spherical target and transmits it to the measurement service platform; Step 3: The measurement service platform extracts and matches features from the same spherical target image to obtain the image plane coordinates of the surface feature points of the spherical target. There are at least 4 surface feature points of the spherical target. Step 4: Using the image plane coordinates of the feature points on the surface of the spherical target, combined with the position and parameter information of the camera acquisition module, spatial intersection calculation is performed to obtain the three-dimensional coordinate information of the feature points on the surface of the spherical target; Step 5: Fit the center coordinates of the spherical target based on the three-dimensional coordinate information of the feature points on the surface of the spherical target to obtain the center coordinates of the tested strand corresponding to the spherical target, and obtain the relative height difference of the tested strand based on the center coordinates of the tested strand.

4. The method for measuring relative height difference based on machine vision and a spherical target according to claim 3, characterized in that, Step 3 includes: using a feature detection algorithm to extract features from multiple spherical target images to obtain several surface feature points of the spherical target, and then using a feature matching algorithm to match the several surface feature points to obtain the image plane coordinates of the surface feature points of the spherical target.

5. The method for measuring relative height difference based on machine vision and a spherical target according to claim 3, characterized in that, The acquisition of position information of the camera acquisition module in step 4 includes: placing a reference marker with known spatial coordinates at the end of the spherical target, and then performing spatial resection calculation based on the image plane coordinates and spatial coordinates of the reference marker.

6. The method for measuring relative height difference based on machine vision and a spherical target according to claim 3, characterized in that, The spatial intersection calculation described in step 4 is achieved using bundle adjustment.

7. The method for measuring relative height difference based on machine vision and a spherical target according to claim 3, characterized in that, The center coordinates of the fitted spherical target mentioned in step 5 include: Step 5.1: Data preparation, given the three-dimensional coordinates of n feature points on the surface of a spherical target: ,in ; Step 5.2: Construct a system of linear equations, where each three-dimensional coordinate satisfies the equation of a sphere: Expanding and rearranging, we obtain the linear form: Where (a,b,c) are the coordinates of the center of the sphere target, and r is the radius. D is an intermediate variable; Step 5.3: Construct matrices and vectors, including: constructing the coefficient matrix. Each line Construct the right-hand vector Each line ; Step 5.4: Solve the linear least squares problem by solving the system of equations. ,in Using the normal equation: The solution was obtained through numerical methods. ; Step 5.5: Calculate the radius, as shown in the following expression: Verification required To ensure a real solution.

8. The method for measuring relative height difference based on machine vision and a spherical target according to claim 3, characterized in that, The strands to be measured in step 5 include general strands and reference strands. The reference sag of the general strand is calculated based on the sphere center coordinates of the general strand and the reference strand. The relative sag between the general strands can be obtained by jointly calculating the reference sag of multiple general strands.

9. The method for measuring relative height difference based on machine vision and a spherical target according to claim 3, characterized in that, The surface feature points include the black and white points where the surface texture of the spherical target meets.

Images