Camera self-calibration measurement method and device

Abstract

The present application discloses a camera self-calibration measurement method and device, the camera self-calibration measurement device includes a measuring camera, a calibration camera and a self-balancing component, the measuring camera is arranged relative to the marker, and the camera self-calibration measurement method includes: the calibration camera collects the component image and sends it to the measuring camera, wherein the component image is an image containing the self-balancing component; the measuring camera collects and processes the component image to obtain the baseline and target line corresponding to the self-balancing component, and calculates the angle between the target line and the baseline, wherein the angle is the change angle of the measuring camera in the pitch direction; the measuring camera collects the marker image containing the marker, and performs angle correction on the marker image according to the angle to obtain the target data. The present application utilizes the gravity of the self-balancing component to realize the real-time measurement and compensation of the camera angle change caused by the shaking of the observation platform, thereby improving the accuracy of the camera angle correction in complex environments.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a camera self-calibration measurement method, a camera self-calibration measurement device, a self-calibration camera combination measurement device and a combination measurement method. Background Art

[0002] Structural deformation is one of the important indicators of structural health monitoring. Visual measurement technology is increasingly being used in structural deformation monitoring due to its advantages such as non-contact, high precision, and real-time monitoring. However, in the long-term monitoring process, the stability of the observation platform is one of the bottlenecks that limit its application. Due to the existence of the optical lever effect, a small angle change of the long-distance measurement camera will lead to a large error, resulting in measurement failure.

[0003] Existing methods for angle correction of long-distance measurement cameras based on unstable observation platforms include reference point method, image registration method and attitude sensor compensation method such as gyroscope or IMU. However, the existing camera angle correction methods have low accuracy under complex environmental conditions.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present application is to provide a camera self-calibration measurement method, a camera self-calibration measurement device, a self-calibration camera combination measurement device and a combination measurement method, aiming to solve the problem in the prior art that a long-distance measurement camera based on an unstable observation platform uses a reference point method, an image alignment method and a sensor compensation method for angle correction, and the accuracy of the camera angle correction is low.

[0006] A first aspect of an embodiment of the present application provides a camera self-calibration measurement method based on a camera self-calibration measurement device, wherein the camera self-calibration measurement device includes a measuring camera, a calibration camera and a self-balancing component, and the measuring camera is arranged relative to a marker. The camera self-calibration measurement method includes: the measuring camera processes the component image to obtain a baseline and a target line corresponding to the self-balancing component, and calculates an angle between the target line and the baseline, wherein the angle is a change angle of the measuring camera in a pitch direction; the measuring camera collects a marker image containing the marker, performs angle correction on the marker image according to the angle, and obtains target data.

[0007] In a possible implementation, the optical axes of the measuring camera and the calibration camera are L-shaped, and the self-balancing component is a plumb line; the processing of the component image to obtain a baseline and a target line corresponding to the self-balancing component specifically includes: preprocessing the component image to obtain a preprocessed component image; performing straight line extraction on the preprocessed component image to obtain a plurality of straight line equations of the plumb line in the image plane; using a straight line equation corresponding to a first frame image among the plurality of straight line equations as a baseline, and using a straight line equation corresponding to a current frame image among the plurality of straight line equations as a target line.

[0008] In a possible implementation, the target data is a compensated target displacement; and performing angle correction on the marker image according to the included angle to obtain the target data specifically includes: calculating a virtual displacement of the measuring camera according to the angle; calculating a target displacement corresponding to the marker in the marker image; and calculating a difference between the target displacement and the virtual displacement to obtain the compensated target displacement.

[0009] In a possible implementation, the angle is calculated as follows:

[0010] ;

[0011] in, is the angle, is the slope corresponding to the baseline, is the slope corresponding to the target line;

[0012] The calculation formula of the virtual displacement is:

[0013] ;

[0014] in, is the virtual displacement, The distance from the measuring camera to the marker;

[0015] The compensation formula of the target displacement after compensation is:

[0016] ;

[0017] in, is the target displacement after compensation, is the target displacement.

[0018] A second aspect of an embodiment of the present application further provides a camera self-calibration measurement device for implementing the camera self-calibration measurement method as described in any one of the above schemes, wherein the camera self-calibration measurement device comprises a measuring camera, a calibration camera and a self-balancing member; the measuring camera is arranged opposite to the marker, the calibration camera is fixedly connected to the measuring camera, and the angle between the optical axes of the measuring camera and the calibration camera is fixed, and the self-balancing member is connected to the measuring camera and the calibration camera, respectively; the calibration camera is used to capture a component image of the self-balancing member and send it to the measuring camera, and the measuring camera is used to capture a mark image of the marker, and perform angle correction on the mark image after determining the angle based on the component image.

[0019] Optionally, the camera self-calibration measuring device, wherein the optical axes of the measuring camera and the calibration camera are L-shaped; the self-balancing member is a plumb line, and the weight of the plumb line passes vertically downward through the plumb line; or the self-balancing member is a U-shaped tube, and the U-shaped tube contains liquid, and the surface of the liquid is parallel to the horizontal plane.

[0020] Optionally, the camera self-calibration measuring device further comprises a housing and a support; the measuring camera and the calibration camera are arranged in the housing, the support is arranged in the housing, and the support is abutted against the calibration camera.

[0021] Optionally, in the camera self-calibration measuring device, the inner wall of the shell has a friction surface, and if the self-balancing member is a plumb line, the plumb line is frictionally connected with the friction surface when it shakes.

[0022] A third aspect of an embodiment of the present application further provides a self-calibration camera combination measuring device, wherein the self-calibration camera combination measuring device comprises a plurality of camera self-calibration measuring devices as described in any one of the above-mentioned schemes, and the plurality of camera self-calibration measuring devices are arranged in sequence; the self-calibration camera combination measuring device also comprises a plurality of markers, each of which is arranged on a corresponding camera self-calibration measuring device; the markers arranged on a preceding camera self-calibration measuring device are opposite to those on a succeeding camera self-calibration measuring device, so that the preceding camera self-calibration measuring device monitors the settlement of the succeeding camera self-calibration measuring device itself.

[0023] The fourth aspect of the embodiment of the present application further provides a combined measurement method based on a self-calibration camera combined measurement device, wherein the combined measurement method comprises: for a previous camera self-calibration measurement device, the previous calibration camera captures a previous component image and sends it to the previous measurement camera, wherein the previous component image is an image containing a previous self-balancing component; the previous measurement camera processes the previous component image to obtain a previous reference line and a previous target line corresponding to the previous self-balancing component, and calculates a previous angle between the previous target line and the previous reference line, wherein the previous angle is a change angle of the previous measurement camera in the pitch direction; the previous measurement camera captures a marker image containing a marker, and performs angle correction on the marker image according to the previous angle to obtain a marker data; Wherein, the marker is arranged on a camera self-calibration measuring device; for the camera self-calibration measuring device, a calibration camera collects a component image and sends it to a measuring camera, wherein the component image is an image containing a self-balancing component; a measuring camera processes the component image to obtain a reference line and a target line corresponding to the self-balancing component, and calculates an angle between the target line and the reference line, wherein the angle is a change angle of the measuring camera in the pitch direction; the measuring camera collects a mark image containing a marker, performs angle correction on the mark image according to the angle, and obtains a mark data; the self-calibration camera combined measuring device obtains the target data of the marker according to the marker data and the mark data.

[0024] Beneficial effects: The present application provides a camera self-calibration measurement method, a camera self-calibration measurement device and a networked measurement device. The present application connects a measuring camera and a calibration camera together, wherein the measuring camera monitors structural deformation and the calibration camera observes a self-balancing component. The gravity of the self-balancing component is utilized to achieve real-time measurement and compensation of camera angle changes caused by shaking of the observation platform, thereby improving the accuracy of camera angle correction in complex environments.

[0025] In addition to the technical problems solved by the present application, the technical features that constitute the technical solutions, and the beneficial effects brought about by the technical features of these technical solutions described above, other technical problems that can be solved by the camera self-calibration measurement method, camera self-calibration measurement device and networked measurement device provided by the present application, other technical features included in the technical solutions, and the beneficial effects brought about by these technical features will be further described in detail in the specific implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A schematic diagram of a preferred embodiment of the camera self-calibration measurement device of the present application;

[0028] Figure 2 A three-dimensional schematic diagram of a preferred embodiment of the camera self-calibration measurement device of the present application;

[0029] Figure 3 Another schematic diagram of a preferred embodiment of the camera self-calibration measurement device of the present application;

[0030] Figure 4 A schematic diagram of another embodiment of the camera self-calibration measurement device of the present application;

[0031] Figure 5 A three-dimensional schematic diagram of a preferred embodiment of the self-calibration camera combination measurement device of the present application;

[0032] Figure 6 It is a flow chart of a preferred embodiment of the camera self-calibration measurement method of the present application;

[0033] Figure 7 It is a flowchart of specific implementation steps of the entire execution process in a preferred embodiment of the camera self-calibration measurement device of the present application;

[0034] Figure 8 It is a flow chart of a preferred embodiment of the combined measurement method of the present application.

[0035] Description of reference numerals:

[0036] 1. Calibration camera; 2. Plumb line; 3. Measuring camera; 4. Housing; 5. Support; 6. Rough surface; 7. Marker.

[0037] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and effect of this application clearer and more specific, the technical solution in the embodiment of this application will be clearly and completely described below in conjunction with the drawings in the embodiment of this application. Obviously, the described embodiment is only a part of the embodiment of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] In the related art, the method for correcting the angle of a long-distance measurement camera based on an unstable observation platform is as follows:

[0040] (1) Reference point method: Place reference points of known positions within the camera's field of view, such as calibration plates or specific patterns. The camera's angle change is estimated by detecting the position change of the reference point and making corresponding corrections. This method is simple and easy to implement, but requires the reference point to remain stable and clear within the field of view. However, in the field, the reference point may be affected by factors such as weather and lighting changes, resulting in recognition difficulties or positioning errors. For example, rain and snow may cover the reference point, and direct sunlight or shadows may also interfere with the detection of the reference point. In addition, the position of the reference point may undergo physical changes (such as movement, wear, etc.), further reducing the reliability of this method.

[0041] (2) Image registration method: Use image registration technology to compare the current image with the reference image, and calculate the camera angle offset by matching feature points. The angle offset calculated by registration can be used to correct the current image. Commonly used image registration methods include feature point detection and matching algorithms such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF). Image registration relies on high-quality feature point matching. In complex outdoor environments, due to changes in lighting, weather factors, and increased background complexity, the stability of feature points may decrease, resulting in increased registration errors. In addition, in environments with a large number of dynamic changes, the probability of misregistration is high.

[0042] (3) Compensation method using attitude sensors such as gyroscopes or IMUs (Inertial Measurement Unit): Integrate high-precision attitude sensors such as gyroscopes or IMUs in the camera device to monitor the angle changes of the camera in real time. Dynamically compensate for the angle changes of the camera based on the output data of the sensor. This method can respond to angle changes in real time and is suitable for scenarios that require high-precision monitoring. However, in the wild, long-term exposure to harsh conditions (such as high temperature, low temperature, and humidity changes) may cause sensor drift or decreased accuracy. In addition, the installation stability of the sensor and the noise of the sensor itself will also affect the accuracy of angle measurement.

[0043] In view of the problem that the accuracy of the camera angle correction is low in the long-distance measurement camera based on an unstable observation platform in the above-mentioned related technologies, the present application provides a camera self-calibration measurement method, a camera self-calibration measurement device and a networked measurement device. In this method, by fixing the measurement camera and the calibration camera together, the measurement camera monitors the structural deformation, the calibration camera observes the self-balancing component, and the gravity of the self-balancing component is used to achieve real-time measurement and compensation of the camera angle change caused by the shaking of the observation platform, thereby improving the accuracy of the camera angle correction in a complex environment. Thus, the technical problem that the accuracy of the camera angle correction is low in the long-distance measurement camera based on an unstable observation platform in the related technologies is solved.

[0044] That is, in response to the common shortcomings of existing camera angle correction methods such as low accuracy, poor anti-interference performance, and high requirements for the use environment, the present application can achieve high-precision real-time correction of camera angle changes based on an unstable observation platform in complex and harsh outdoor environments.

[0045] First, the following will be combined with the attached Figures 1 to 5 , the structure of the embodiment of the present application is introduced.

[0046] In the embodiment of the present application, the main camera (measurement camera), the auxiliary camera (calibration camera) and the plumb line are rigidly connected, the auxiliary camera tracks the plumb line to obtain the angle change of the camera, and the fixed connection constraint between the main and auxiliary cameras is used to perform angle compensation to achieve self-calibration measurement under an unstable observation platform. On this basis, in order to solve the problem of limited observation distance of a single camera, multiple camera self-calibration measurement devices can be combined to achieve full coverage observation of large structures.

[0047] The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0048] like Figure 1 and Figure 2 As shown, an embodiment of the present application provides a camera self-calibration measuring device, which includes a measuring camera 3 (main camera), a calibration camera 1 (auxiliary camera) and a self-balancing member; the measuring camera 3 is arranged opposite to the marker 7, the calibration camera 1 is fixedly connected to the measuring camera 3, and the optical axis angle between the measuring camera 3 and the calibration camera 1 is fixed, and the self-balancing member is respectively connected to the measuring camera 3 and the calibration camera 1; the calibration camera 1 is used to collect the component image of the self-balancing member and send it to the measuring camera, the measuring camera 3 is used to collect the mark image of the marker 7, and after determining the angle according to the component image, the mark image is angle-corrected, so that the camera self-calibration measuring device can be accurately monitored.

[0049] See also Figure 1 and Figure 2 In the perspective of the schematic diagram, the measuring camera 3 faces the right side, and the calibration camera 1 faces the front side. A bracket is provided at the connection between the measuring camera 3 and the calibration camera 1, and the end of the bracket is connected to a self-balancing component, which is a plumb line 2. Figure 1 The middle shadow is the position of the camera self-calibration measurement device before shaking. Figure 1 The real scene in the middle is the position of the camera self-calibration measurement device after relative shaking. That is to say, when the calibration camera 1 shakes under the influence of a complex environment, the calibration camera 1 rotates roughly along the optical axis direction, such as Figure 1 The arrow direction of the calibration camera 1 side close to the plumb line 2 (note that the calibration camera 1 does not shake left and right). The calibration camera 1 and the measurement camera 3 are fixed together. Therefore, the shaking of the calibration camera 1 will cause the measurement camera 3 to shake (or the shaking of the measurement camera 3 will cause the calibration camera 1 to shake). Figure 1 The arrow direction on the side of the measuring camera 3 shakes, which causes the angle of the measuring camera 3 in the pitch direction to change. The present application calculates the angle change and then eliminates the virtual displacement caused by the camera's own angle change in the settlement of the marker 7 (with a marker point) monitored by the measuring camera 3, so as to achieve the purpose of correcting the influence of the camera's own shaking on the measurement, improve the accuracy of the camera angle correction in a complex environment, and thus improve the accuracy of the measurement.

[0050] It is understandable that the measuring camera has a processor for image data processing, but it is not limited thereto, and the measuring camera and the processor may also be provided separately.

[0051] In one embodiment of the present application, the optical axes of the measuring camera 3 (main camera) and the calibration camera 1 (auxiliary camera) are L-shaped; the self-balancing member is a plumb line 2, and the weight of the plumb line 2 is vertically downward through the plumb line.

[0052] The present application adopts an L-shaped dual-camera structure and utilizes gravity to achieve self-balancing of the overall camera structure.

[0053] For details, see Figure 2 , connect the main and auxiliary cameras together in an L shape, and ensure that the main optical axes of the main and auxiliary cameras are as vertical as possible. The camera used to monitor structural deformation is called the main camera, and the camera used to observe the plumb line 2 for angle correction is called the auxiliary camera. Since the main and auxiliary cameras are strictly connected vertically, when the main camera changes its angle in the pitch direction, it can be considered that the auxiliary camera has undergone an angle change of the same magnitude in the roll direction. Although the plumb line 2 is rigidly connected to the main and auxiliary cameras, the direction of gravity is always vertically downward. Even if the camera shakes, the plumb line 2 can still remain vertically downward.

[0054] This application can improve measurement accuracy and reliability, reduce environmental interference through the principle of gravity self-balancing, and improve the accuracy of measurement results. It has high accuracy and improves the measurement accuracy of angle changes based on the stability of the gravity direction; it has strong anti-interference ability, reduces the impact of environmental factors on measurement results, and is adaptable to various complex environments.

[0055] In another embodiment of the present application, the optical axes of the measuring camera 3 and the calibration camera 1 are L-shaped; the self-balancing member is a U-shaped tube, the U-shaped tube is filled with liquid, and the surface of the liquid is parallel to the horizontal plane.

[0056] For details, see Figure 3 In addition to using the plumb line 2 to achieve gravity self-balancing, other devices may also be used to achieve similar functions, such as a U-shaped tube. A U-shaped tube is used to replace the plumb line 2 and is fixedly connected to the main and auxiliary cameras. After the camera moves, the U-shaped tube tilts, but the liquid columns at both ends of the U-shaped tube still maintain the same liquid level. At this time, the tilt angle of the U-shaped tube can be calculated by the liquid level position before and after the change, and the camera movement can be corrected after obtaining the angle.

[0057] In one embodiment of the present application, the camera self-calibration measuring device further includes a housing 4 and a support member 5; the measuring camera 3 and the calibration camera 1 are arranged in the housing 4, the support member 5 is arranged in the housing 4, and the support member 5 is abutted against the calibration camera 1.

[0058] For details, see Figure 2 The housing 4 is square, but not limited thereto, and an opening is provided on the housing 4 . The lens of the measuring camera 3 is arranged opposite to the opening, and the opening is arranged opposite to the marker 7 .

[0059] In one embodiment of the present application, the inner wall of the housing 4 has a friction surface 6, and the plumb line 2 is frictionally connected with the friction surface 6 when shaking.

[0060] Specifically, in order to ensure the measurement accuracy, a relatively rough material is used on the contact surface between the device housing 4 and the plumb line so that the plumb line 2 can stop as soon as possible after shaking.

[0061] In this application, gravity and camera recognition algorithms are used to achieve real-time measurement and compensation of camera angle changes caused by the shaking of the observation platform. No operation is required to achieve self-balancing, which is equivalent to leveling measurement.

[0062] In another embodiment of this scheme, see Figure 4 In addition to using the L-shaped dual-camera structure, other solutions can also be used to achieve similar functions. Because the calibration camera 1 only needs to shoot the plumb line 2, and the width of the plumb line 2 in the image is usually only one or two hundred pixels, this is undoubtedly a waste for a camera with a lateral resolution of thousands of pixels. Therefore, objects from two different directions can be observed by placing a spectroscope in front of the camera. One direction is used to monitor the displacement of the target object, and the other direction is used to monitor the plumb line 2 to compensate for the angle change caused by the instability of the camera. It should be noted that the distance from the camera to the target object is usually much greater than the distance to the plumb line 2, and the depth of field of the camera is limited, so it is impossible to make two objects far away from each other clearly imaged at the same time. In order to solve this problem, it is necessary to introduce a convex lens-based setting in front of the plumb line 2. According to the principle of geometric optics, when the plumb line 2 is placed near the focal plane of the convex lens, the convex lens collimates the light from a point on the plumb line 2 to form approximately parallel light, thereby changing the distance between the virtual image of the plumb line 2 and the virtual image of the plumb line 2. In this way, by adjusting the distance between the convex lens and the plumb line 2, a clear plumb line image can be formed on the camera sensor without affecting the clarity of the target image. The plumb line 2 is fixedly connected to the camera, so the camera self-calibration can be achieved according to the L-type dual camera process. By introducing a beam splitter, a single camera can simultaneously monitor the structural deformation of the target and self-calibrate the camera.

[0063] Next, a self-calibration camera combination measuring device proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0064] like Figure 5As shown, the self-calibration camera combination measuring device includes a plurality of camera self-calibration measuring devices as described in any one of the above schemes, and the plurality of camera self-calibration measuring devices are arranged in sequence; the self-calibration camera combination measuring device also includes a plurality of markers, and the markers are arranged on the camera self-calibration measuring devices; the markers arranged on the previous camera self-calibration measuring device and the markers arranged on the next camera self-calibration measuring device are opposite, so that the previous camera self-calibration measuring device monitors the settlement of the next camera self-calibration measuring device itself. It can be understood that one marker is arranged on each camera self-calibration measuring device, and the remaining markers are arranged on the target to be monitored.

[0065] The present application combines the camera self-calibration measurement devices to form a camera measurement network, which is suitable for full coverage observation of large structures and makes up for the defect of limited observation distance of a single camera.

[0066] The self-calibration camera combination measuring device provided in the present application has all the above beneficial effects because it is provided with the camera self-calibration measuring device described in any of the above technical solutions, which will not be repeated here.

[0067] Next, a camera self-calibration measurement method proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0068] like Figure 6 As shown, the camera self-calibration measurement method includes the following steps:

[0069] In step S101, the calibration camera collects a component image and sends it to the measurement camera, wherein the component image is an image containing the self-balancing component.

[0070] Specifically, the calibration camera photographs the plumb line at a certain frame rate, and the measurement camera then photographs the marker at the same frame rate as the calibration camera.

[0071] In step S102, the measuring camera processes the component image to obtain a baseline and a target line corresponding to the self-balancing component, and calculates an angle between the target line and the baseline, wherein the angle is a change angle of the measuring camera in the pitch direction.

[0072] In a possible implementation, the component image is preprocessed to obtain a processed component image; straight line extraction is performed on the processed component image to obtain a straight line equation of the plumb line in the image plane; a straight line equation corresponding to a first frame image among the multiple straight line equations is used as a baseline, and a straight line equation corresponding to a current frame image among the multiple straight line equations is used as a target line.

[0073] For details, see Figure 7, use the secondary camera to shoot the plumb line at a certain frame rate, first perform a series of preprocessing operations such as contrast enhancement and Gaussian filtering on each image, then perform edge extraction on the preprocessed image and use the straight line extraction method to obtain the pixel coordinates of the plumb line, and then obtain the straight line equation of the plumb line in the image plane, and select the plumb line in a frame of the image as the baseline.

[0074] Furthermore, the equation of the plumb line in the image coordinate system is obtained by straight line extraction as follows:

[0075] ;

[0076] ;

[0077] in, Represents the baseline, Representative The equation of the plumb line obtained from the picture is represents the slope corresponding to the baseline, represents the intercept corresponding to the baseline, Representative The slope of the target line of the picture, Representative The intercept corresponding to the target line of the first picture.

[0078] See also Figure 7 , calculate the angle between the plumb line and the baseline in the subsequent pictures. The calculated angle is the angle change of the secondary camera in the roll direction, and also the angle change of the main camera in the pitch direction.

[0079] Furthermore, after obtaining the straight line equation, the angle between the straight lines is calculated. The angle calculation formula is:

[0080] ;

[0081] in, is the angle.

[0082] In step S103, the measuring camera collects a marker image containing the marker, and performs angle correction on the marker image according to the included angle to obtain target data.

[0083] In a possible implementation, the virtual displacement of the measuring camera is calculated according to the angle; the target displacement corresponding to the marker in the marker image is calculated; and the difference between the target displacement and the virtual displacement is calculated to obtain the compensated target displacement.

[0084] For details, see Figure 7The obtained angle can be used to eliminate the virtual displacement caused by the change of the camera's own angle in the settlement of the mark point monitored by the main camera, so as to achieve the purpose of correcting the influence of the camera's own shaking on the measurement.

[0085] Further, reusing the obtained angle , calculate the virtual displacement caused by the camera's own angle change. The calculation formula for the virtual displacement is:

[0086] ;

[0087] in, is the virtual displacement, To measure the distance from the camera to the marker (measurement target).

[0088] Then the virtual displacement is obtained from the target displacement obtained by the measurement camera Subtract it from the value to get the compensated value. The compensation formula is:

[0089] ;

[0090] in, is the target displacement after compensation, is the target displacement.

[0091] Next, a phase combination measurement method proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0092] like Figure 8 As shown, the combined measurement method comprises the following steps:

[0093] In step S201, for the previous camera self-calibration measurement device, the previous calibration camera captures the previous component image and sends it to the previous measurement camera, wherein the previous component image is an image containing the previous self-balancing component;

[0094] In step S202, the previous measuring camera processes the previous component image to obtain the previous reference line and the previous target line corresponding to the previous self-balancing component, and calculates the previous angle between the previous target line and the previous reference line, wherein the previous angle is the change angle of the previous measuring camera in the pitch direction;

[0095] In step S203, the previous measurement camera collects a marking image containing a marking object, and performs angle correction on the marking image according to the previous included angle to obtain marking data; wherein the marking object is set on a camera self-calibration measurement device;

[0096] In step S204, for the camera self-calibration measurement device, a calibration camera captures a component image and sends it to a measurement camera, wherein the component image is an image containing a self-balancing component;

[0097] In step S205, a measuring camera processes the component image to obtain a reference line and a target line corresponding to the self-balancing component, and calculates an angle between the target line and the reference line, wherein the angle is a change angle of the measuring camera in the pitch direction;

[0098] In step S206, the measuring camera collects a marker image containing a marker, and performs angle correction on the marker image according to the included angle to obtain marker data;

[0099] In step S207, the self-calibration camera combination measuring device obtains target data of the marker according to the identification data and the marker data.

[0100] It should be noted that the camera may have angle and displacement changes during measurement, causing measurement errors. A single camera self-calibration measurement device can calibrate angle changes, while displacement changes depend on the markers set on the next camera monitored by the previous camera during combined measurement. This application can reflect the constraint relationship between multiple stations in combined measurement.

[0101] It should be noted that for the self-calibration camera combination measurement device (i.e., camera measurement network) composed of multiple camera self-calibration measurement devices, since settlement may occur at the location of the device, and this settlement cannot be compensated by the device itself, a marker (i.e., identification object) is also installed on the device, and the previous camera monitors the device, so that the true displacement of the entire network can be obtained.

[0102] See also Figure 5 , the camera network numbers from left to right are , , , , , , , .

[0103] For the markers at both ends , For , the target displacement formula is:

[0104] ;

[0105] ;

[0106] in, , It is a camera self-calibration measurement device , The landmark points obtained by the measurement camera Displacement, landmarks Displacement, , yes , The virtual displacement due to shaking obtained by calibrating the camera.

[0107] For the landmarks in the middle of the network , Markers In terms of target displacement The formula is:

[0108] ;

[0109] ;

[0110] in, , It is a camera self-calibration measurement device The landmark points obtained by the measurement camera Displacement, markers Displacement, , yes The virtual displacement caused by the shaking of the calibrated camera is Is installed in the camera self-calibration measurement device By analogy, we can get the real displacement of all nodes in the entire network.

[0111] The present application combines and uses multiple camera self-calibration measurement devices to form a self-calibration camera combination measurement device (i.e., a camera measurement network), and achieves full coverage observation of large structures based on the combined measurement method of the self-calibration camera combination measurement device.

[0112] In the description of this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0113] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0114] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0115] It should be noted that: in this application, unless otherwise clearly specified and limited, the first feature being "above" or "below" the second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, the first feature being "above", "above" and "above" the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. The first feature being "below", "below" and "below" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0116] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0117] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A combined measurement method based on a self-calibration camera combined measurement device, characterized in that: The self-calibration camera combination measuring device comprises a plurality of camera self-calibration measuring devices, and the plurality of camera self-calibration measuring devices are arranged in sequence; The self-calibration camera combination measuring device further comprises a plurality of markers, each of which is arranged on a corresponding camera self-calibration measuring device; The markers provided on the first camera self-calibration measuring device and the second camera self-calibration measuring device are opposite to each other, so that the first camera self-calibration measuring device monitors the settlement of the second camera self-calibration measuring device itself; A marker is also provided on the target to be monitored; The camera self-calibration measuring device comprises a measuring camera, a calibration camera and a self-balancing member, wherein the measuring camera is arranged opposite to the marker; Camera self-calibration measurement methods include: The calibration camera collects a component image and sends it to the measurement camera, wherein the component image is an image containing the self-balancing component; The measuring camera processes the component image to obtain a reference line and a target line corresponding to the self-balancing component, and calculates an angle between the target line and the reference line, wherein the angle is a change angle of the measuring camera in a pitch direction; The measuring camera collects a marker image containing the marker, performs angle correction on the marker image according to the included angle, and obtains target data; The self-balancing member is a plumb line, The processing of the component image to obtain the reference line and the target line corresponding to the self-balancing component specifically includes: Preprocessing the component image to obtain a preprocessed component image; Performing straight line extraction on the preprocessed component image to obtain multiple straight line equations of the plumb line in the image plane; The linear equation corresponding to the first frame image among the plurality of linear equations is used as a reference line, and the linear equation corresponding to the current frame image among the plurality of linear equations is used as a target line; The combined measurement method comprises: For the previous camera self-calibration measurement device, the previous calibration camera captures the previous component image and sends it to the previous measurement camera, wherein the previous component image is an image containing the previous self-balancing component; The previous measuring camera processes the previous component image to obtain the previous reference line and the previous target line corresponding to the previous self-balancing component, and calculates the previous angle between the previous target line and the previous reference line, wherein the previous angle is the change angle of the previous measuring camera in the pitch direction; The previous measuring camera collects a marking image containing a marking object, and performs angle correction on the marking image according to the previous angle to obtain marking data; wherein the marking object is set on the subsequent camera self-calibration measuring device; For the rear camera self-calibration measurement device, the rear calibration camera collects the rear component image and sends it to the rear measurement camera, wherein the rear component image is an image containing the rear self-balancing component; The rear measuring camera processes the rear component image to obtain a rear reference line and a rear target line corresponding to the rear self-balancing component, and calculates a rear angle between the rear target line and the rear reference line, wherein the rear angle is a change angle of the rear measuring camera in the pitch direction; The latter measuring camera collects a mark image containing a mark object, and performs angle correction on the mark image according to the included angle to obtain a mark data; The self-calibration camera combination measuring device obtains target data of the one marker according to the one identification data and the one marker data; The camera network numbers from left to right are: , ,in, Represents the camera self-calibration device, Indicates a marker, Indicates the marker, is a positive integer; For the markers at both ends , For , the target displacement formula is: ； ； in, , It is a camera self-calibration measurement device , The landmark points obtained by the measurement camera Displacement, landmarks Displacement, , yes , The virtual displacement due to shaking is obtained by calibrating the camera; For the landmarks in the middle of the network , Markers In terms of target displacement The formula is: ； ； in, , It is a camera self-calibration measurement device The landmark points obtained by the measurement camera Displacement, markers Displacement, , yes The virtual displacement caused by the shaking of the calibrated camera is Is installed in the camera self-calibration measurement device The displacement of the marked point on ; The calibration camera and the measuring camera are the same camera, that is, a single camera simultaneously monitors the structural deformation of the target object and the self-calibration of the camera. A spectroscope is set in front of the single camera to observe objects from two different directions, one direction is used to monitor the displacement of markers and markers, and the other direction is used to monitor the plumb line. At the same time, a convex lens is set in front of the plumb line.

2. The combined measurement method according to claim 1, characterized in that: The target data is the target displacement after compensation; The performing angle correction on the sign image according to the included angle to obtain target data specifically includes: According to the angle, calculating the virtual displacement of the measuring camera; Calculating the target displacement corresponding to the marker in the marker image; The difference between the target displacement and the virtual displacement is calculated to obtain the compensated target displacement.

3. The combined measurement method according to claim 2, characterized in that: The calculation formula of the angle is: ； in, is the angle, is the slope corresponding to the baseline, is the slope corresponding to the target line; The calculation formula of the virtual displacement is: ； in, is the virtual displacement, The distance from the measuring camera to the marker; The compensation formula of the target displacement after compensation is: ； in, is the target displacement after compensation, is the target displacement.

4. The combined measurement method according to claim 1, characterized in that: The optical axes of the measuring camera and the calibration camera are L-shaped; the weight of the plumb line is vertically downward through the plumb line; the calibration camera is fixedly connected to the measuring camera, and the angle between the optical axes of the measuring camera and the calibration camera is fixed; the self-balancing member is respectively connected to the measuring camera and the calibration camera.

5. The combined measurement method according to claim 4, characterized in that: The camera self-calibration measuring device also includes a housing and a support; The measuring camera and the calibration camera are disposed in the housing, the supporting member is disposed in the housing, and the supporting member abuts against the calibration camera.

6. The combined measurement method according to claim 5, characterized in that: The inner wall of the shell has a friction surface, and the plumb line is frictionally connected with the friction surface when it swings.

Images