High-precision binocular vision measurement method and device for complex environmental conditions

Abstract

The present invention provides a high-precision binocular vision measurement method and device for complex environmental conditions. The method includes: Step 1, arranging active light-emitting special marker points on the surface of the object area where spatial position measurement is required; Step 2, obtaining the internal parameter matrix and external parameter matrix of two cameras by using the Zhang Zhengyou calibration method; Step 3, optimizing the exposure time of the binocular cameras, collecting the special marker points and surrounding images under different environmental and weather conditions, and realizing the automatic detection of the boundaries of the light-emitting marker points of the binocular system; Step 4, fitting and identifying the shapes of the marker points obtained at different shooting angles, and determining the central position coordinates of the marker points; Step 5, converting the central position coordinates of the marker points into three-dimensional spatial coordinates to obtain the spatial position information of the object to be measured. The method of the present invention solves the problem of the influence of image degradation on the measurement accuracy in a complex imaging environment, and realizes non-contact and high-precision measurement of the target.

Description

Technical Field

[0001] The present invention relates to the field of measurement technology, and particularly to a high-precision binocular vision measurement method and device for complex environmental conditions. Background Art

[0002] In complex weather and environmental conditions such as clouds, fog, rain, snow, and darkness, the binocular vision measurement system will be affected by the complex imaging environment, resulting in inaccurate measurement results. Deformation monitoring methods are divided into contact measurement and non-contact measurement. Contact measurement requires the installation of optical fibers or strain sensors inside or on the surface, which has disadvantages such as the sensors being sensitive to temperature, the limited number of measurement points, and the sensors and cables changing the weight. The main method of non-contact measurement is the machine vision system, which arranges feature marker points on the surface and uses cameras to collect images and measure the pixel coordinates of the feature marker points. The binocular vision measurement system can convert the pixel coordinates of the feature marker points into three-dimensional space coordinates, thereby obtaining the deformation amount. This method does not change the original weight and has the characteristics of multi-point measurement, real-time performance, and high accuracy.

[0003] The binocular vision measurement system has high requirements for the quality of the captured images. Only in clear images can the accurate coordinates of the feature marker points be identified. In actual use, in harsh operating environments such as clouds, fog, dust, rain, ice, insufficient light, and darkness, image degradation will occur during the imaging process, resulting in reduced test accuracy or the inability to use the binocular vision test technology. Summary of the Invention

[0004] Object of the Invention: The technical problem to be solved by the present invention is to provide a high-precision binocular vision measurement method and device for complex environmental conditions in view of the deficiencies of the prior art, so as to solve the problem of image degradation in complex imaging environments, realize the recognition of feature marker points under conditions such as clouds, fog, dust, rain, ice, darkness, and insufficient light, and thus achieve the precise measurement of the object to be measured.

[0005] The method of the present invention includes the following steps:

[0006] Step 1, arrange active light-emitting special marker points on the surface area of the object where spatial position measurement is required;

[0007] Step 2, calibrate the binocular vision system, where the binocular vision system includes binocular cameras;

[0008] Step 3, optimize the exposure time of the binocular cameras. Using the prior art, adopt different exposure times according to different weather environments, collect the special marker points and surrounding images under different environmental and weather conditions, and use the contour detection image feature extraction algorithm to realize the automatic detection of the boundaries of the light-emitting marker points of the binocular system;

[0009] Step 4: Using the marker point shape fitting method, fit and identify the marker point shapes obtained at different shooting angles, and determine the central position coordinates of the marker points;

[0010] Step 5: According to the binocular system three-dimensional reconstruction algorithm, convert the central position coordinates of the marker points into three-dimensional space coordinates to obtain the spatial position information of the object to be measured.

[0011] In Step 1, the active light-emitting special marker points include a light-emitting lamp panel and circular marker points. The circular marker points are arranged at the center of the light-emitting lamp panel, and the marker points are positioned according to the light-emitting area. The active light-emitting special marker points are distributed at key positions of the object to be measured, such as stress concentration areas, areas where imaging is easily affected by complex environments, areas with large changes in spatial positions, and other areas where spatial position changes need to be monitored. For an object to be measured that requires measuring the spatial changes in the overall area, multiple special marker points can be evenly arranged on the surface, and the spatial positions of each special marker point are measured to restore the change information of the entire key area.

[0012] In Step 2, the imaging model of the camera includes the world coordinate system O W X W Y W Z W , the camera coordinate system O C X C Y C Z C , the image pixel coordinate system O uv UV and the image physical coordinate system O xy xy; According to the Zhang Zhengyou calibration method, the internal parameter matrices and external parameter matrices of the two cameras are obtained, and a point P(X W ,Y W ,Z W ) in the world coordinate system is used to generate a point p(x,y) in the image physical coordinate system through the camera imaging model.

[0013] In Step 3, for the optimization of the exposure time of the binocular cameras, the images at low exposure times will weaken the background and reduce the influence of the surrounding environment on imaging. The low exposure time can strengthen the highlighted parts in the image, highlight the active light-emitting area, making the active light-emitting marker points form a strong contrast with the surrounding environment in the image, and realizing the acquisition of clear images of the marker points in complex environments.

[0014] In Step 4, for the fitting of the marker point shapes obtained at different shooting angles, for the circular marker points inside the light-emitting area, the shooting shapes at different angles are all ellipses; the edge pixel coordinates of the circular marker points are identified using the edge detection method, and the elliptical equation is fitted using the discrete edge pixel coordinates to reduce the pixel deviation generated by boundary extraction and realize the detection of the central position coordinates of the marker points with high precision. The specific steps are as follows:

[0015] Step 4-1, convert the grayscale image to a binary image;

[0016] Step 4-2, identify the luminous marker points;

[0017] Step 4-3, erode the binary image;

[0018] Step 4-4, perform edge detection using anisotropic Gaussian directional derivatives;

[0019] Step 4-5, fit the elliptic equation to the edge pixel coordinates;

[0020] Step 4-1 includes: The captured image by the binocular camera is a grayscale image. Set the grayscale threshold according to the image characteristics. The value range of the grayscale threshold is 180-220. For each pixel in the image, when 0≤pixel grayscale value≤grayscale threshold, the grayscale value of this pixel point is set to 0; when grayscale threshold<pixel grayscale value≤255, the grayscale value of this pixel point is set to 255, and convert the grayscale image to a binary image;

[0021] Step 4-2 includes: Set the area ratio of the special marker point to the luminous area as the detection threshold. Calculate the ratio of the number of pixel points in the marker area to the number of pixel points in the luminous area in the binary image obtained in Step 4-1. If the ratio is less than 80% of the detection threshold, it indicates that the marker point image is greatly affected by the environment and cannot be correctly used for binocular vision measurement; for the marker points less than the threshold, delete the images of these marker points and do not use them in the subsequent steps.

[0022] Step 4-3 includes: Set the center coordinates of the marker point at the i-th iteration as (x i , y i ), where i represents the iteration number of image erosion. The offset δ i of the center coordinates of the marker point between two erosion iterations is defined as:

[0023]

[0024] The change Δ j of the center coordinate offset of the marker point is defined as:

[0025] Δ j = |δ j+1 - δ j |

[0026] where δ j+1 represents the (j + 1)-th image erosion, and δ j represents the j-th image erosion. Multiple erosion iterations can eliminate some noise in the image and reduce the influence of the complex imaging environment on the recognition of image marker points. When Δ jWhen it is ≤ 0.05, the deviation of the marked point pixel coordinates is less than or equal to 0.05 pixels, and the center of the marked point after the j-th image erosion is determined as the actual value. The image stops eroding, and the final marked point center coordinates are (x j , y j );

[0027] Step 4-4 includes: Set I(x) to represent an image, x = (x, y) T represents the pixel coordinates of each pixel, where x and y represent the abscissa and ordinate of the pixel respectively, T represents matrix transpose, and the discrete two-dimensional anisotropic Gaussian kernel function g σ,ρ (x) is expressed as:

[0028]

[0029] where σ is the scale factor and ρ is the anisotropy factor;

[0030] Derive x to obtain the directional derivative g σ,ρ,θ (x) of the two-dimensional Gaussian kernel function:

[0031]

[0032] R θ is the rotation matrix, expressed as:

[0033]

[0034] where θ is the rotation angle;

[0035] Convolve the directional derivative of the two-dimensional Gaussian kernel function with the image, and select the maximum local gradient value within each pixel neighborhood as the edge detection result. The discrete pixel coordinates after the final edge detection are denoted as (x i , y i ) T ;

[0036] Step 4-5 includes: The ellipse equation is expressed as:

[0037] ax 2 + bxy + cy 2 + dx + ey + f = 0

[0038] The constraint of the ellipse equation is b 2 - 4ac < 0. Perform least squares processing on the discrete pixel coordinates after edge detection to obtain the coefficients of the ellipse equation, that is, find the minimum value of the objective function f(a, b, c, d, e, f) in the following formula to determine each coefficient:

[0039]

[0040] Perform elliptical equation fitting on the edge pixel coordinates of the marked points, and the coordinates of the center point of the ellipse are the characteristic coordinates of the marked points.

[0041] The present invention also provides a high-precision binocular vision measurement device for complex environmental conditions, including:

[0042] A marking module for arranging actively emitting special marked points on the surface area of the object whose spatial position needs to be measured;

[0043] A calibration module for calibrating the binocular vision system, and the binocular vision system includes binocular cameras;

[0044] An optimization module for optimizing the exposure time of the binocular cameras, collecting images of the special marked points and their surroundings under different environmental and weather conditions, and using the contour detection image feature extraction algorithm to realize the automatic detection of the boundaries of the light-emitting marked points of the binocular system;

[0045] A coordinate calculation module for using the marked point shape fitting method to fit and identify the shapes of the marked points obtained at different shooting angles, and determining the central position coordinates of the marked points;

[0046] A spatial position information acquisition module for converting the central position coordinates of the marked points into three-dimensional spatial coordinates according to the three-dimensional reconstruction algorithm of the binocular system, and obtaining the spatial position information of the object to be measured.

[0047] Compared with the prior art, the present invention has the following technical effects:

[0048] (1) The non-contact measurement adopted by the present invention can not change the structural weight, and has small temperature drift, and can measure the spatial position of the object more accurately.

[0049] (2) The present invention adopts an actively emitting special marked point, which meets the requirements of image acquisition at night, and can still capture clear marked point images under poor external environmental light conditions.

[0050] (3) The present invention can realize accurate measurement of images in different complex weather environments such as clouds, fog, sand and dust, rain, and ice. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The following further specific description of the present invention is made in conjunction with the drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0052] Figure 1 is a schematic diagram of the installation of the binocular cameras of the present invention.

[0053] Figure 2a 、 Figure 2b 、 Figure 2c 、 Figure 2dare images of the present invention in cloud, fog, dust, and rain imaging environments, respectively. Figure 2e , Figure 2f It is an image of the present invention in an ice imaging environment.

[0054] Figure 3a , Figure 3b , Figure 3c This is an embodiment of the present invention which takes the winglet deflection as the detection object, and illustrates the measurement results of the winglet deflection angle in simulated images under different imaging environments. DETAILED DESCRIPTION

[0055] Example

[0056] The structure of the present invention adopts an adaptive deflectable winglet model, a binocular camera installation and a winglet model as shown in FIG. Figure 1 As shown, the steps for measuring the winglet deflection angle are as follows:

[0057] (1) Arrange marking points on the surface of the deflectable area of the winglet. A special active luminous marking point consists of a white LED square light panel and a black circular marking point, and the black circular marking point is arranged in the center of the light panel. Paste three luminous marking points on the surface of the winglet at the leading edge, middle and trailing edge, and calculate the deformation according to the coordinate change of the marking points.

[0058] (2) Calibrate the binocular vision system. The camera imaging model includes the world coordinate system O W X W Y W Z W , camera coordinate system O C X C Y C Z C , image pixel coordinate system O uv UV and image physical coordinate system O xy According to Zhang Zhengyou's calibration method, we get the intrinsic parameter matrix and extrinsic parameter matrix of the left and right cameras, and convert a point P(X W ,Y W ,Z W ) Generates a point p(x,y) in the image physical coordinate system through the camera imaging model.

[0059] (3) The binocular camera collects images of the winglets at different deflection angles in complex environments, and uses an image feature extraction algorithm to automatically detect and locate the luminous marker points of the binocular system. Figures 2a to 2e shown.

[0060] Furthermore, the image feature extraction algorithm in step (3) includes:

[0061] Step 3-1, converting grayscale image to binary image: The acquired image by the binocular camera is a grayscale image. Set the grayscale threshold to 200 according to the image characteristics. For each pixel in the image, when 0 ≤ pixel grayscale value ≤ 200, set the grayscale value of this pixel point to 0; when 200 < pixel grayscale value ≤ 255, set the grayscale value of this pixel point to 255, and convert the grayscale image to a binary image;

[0062] Step 3-2, identifying the luminous marker points: The ratio of the area of the special marker point to the luminous area is the detection threshold. Set the detection threshold to 0.40. Calculate the ratio of the number of pixel points in the marker area to the number of pixel points in the luminous area in the binary image obtained in Step 3-1. If the ratio is less than 0.40×80% = 0.32 of the detection threshold, it indicates that the marker point image is greatly affected by the environment and cannot be correctly used for binocular vision measurement. For the marker points with a ratio less than the threshold, as Figure 2f shown, delete the image of this marker point and do not use it for the subsequent calculation steps; for the marker points with a ratio greater than the threshold, as Figure 2a , Figure 2b , Figure 2c , Figure 2d , Figure 2e shown, it indicates that the marker point image can accurately identify the luminous marker points and proceed with the subsequent calculation steps.

[0063] Step 3-3, eroding the binary image: Set the center coordinates of the marker point as (x i , y i ). Let i represent the number of iterations of image erosion. The offset δ i of the center coordinates of the marker point between two consecutive erosion iterations is defined as:

[0064]

[0065] The change Δ j in the offset of the center coordinates of the marker point is defined as:

[0066] Δ j = |δ j+1 - δ j |

[0067] where δ j+1 represents the (j + 1)-th image erosion, and δ j represents the j-th image erosion. Multiple erosion iterations can eliminate some noise in the image and reduce the influence of the complex imaging environment on the recognition of image marker points. When Δ j ≤ 0.05, the deviation of the pixel coordinates of the marker point is less than or equal to 0.05 pixels, and the center of the marker point after the j-th image erosion is considered the actual value. Stop eroding the image, and the final center coordinates of the marker point are (x j , y j );

[0068] Step 3-4, Anisotropic Gaussian Direction Derivative Edge Detection: Assume that I(x) represents an image, where x = (x, y) T represents the pixel coordinates of each pixel, and the discrete two-dimensional anisotropic Gaussian kernel function g σ,ρ (x) is expressed as:

[0069]

[0070] where σ is the scale factor and ρ is the anisotropic factor;

[0071] Derive x to obtain the direction derivative g σ,ρ,θ (x) of the two-dimensional Gaussian kernel function:

[0072]

[0073] R θ is the rotation matrix, expressed as:

[0074]

[0075] where θ is the rotation angle;

[0076] Convolve the direction derivative of the two-dimensional Gaussian kernel function with the image, and select the local gradient maximum value within each pixel neighborhood as the edge detection result. The discrete pixel coordinates after the final edge detection are denoted as (x i , y i ); T ;

[0077] Step 3-5, Ellipse Equation Fitting of Edge Pixel Coordinates: The ellipse equation is expressed as:

[0078] ax 2 + bxy + cy 2 + dx + ey + f = 0

[0079] The constraint of the ellipse equation is b 2 - 4ac < 0. Perform least squares processing on the discrete pixel coordinates after edge detection to obtain the coefficients of the ellipse equation, that is, find the minimum value of the objective function f(a, b, c, d, e, f) in the following formula to determine each coefficient:

[0080]

[0081] Perform ellipse equation fitting on the edge pixel coordinates of the marked points, and the center point coordinates of the ellipse are the characteristic coordinates of the marked points.

[0082] (4) Detect the characteristic coordinates of the marked points in images of different flight environments, convert the pixel coordinates of the marked points into three-dimensional space coordinates according to the calibration results in step (2), and obtain the deformation amount based on the three-dimensional coordinate changes.

[0083] (5) The angular deflection is mainly manifested as the change of the Z coordinate of the marker point in the three-dimensional space. The small wing deflection angle is calculated through the Z coordinate, and the average value of the angles of the three marker points is taken as the binocular vision angle measurement value. The angle measurement values of the simulated images in different weather environments are compared with those of the clear original images, and the measurement errors are as Figure 3a , Figure 3b , Figure 3c shown. The measurement errors indicate that within the range of -15° to 20° of the small wing deflection, the angular measurement deviation between the simulated images in different weather environments and the clear original images is less than 0.08°. Compared with the prior art, the present invention can achieve high-precision identification of the small wing deflection angle in a complex imaging environment. In specific implementation, the present application provides a computer storage medium and a corresponding data processing unit. Among them, the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, it can run the inventive content and some or all of the steps in each embodiment of the high-precision binocular vision measurement method provided by the present invention for complex environmental conditions. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.

[0084] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of a computer program and its corresponding general hardware platform. Based on such an understanding, the essence of the technical solutions in the embodiments of the present invention, or the part that contributes to the prior art, can be embodied in the form of a computer program, that is, a software product. The computer program software product can be stored in a storage medium, including several instructions for causing a device (which can be a personal computer, a server, a single-chip microcomputer, a MUU, or a network device, etc.) including a data processing unit to execute the methods described in each embodiment or some parts of the embodiments of the present invention.

[0085] The present invention provides a high-precision binocular vision measurement method and device for complex environmental conditions. There are many methods and ways to specifically implement this technical solution. The above description is only the preferred implementation manner of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by the prior art.

Claims

1. A high-precision binocular vision measurement method for complex environmental conditions, characterized in that, It includes the following steps: Step 1: Arrange active light-emitting special marker points on the surface area of the object where spatial position measurement is required; Step 2: Calibrate the binocular vision system, and the binocular vision system includes a binocular camera; Step 3: Optimize the exposure time of the binocular camera, collect images of the special marker points and the surrounding area under different environmental and weather conditions, and use the contour detection image feature extraction algorithm to automatically detect the boundaries of the light-emitting marker points in the binocular system; Step 4: Adopt a marker point shape fitting method to fit and identify the shapes of the marker points obtained at different shooting angles, and determine the central position coordinates of the marker points; Step 4 includes: Step 4-1: Convert the grayscale image to a binary image; Step 4-2: Identify the light-emitting marker points; Step 4-3: Erode the binary image; Step 4-3 includes: setting the center coordinates of the marked points at the i-th iteration as (x i , y i ), where i represents the number of iterations of image erosion, and the offset δ i of the center coordinates of the marked points between two erosion iterations is defined as: Change Δ in the offset of the center coordinates of the marked points j is defined as: Δ j = |δ j+1 - δ j | Among them, δ j+1 represents the (j + 1)-th image erosion, and δ j represents the j-th image erosion; when Δ j ≤ 0.05, the deviation of the marked point pixel coordinates is less than or equal to 0.05 pixels, and it is determined that the center of the marked point after the j-th image erosion is the actual value; the image stops eroding, and the final marked point center coordinates are (x j , y j ); Step 4-4: Anisotropic Gaussian directional derivative edge detection; Step 4-5: Fit the elliptic equation to the edge pixel coordinates; Step 5: According to the three-dimensional reconstruction algorithm of the binocular system, convert the central position coordinates of the marker points into three-dimensional spatial coordinates to obtain the spatial position information of the object to be measured.

2. The method according to claim 1, wherein In Step 1, the active light-emitting special marker points include a light-emitting lamp board and a circular marker point. The circular marker point is arranged at the center of the light-emitting lamp board, and the marker point positioning is realized according to the light-emitting area. The active light-emitting special marker points are distributed at the key positions of the object to be measured.

3. The method according to claim 2, wherein In Step 2, the imaging model of the camera includes the world coordinate system O W X W Y W Z W , the camera coordinate system O C X C Y C Z C , the image pixel coordinate system O uv UV and the image physical coordinate system O xy xy; According to the Zhang-Zhengyou calibration method, the intrinsic parameter matrices and extrinsic parameter matrices of two cameras are obtained. A point P(X W , Y W , Z W ) in the world coordinate system is used to generate a point p(x, y) in the image physical coordinate system through the camera imaging model.

4. The method according to claim 3, characterized in that Step 4-1 includes: The collected image of the binocular camera is a grayscale image. Set the grayscale threshold according to the image characteristics. For each pixel in the image, when 0 ≤ pixel grayscale value ≤ grayscale threshold, set the grayscale value of the pixel point to 0; when grayscale threshold < pixel grayscale value ≤ 255, set the grayscale value of the pixel point to 255, and convert the grayscale image into a binary image.

5. The method according to claim 4, characterized in that, Step 4-2 includes: Set the area ratio of the special marker point to the light-emitting area as the detection threshold. Compare the number of pixel points in the marker area with the number of pixel points in the light-emitting area in the binary image obtained in Step 4-1. If the ratio is less than 80% of the detection threshold, it indicates that the marker point image is affected by the environment and cannot be correctly used for binocular vision measurement; for the marker points less than the threshold, delete the images of the marker points and do not use them in the subsequent steps.

6. The method according to claim 5, characterized in that Step 4-4 includes: setting I(x) to represent an image, where x = (x, y) T represents the pixel coordinates of each pixel, where x and y respectively represent the abscissa and ordinate of the pixel, T represents matrix transpose, and the discrete two-dimensional anisotropic Gaussian kernel function g σ,ρ (x) is expressed as: Wherein, σ is the scale factor and ρ is the anisotropy factor; Derive the derivative of \(x\) to obtain the directional derivative \(g\) of the two-dimensional Gaussian kernel function σ,ρ,θ (x): R θ is a rotation matrix, expressed as: Where θ is the rotation angle; Convolve the directional derivative of the two-dimensional Gaussian kernel with the image, select the maximum local gradient within each pixel neighborhood as the edge detection result, and finally record the discrete pixel coordinates after edge detection as (x i , y i ). T .

7. The method according to claim 6, wherein Step 4-5 includes: The elliptic equation is expressed as: ax 2 +bxy + cy 2 +dx + ey + f = 0 The constraint of the ellipse equation is b 2 -4ac < 0. Perform least squares processing on the discrete pixel coordinates after edge detection to find the minimum value of the objective function f(a, b, c, d, e, f) in the following formula to determine each coefficient: Fit the elliptic equation to the edge pixel coordinates of the marker point, and the center point coordinates of the ellipse are the characteristic coordinates of the marker point.

8. A high-precision binocular vision measurement device for complex environmental conditions implemented by the method according to claim 1, characterized in that, It includes: A marking module for arranging active light-emitting special marker points on the surface area of the object where spatial position measurement is required; A calibration module for calibrating the binocular vision system, and the binocular vision system includes a binocular camera; An optimization module for optimizing the exposure time of the binocular camera, collecting images of the special marker points and the surrounding area under different environmental and weather conditions, and using the contour detection image feature extraction algorithm to automatically detect the boundaries of the light-emitting marker points in the binocular system; A coordinate calculation module for adopting a marker point shape fitting method to fit and identify the shapes of the marker points obtained at different shooting angles, and determining the central position coordinates of the marker points; A spatial position information acquisition module, which is used to convert the central position coordinates of the marked points into three-dimensional spatial coordinates according to the binocular system three-dimensional reconstruction algorithm, so as to obtain the spatial position information of the object to be measured.

Images