Target for microwave radar and computer vision fusion displacement measurement, calibration method and system thereof

Abstract

The invention discloses a target for microwave radar and computer vision fusion displacement measurement, a calibration method and a system thereof, belongs to the technical field of structural displacement measurement, and solves the problem that a proper target is lacked when microwave radar and computer vision are used for measuring displacement at the same time. The target covers a transparent calibration plate on the open surface of the corner reflector, and a uniform light diffuse reflection plate is attached to the interior of the transparent calibration plate; and a light source is arranged on the corner reflector. The transparent calibration plate is provided with special mark points and common mark points, the special mark points are distributed on two intersecting straight lines, the intersection points are the special mark points, and the spacing distances among all the special mark points are different. According to the calibration method, the position of the special mark point is verified by utilizing cross ratio invariance and simple ratio invariance. The measurement system adopts the target and the calibration method. According to the invention, three-dimensional displacement testing based on microwave radar and computer vision fusion can be realized, and the three-dimensional displacement change condition of a space point can be accurately obtained, so that the vibration displacement of an engineering structure can be monitored and early warned.

Description

Technical Field

[0001] The present invention relates to the technical field of structural displacement measurement, and in particular to a target, a calibration method and a system thereof for displacement measurement by integrating microwave radar with computer vision. Background Art

[0002] Displacement response is the most intuitive physical quantity characterizing a structure's state and an indispensable metric for structural inspection. Accurately acquiring displacement response remains a key challenge. In a three-dimensional world, the movement of objects is a spatial motion. Therefore, to ensure the safety of structures, non-contact three-dimensional displacement and vibration monitoring and early warning of engineering structures are essential. However, during operation, structures such as bridges, buildings, transmission towers, and wind turbine towers are subject to various external loads (such as wind and seismic loads) and inherent material aging. These structures gradually deteriorate over their service life, exhibiting a certain degree of oscillation, impacting comfort and structural safety. Currently, non-contact displacement measurement of engineering structures can be achieved using technologies such as total stations, laser vibrometers, GNSS, vision measurement, and microwave radar. The more popular monocular vision and microwave radar technologies can both measure the three-dimensional displacement of target points, but each has its limitations. Monocular vision technology offers low out-of-plane measurement accuracy. Microwave radar technology can obtain highly accurate radial displacement, but it struggles to accurately measure in-plane displacement.

[0003] In practice, both monocular vision and microwave radar technologies still require a target and corner reflector mounted on the object to be measured to accurately measure three-dimensional displacement. However, there is a lack of targets available on the market that can simultaneously perform both monocular vision and microwave radar measurements. Furthermore, industrial cameras used in monocular vision are significantly affected by ambient lighting outdoors, necessitating innovation to ensure they can capture high-quality images despite significant variations in light intensity. Existing common dot-matrix transparent calibration plates and their calibration methods for monocular vision targets lack clear scientific principles as their technical underpinnings. These principles are mostly based on a mechanical combination of special patterns and common image processing algorithms, making it difficult to achieve stable, efficient, and robust calibration of monocular vision technology in practice. The calibration method used in the present invention is simpler and has fewer restrictions on the selection of special landmarks. Projective invariance (including cross-ratio invariance and simple ratio invariance) is employed as a means of verifying the position of special landmarks, resulting in robust recognition algorithms.

[0004] Therefore, a target is needed that enables both monocular vision technology and microwave radar to simultaneously perform three-dimensional displacement measurements. This target should allow the monocular vision device to capture high-quality images to measure in-plane displacement while also reflecting microwave signals of sufficient intensity for the microwave radar to measure radial displacement. Furthermore, the target must contain a pattern that enables stable and efficient calibration of the monocular vision technology. This pattern must adhere to clear scientific principles and possess constant parameters within a digital image that conforms to the laws of perspective transformation. Furthermore, a corresponding recognition algorithm must be able to eliminate the influence of complex background noise and stably identify the target's constant parameters, thereby identifying the target. Summary of the Invention

[0005] The present invention aims to solve the problem of image distortion caused by drastic changes in outdoor light intensity when using the Digital Image Correlation method for measuring outdoor scenes, and to address the problem of a lack of a target that can be used simultaneously for monocular camera calibration, pixel displacement measurement, and radar signal reflection when microwave radar and computer vision are simultaneously measuring displacement. The present invention provides a target, calibration method, and system for displacement measurement that integrates microwave radar and computer vision.

[0006] The present invention consists of three parts: the first part is the target, the second part is a transparent calibration plate installed on the target surface for visual identification and its calibration method, and the third part is a measurement system using the target and calibration method of the present invention.

[0007] Part I: Target

[0008] The target of the present invention includes a corner reflector, a transparent calibration plate, and a uniform light diffuse reflection plate. The specific positions and connection relationships are as follows:

[0009] A transparent calibration plate is provided on the open surface of the corner reflector. A self-luminous component is provided in the cavity between the transparent calibration plate and the corner reflector. A uniform light diffuse reflection plate is provided between the transparent calibration plate and the self-luminous component. The power cord of the self-luminous component is connected to an external power supply through a reserved hole in the corner reflector.

[0010] The transparent calibration plate is a dot matrix transparent calibration plate, and a calibration pattern is etched on the transparent calibration plate. The calibration pattern is a mark point, and the mark points are divided into ordinary mark points and special mark points. The special mark points and the ordinary mark points are arranged in an array form under known local coordinates; there are at least 6 special mark points, and among all the special mark points, one of the special mark points is selected as the intersection point of two groups of collinear special mark points with different slopes. The remaining special mark points extend outward according to the selected slope with the intersection point as the reference point, and the interval distance between all special mark points is different.

[0011] The transparent calibration plate is made of a transparent material. In the present invention, a PVC plate is used. The transparent calibration plate has a pattern with rich texture information in the area other than the etched calibration pattern.

[0012] The uniform light diffuse reflection plate is attached to the inner surface of the transparent calibration plate, the self-luminous component is an infrared light source, and an infrared filter is attached to the outer surface of the transparent calibration plate.

[0013] The special marking points are double concentric circles, and the ordinary marking points are solid circles. To ensure the recognition effect, the background color of the transparent calibration plate and the color of the marking points have a clear contrast.

[0014] Furthermore, the uniform light diffuse reflection plate is pasted on the side of the transparent calibration plate without the calibration pattern etched thereon, and the infrared filter is installed on the side with the etched pattern. The transparent calibration plate, the uniform light diffuse reflection plate and the corner reflector are detachably connected.

[0015] Furthermore, the corner reflector is composed of three equal-sized right-angled isosceles triangular aluminum plates, one of which is provided with two holes for fixing the self-luminous component, and a heat sink is provided on the outer surface of the aluminum plate for fixing the self-luminous component.

[0016] The main purpose of the first part is to ensure that the microwave radar can capture phase signals of sufficient intensity, accurately find the target point in actual application, and calculate the radial displacement of the target point through the phase signal returned by the target point. In order to solve the problem of error accuracy of industrial cameras in complex light field conditions, an infrared light-emitting device that emits light that can be captured by the industrial camera is installed inside the target, and a transparent calibration plate that can be identified by computer vision technology is installed on the surface of the device perpendicular to the field of view of the industrial camera. The illumination of the infrared light-emitting device ensures that the brightness of the target meets the working requirements of computer vision displacement measurement, which can ensure that the industrial camera can still capture high-definition images under complex lighting conditions and avoid the problem of image quality degradation caused by changes in light intensity.

[0017] Part 2: Transparent calibration plate installed on the target surface for visual identification and its calibration method

[0018] The dot matrix transparent calibration plate includes at least 6 special marking points, wherein the distribution of the special marking points in the dot matrix constitutes a pattern structure containing projective information; the projective information is mainly reflected in the arrangement of the special marking points. In actual use, by identifying the pattern structure of the transparent calibration plate in the digital image, all special marking points are identified, the linear relationship between each two points is calculated, and the intersection of the linear relationship with the most collinear points is obtained. The absolute pixel distance between the intersection and other special marking points is calculated to achieve position matching of the special marking points. Finally, the cross ratio invariance and the simplicity ratio invariance in the projective transformation are used to calculate and verify the cross ratio and simplicity ratio, thereby achieving accurate recognition of the dot matrix transparent calibration plate.

[0019] The distribution requirements of the special landmarks are as follows:

[0020] Among all the special marking points, one of the special marking points is selected as the intersection point of two groups of collinear special marking points with different slopes, and the remaining special marking points are extended outward according to the selected slope with the intersection point as the reference point.

[0021] The spacing distances between all special marking points should be different.

[0022] The calibration method is as follows:

[0023] (1) Image acquisition: Use an industrial camera to acquire a calibration image containing a dot matrix transparent calibration plate; set up an infrared light source next to the camera used for image acquisition, and use the light source to directly illuminate the target pattern during the continuous monitoring phase, so that the camera can acquire images containing rich texture information during the continuous monitoring phase;

[0024] (2) Image processing: convert the calibration image into a single-channel 8-byte grayscale image and perform binarization to obtain a binary image; perform contour extraction on the binary image and establish a complete contour hierarchical topology structure;

[0025] (3) Based on the contour hierarchical topological structure, the image coordinates of all special landmarks in the calibration image are extracted, and the linear relationship between the special landmarks is calculated. The groups are summarized according to the slope, the intersection points are solved, and the intersection ratio and the simple ratio are verified. If the calculated intersection ratio is equal to the target intersection ratio, the target Equality, simple ratio and target simple ratio If they are equal, the match is successful;

[0026] (4) Projection matrix calculation: Calculate the perspective transformation projection matrix based on the local coordinates of special landmarks and their corresponding image coordinates;

[0027] (5) Projection result acquisition: Based on the obtained perspective transformation projection matrix, all the marker points in the dot matrix are projected to the image coordinate system according to the local coordinates to obtain the projection results of all the marker points in the dot matrix in the image coordinate system;

[0028] (6) Recognition result formation: Based on the projection results, the real image coordinates of all landmark points are obtained to form the final transparent calibration plate dot matrix recognition result set.

[0029] Wherein, step (3) is performed as follows:

[0030] S3.1: First, extract contours from the original image and construct a hierarchical topological structure of contours. Then, traverse this structure to find all contours without internal sub-contours and store the numbers of these contours in a set N to form a meta-contour number set N.

[0031] S3.2: Traverse each element in the set N, assuming that the current element is the i-th meta-contour; if the meta-contour has a parent contour, and the first child contour number of the parent contour is equal to i, then perform ellipse fitting on the i-th meta-contour and its parent contour; if the fitting result meets the following conditions:

[0032] (1) The distance between the centers of the two ellipses is less than a preset threshold;

[0033] (2) The ratio of the major axis of the parent ellipse to the major axis of the child ellipse is within the range [a, b];

[0034] (3) The ratio of the perimeter of the parent ellipse to the perimeter of the child ellipse is also in the range [a, b] (where a and b are constants);

[0035] Then it is determined that a special mark point is successfully identified; repeat the above process until all special mark points are identified, and record the image coordinates of each special mark point (n is used to represent the nth special mark point). The array recording the image coordinates is recorded as , whose elements are recorded as , , , , , , and finally end the traversal;

[0036] S3.3: Matching:

[0037] (1) As the vertex, calculate the remaining image coordinates in the same way The linear relationship between the lines (i.e. the slope-intercept equation ,by and Take two points as an example. , where k is the slope and b is the intercept), repeat this step until the linear relationship between all elements is found; The coordinates of , ), The coordinates of , );

[0038] (2) The slope of the linear relationship The same image coordinate combination is grouped together, and images with four collinear coordinates are grouped together. , with three images with collinear coordinates defined as a group The common image coordinates of the two sets of image coordinates are the intersection of the lines connecting the two sets of special markers. (i.e. the intersection point of two linear equations in two variables);

[0039] (3) Compare the absolute distances of the intersection points to the coordinates of other images in the two groups. The absolute distance between the intersection point and the image coordinates, from small to large Rearrange the image coordinate order and compare The absolute distance between the intersection point and the image coordinates, from small to large The image coordinates are rearranged in sequence. After the image coordinates are rearranged, it can be regarded as realizing a one-to-one correspondence between the image coordinates of the special marker points and the local coordinates of the special marker points on the transparent calibration plate, forming a new set of special marker point image coordinates.

[0040] (4) According to the calculation rules of cross ratio and simple ratio, solve the cross ratio and simple ratio to verify the correctness of the arrangement of the special mark point image coordinates. For the four collinear image coordinates on the same straight line, solve the cross ratio. , for the three collinear image coordinates on the same straight line, solve the simple ratio , if the cross ratio Compare with target , Jianbi Compared with the target Equal (target cross ratio , target simplicity The matching is successful if the cross ratio and simple ratio are the ideal state calculated according to the calibration pattern.

[0041] If the two sets of linear equations in two variables have four slopes respectively Special marking points of direction are collinear, 3 on the slope The special landmarks in the direction are collinear, and the four are on the slope The special landmarks of the direction satisfy the cross ratio invariance, and the three points on the slope If the special landmarks in the direction satisfy the invariance of the cross ratio, it is considered that the arrangement pattern of all special landmarks is successfully recognized, such as the cross ratio Compare with target Difference, simple ratio Compared with the target If the absolute value of the difference is less than 0.001, it is considered that the arrangement pattern of all special landmarks is successfully recognized; after the matching is successful, based on the ellipse fitting results of all special landmarks, the mean of the major axis of the small circle, the mean of the circumference of the small circle, and the mean of the circumference of the large circle are calculated and recorded as 、 、 .

[0042] Part III: Measurement system using the target and calibration method of the present invention

[0043] The measurement system includes: a microwave radar for measuring the radial displacement of the target; an industrial camera for collecting images of the target and measuring the in-plane displacement; a computer for processing radar signals and image data and calculating three-dimensional displacement; the target is set at an observation point.

[0044] The measuring system realizes three-dimensional displacement measurement by the following method:

[0045] Measuring the radial displacement of the target by microwave radar;

[0046] Multiple target images are captured by an industrial camera, and a transparent calibration plate is identified using a calibration method. The recognition results of each calibration image are summarized, and the Zhang Zhengyou calibration method and stereo calibration principle are used to perform internal and external parameter calibration. The three-dimensional displacement of the target is calculated.

[0047] The beneficial effects obtained by the present invention are:

[0048] Based on the present invention, microwave radar and computer vision can realize three-dimensional displacement measurement of the target to be measured.

[0049] The proposed self-luminous target fills the gap in displacement measurement targets that can be used for both microwave radar and computer vision fusion. Furthermore, because it uses an infrared light source, it can still ensure that industrial cameras can obtain high-quality, stable images in drastically changing light fields when used outdoors.

[0050] The proposed dot matrix transparent calibration plate contains multiple unique markers that can be combined to form a specific pattern structure that satisfies the principle of projective invariance. Through a highly robust recognition algorithm, the transparent calibration plate can stably identify specific patterns in digital images with complex background noise. Incorporating the principle of projective invariance in projective transformations, the system can calculate and match cross-ratios, thereby achieving accurate recognition of the transparent calibration plate.

[0051] The invention has a clear scientific principle, a simple manufacturing process, an easy-to-implement recognition algorithm, and high robustness. It can stably and quickly identify the dot matrix transparent calibration plate, even in calibration images with complex background noise, and obtain highly accurate calibration results. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a schematic front view of the self-luminous target of the present invention;

[0053] Figure 2 This is a cross-sectional view of the main body of the self-luminous target of the present invention;

[0054] Figure 3 This is an interior view of the main body of the self-luminous target of the present invention;

[0055] Figure 4 This is a schematic diagram of the effect of installing an infrared light source in the target cavity of the present invention when used at night;

[0056] Figure 5 This is a schematic diagram of the effect of using the target external infrared light source of the present invention at night;

[0057] Figure 6 It is a schematic diagram of the transparent calibration plate and the etched calibration pattern of the present invention;

[0058] Figure 7 1 is a schematic diagram of a dot matrix calibration pattern based on cross ratio invariance and simplicity ratio invariance according to a preferred embodiment of the present invention;

[0059] Figure 8 1 is a schematic diagram of a dot matrix calibration pattern recognition process based on cross ratio invariance and simplicity ratio invariance according to a preferred embodiment of the present invention;

[0060] Figure 9 This is a schematic diagram of the recognition effect of a dot matrix calibration pattern based on cross ratio invariance and simplicity ratio invariance according to a preferred embodiment of the present invention;

[0061] Figure 10 This is a schematic diagram of the effect of the present invention when installed in actual use;

[0062] Figure 1 、 Figure 2 and Figure 7 The marks in the middle represent: 1 corner reflector, 2 infrared self-luminous device, 3 reserved bolt hole, 4 reserved power connection hole, 5 fixing buckle, 6 heat sink, 7 transparent calibration plate, 8 uniform light diffuse reflection plate, 9 calibration pattern, 10 special mark point, 11 ordinary mark point. DETAILED DESCRIPTION

[0063] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings and technical solutions.

[0064] Figure 1-3 Schematic diagram of the structure of the self-luminous target of the present invention.

[0065] During the target assembly process, first assemble three aluminum plates into a triangular pyramid shape, fix the three aluminum plates by welding, and after completing the production of the corner reflector 1, install the infrared self-luminous device 2 on the aluminum plate with the reserved bolt hole 3 inside the shell of the corner reflector 1, and install the radiator 6 on the aluminum plate with the reserved bolt hole 3 outside the corner reflector 1. Fix the three with bolts and lead the power cord through the reserved power connection hole 4. After the transparent calibration plate 7 is tightly fitted with the uniform light diffuse reflection plate 8, fix the uniform light diffuse reflection plate 8 with the fixing buckle 5. After completing the assembly, fix it on the surface of the object to be measured and power it on before use.

[0066] 1. Corner reflector

[0067] Considering that aluminum has good reflective properties for microwaves, it can effectively reflect radar signals and improve detection accuracy, as well as the characteristics of aluminum being lightweight, corrosion-resistant, easy to process and low-cost. Experimental tests have shown that the corner reflector 1 with a triangular pyramid structure can achieve a good reflection effect on microwaves emitted by microwave radar. Therefore, the shell of the corner reflector 1 of the present invention is composed of three aluminum plates, which are an important component of the target of the present invention and are used to support and install other components. The three aluminum plates are selected from right-angled isosceles triangles of equal size and fixed into a triangular pyramid structure (the remaining one surface is used to install the transparent calibration plate 7), and the cavity structure in the middle is used to install the self-luminous component. A fixing buckle 5 is provided on the corner reflector 1, and the fixing buckle 5 is used to facilitate the disassembly of the transparent calibration plate 7.

[0068] 2. Transparent calibration plate

[0069] Taking into account the characteristics of PVC board, such as light weight, high strength, corrosion resistance, easy processing and good plasticity, it is convenient to manufacture a high-precision transparent calibration plate 7 containing ordinary marking points 11 and special marking points 10. Therefore, the transparent calibration plate 7 is made of PVC material, and the calibration pattern 9 etched on the PVC transparent plate and its calibration method are introduced as the second part. Taking into account that the displacement detection accuracy of digital image correlation technology (DIC) is mainly affected by the image texture characteristics, its working principle includes the following key processes: first, the grayscale distribution characteristics of the reference area are determined in the image before the object is deformed, and then the matching calculation is performed on the area with similar grayscale characteristics in the image after the object is deformed. By quantifying the spatial difference between the original coordinates of the reference area and the new coordinates of the matching area, the pixel-level displacement of the object under test can be analyzed. In this process, the richer the texture information contained in the image, the more significant the grayscale gradient characteristics, and the more conducive to achieving high-precision displacement calculation. Therefore, the area outside the calibration pattern 9 etched on the transparent calibration plate 7 is supplemented with a pattern with rich texture information, such as Figure 6 As shown. Considering the need for projective invariance through circular markers during recognition, the calibration pattern 9 must be etched with micron-level precision to ensure accuracy. Furthermore, to achieve better results when using the transparent calibration plate 7, the pattern information on the PVC board is processed with reflective paint, and an infrared filter is installed on the transparent calibration plate 7 to ensure that the camera can capture high-quality images when illuminating the pattern with an infrared device.

[0070] 3. Infrared self-luminous device

[0071] The present invention forms a measurement benchmark by etching high-precision common markers 11 and special markers 10 on a transparent calibration plate 7. In practical applications, directly illuminating the transparent calibration plate 7 with a front light source requires considerable time to adjust the light source to ensure complete illumination of the pattern. Furthermore, due to refraction and reflection of light, the marker images are prone to shadows, affecting the accuracy of marker center extraction. To ensure ease of use and accurate marker center extraction, an infrared self-luminous device 2 is built into the target to ensure that the markers on the transparent calibration plate 7 can be accurately identified.

[0072] When using direct infrared self-luminous device 2, the illumination intensity is uneven, reducing the imaging quality of the special marker 10 and affecting the positioning accuracy of the special marker 10. Therefore, a uniform light diffuse reflector 8 (also called a diffuser, with a transmittance of 60% to 80%) is required. The uniform light diffuse reflector 8 is placed in close contact with the transparent calibration plate 7. Its diffusion effect evenly illuminates the transparent calibration plate 7, ensuring that the illuminated surface contains no dark areas and no residual images are formed on the image. This effectively improves the camera imaging effect and further enhances the accuracy of visual measurement. The uniform light diffuse reflector 8 designed in the present invention is bonded to the transparent calibration plate 7. In actual application, during the calibration phase, only the infrared self-luminous device 2 built into the target cavity can be used to illuminate the transparent calibration plate 7 to avoid interference from complex texture patterns. During the displacement detection phase, infrared light is used to illuminate the target. During this process, the infrared light is filtered by an infrared filter, illuminates the reflective coating, and is ultimately reflected by the industrial camera, thereby obtaining high-quality images.

[0073] 4. Heat dissipation device

[0074] When the infrared self-luminous device 2 is in use, it generates a lot of heat. If the heat is not dissipated in time, it will affect the service life of the lamp beads. Therefore, the radiator 6 is closely connected to the infrared self-luminous device 2. Aluminum plate is a commonly used heat dissipation plate with a thermal conductivity of , low cost, good mechanical properties, and can be processed into a variety of complex shapes to increase the heat dissipation area. Therefore, the first part of the present invention considers closely connecting the infrared self-luminous device 2 with the corner reflector 1 to quickly conduct heat, and placing a heat sink 6 on the outer shell of the corner reflector 1 corresponding to the infrared self-luminous device 2 to quickly dissipate the heat conducted by the corner reflector 1 outside the target.

[0075] The first part of the present invention achieves unified targets for microwave radar displacement measurement and computer vision displacement measurement by assembling a corner reflector 1 with a transparent calibration plate 7. By using an infrared self-luminous device 2 to illuminate a uniform diffuse reflector 8, the present invention ensures uniform illumination of a calibration pattern 9 on the transparent calibration plate 7, improving the image quality of the calibration pattern 9 on the transparent calibration plate 7 in the captured image. Furthermore, the first part of the present invention can be used immediately after being installed on the object to be measured and powered on. The overall device is simple to install, portable, and reliable.

[0076] With respect to the second part of the invention, this part of the invention is described in detail below in conjunction with specific embodiments.

[0077] Reference Figure 7 Figure 2 shows a schematic diagram of a dot matrix transparent calibration plate 7 based on cross-ratio invariance according to a preferred embodiment of the present invention. The calibration pattern 9 of the transparent calibration plate 7 includes at least six special markers 10, wherein the distribution of all special markers 10 forms a pattern structure with cross-ratio information. The pattern structure of the transparent calibration plate 7 in the digital image is identified by using the cross-ratio invariance and simple ratio invariance in projective transformations, and the cross-ratio values ​​are calculated and matched to accurately identify the calibration pattern 9 of the transparent calibration plate 7. The calibration pattern 9 also includes uniformly distributed common markers 11; the common markers 11 and the special markers 10 are arranged in an array at known local coordinates. As a preferred embodiment, the common markers 11 and the special markers 10 are arranged in a matrix.

[0078] In actual scenarios, the radius of the marker point can be adjusted according to actual needs. The following radius is selected only for demonstration examples;

[0079] The horizontal and vertical spacing between the marker points in the dot matrix is ; The special mark point 10 in the dot matrix is ​​a circular double concentric circle, and the radius of the small circle of the circular double concentric circle is , the radius of the great circle is ; Ordinary mark point 11 is a solid circle, the radius of the solid circle .

[0080] In other preferred embodiments, the special marking point 10 is a circular double concentric circle, and the radius of the small circle of the circular double concentric circle is ; Great circle radius ; Where g is the horizontal and vertical spacing of the dot matrix.

[0081] Ordinary mark point 11 is a solid circle, the radius of the solid circle is The special marking points 10 in the dot matrix are clearly distinguishable from the ordinary marking points 11, but the marking point forms are not unique. For example, the ordinary marking points 11 can be solid circles, while the special marking points 10 can be annular with multiple concentric circles. The special marking points 10 can also be non-circular.

[0082] In other preferred embodiments, the local coordinates of the overall arrangement of the dot matrix must be known in advance for all the marker points in the dot matrix. The distribution requirements of the special marker points 10 are as follows: the distribution of all the special marker points 10 cannot be arranged in a straight line; the overall distribution of all the special marker points 10 is an asymmetric arrangement; among all the special marker points 10, one special marker point 10 is selected as the intersection of two sets of linear equations with different slopes, and the remaining special marker points 10 extend outward toward the two selected sets of slopes with the intersection as the origin. The pattern structure with cross-ratio information formed by the distribution of all the special marker points 10 has specific cross-ratio information, and the cross-ratio information is designated as the target cross-ratio of the transparent calibration plate 7.

[0083] In other preferred embodiments, the cross-ratio of the transparent calibration plate 7 is The calculation rules are the same as those explained in the summary of the invention.

[0084] In other preferred embodiments, the color of the dot matrix is ​​different from the background color of the transparent calibration plate 7 and has a clear contrast.

[0085] In another embodiment, a calibration method of a dot matrix transparent calibration plate 7 is provided, which is performed using the above-mentioned dot matrix transparent calibration plate 7 based on cross ratio invariance.

[0086] In other preferred embodiments, the calibration method of the dot matrix transparent calibration plate 7 is the same as the rules explained in the summary of the invention.

[0087] The above-mentioned dot matrix transparent calibration plate 7 is identified in the simulation scene, referring to Figure 4 As shown, a dot matrix transparent calibration plate 7 with 6 special marking points 10 is placed in a complex background noise interference scene. It can be seen from the figure that the calibration method using the above-mentioned dot matrix transparent calibration plate 7 can successfully identify the dot matrix transparent calibration plate 7 under complex background noise interference.

[0088] Figure 5 In order to use the target of the present invention with an external infrared light source for illumination at night, the results show that the calibration method using the above-mentioned dot matrix transparent calibration plate can successfully identify the dot matrix transparent calibration plate 7 under complex background noise interference.

[0089] In a specific embodiment, a dot matrix transparent calibration plate is provided.

[0090] Reference Figure 7 As shown, the background of the dot matrix transparent calibration plate 7 is white, the dot matrix is ​​black, and the dot matrix consists of common marking points 11 and 6 special marking points 10 evenly arranged in a matrix form.

[0091] The number of dot matrix rows is h = 9, the number of columns is w = 12, the horizontal and vertical spacing of the dot matrix is ​​g, the unit of g is pixel, and its value can be flexibly selected according to the image resolution requirements. .

[0092] The 6 special marking points 10 are double concentric circles. The radius of the small circle of the double concentric circles is , the radius of the great circle is .

[0093] Ordinary landmark point 11 is a solid circle with a radius of . Take the mark point on the upper left corner of the transparent calibration plate 7 as the origin and establish the local coordinate system ,in ∈[0 ,8], ∈[0 , 11], and The domain of definition covers all the landmarks in the lattice; in the local coordinate system In this case, the local coordinates of the six special landmarks 10 (circular double concentric circles) are , ∈[1 ,2 ,…,6], it is stipulated that P1=L(0 ,0), P2=L(0 ,2), P3=L(0,8), P4=L(0 ,11), P5=L(5 ,0), P6=L(8,0); It should be noted that although the form of the above-mentioned transparent calibration plate 7 is not unique, the above-mentioned specific transparent calibration plate 7 can already meet the calibration requirements in most cases.

[0094] Determine parameters w, h, g and special landmarks After obtaining the local coordinates of the transparent calibration plate 7, the transparent calibration plate 7 can be uniquely determined at the digital image level. Although the form of the transparent calibration plate 7 is not unique, it should have the following three characteristics:

[0095] ① The parameters w, h, and g can be selected flexibly, but it is necessary to ensure that the remaining special landmarks 10 are not equidistantly distributed on the two intersecting straight lines;

[0096] ② The arrangement of the special marking points 10 must be asymmetrical, that is, in the local coordinate system In the figure, the local features of the special landmark 10 do not have axisymmetric and centrosymmetric forms;

[0097] ③. The actual size of the transparent calibration plate 7 is determined by the printing resolution of the printer.

[0098] The constant characteristic parameter of the transparent calibration plate 7 is its target cross ratio. The target cross ratio of the dot matrix transparent calibration plate 7 is The calculation rules are the same as those of the cross ratio of the invention. Figure 7The specific transparent calibration plate 7, its target cross ratio , target simplicity , are dimensionless constants.

[0099] It should be noted that the specific dot matrix transparent calibration plate 7 provided above only adopts the technical principle of the present invention, and is not the only concrete form of expression of the present invention.

[0100] Based on the above-mentioned dot matrix transparent calibration plate 7, the following calibration method is adopted, referring to Figure 8 The specific steps are as follows.

[0101] S10: Use an industrial camera to capture a calibration image of the dot matrix transparent calibration plate 7. In actual operation, multiple calibration images of the transparent calibration plate 7 at different positions and angles are captured. Generally, more than 20 image samples are collected.

[0102] S20: Convert the calibration image into a single-channel 8-byte grayscale image and perform binarization processing to obtain a binary image; perform contour extraction on the binary image and establish a complete contour hierarchical topology structure.

[0103] S30: traverse the contour hierarchical topology structure, find the contour without sub-contours inside, and store the corresponding contour number into the meta-contour number set N, that is, establish the meta-contour number set N.

[0104] S40: Traverse the element contour number set N, and let the elements be , if The parent contour exists and the number of the first child contour of the parent contour is equal to , then for the Ellipse fitting is performed on the element contour and its parent contour. If the ellipse fitting result satisfies the following three conditions at the same time: ①, the distance between the centers of the two ellipses is less than 3 pixels; ②, the ratio of the major axis of the parent ellipse to the major axis of the child ellipse is within the range [1.5, 4.5]; ③, the ratio of the perimeter of the parent ellipse to the perimeter of the child ellipse is within the range [1.5, 4.5]; at this time, it is considered that one special marker point has been successfully identified. Until 6 special marker points are successfully identified, the image coordinates of each special marker point are recorded, and the array is recorded as , whose elements are recorded as , , , , , Etc., n∈[1, 2, …, 6], exit the traversal. In this step, the ellipse perimeter is replaced by the number of points that compose the contour of the fitted ellipse in the actual calculation.

[0105] S50: Match

[0106] (1) As the vertex, calculate the remaining image coordinates in the same way The linear relationship between the lines (i.e. the slope-intercept equation ,by and Take two points as an example. , where k is the slope and b is the intercept), repeat this step until the linear relationship between all elements is found; The coordinates of , ), The coordinates of , ).

[0107] (2) The slope of the linear relationship The same image coordinate combination is grouped together, and images with four collinear coordinates are grouped together. , with three images with collinear coordinates defined as a group The common image coordinates of the two sets of image coordinates are the intersection of the lines connecting the two sets of special markers. (i.e. the intersection point of two linear equations in two variables);

[0108] (3) Compare the absolute distances of the intersection points to the coordinates of other images in the two groups. The absolute distance between the intersection point and the image coordinates, from small to large Rearrange the image coordinate order; compare The absolute distance between the intersection point and the image coordinates, from small to large After the image coordinates are rearranged, the order of the image coordinates can be regarded as realizing a one-to-one correspondence between the image coordinates of the special mark point and the local coordinates of the special mark point on the transparent calibration plate 7. Finally, The order of the special marking points is ( ), The order of the special marking points is ( ), the rearranged special marker points are combined into a new special marker point image coordinate set ( );

[0109] (4) According to the calculation rules of cross ratio and simple ratio, solve the cross ratio and simple ratio, and verify the correctness of the arrangement of the special mark point image coordinates. For the four collinear image coordinates on the same straight line, solve the cross ratio. ; For the three collinear image coordinates on the same straight line, solve the simple ratio .

[0110] For the four collinear special landmarks, the intersection ratio is calculated according to the following formula:

[0111] ;

[0112] For the three special landmark points on the same line, calculate the simple ratio according to the following formula:

[0113] ;

[0114] in, , , , are four special landmark points on the same collinear line with the same slope. , , These are three special landmark points on the same collinear line with the same slope; Represents the point Pointing Point The length of the directed line segment; Represents the point Pointing Point The length of the directed line segment; Represents the point Pointing Point The length of the directed line segment; Represents the point Pointing Point The length of the directed line segment; Represents the point Pointing Point The length of the directed line segment; Represents the point Pointing Point The length of the directed line segment.

[0115] If the cross ratio , Jianbi Compare with target , target simplicity If the absolute value of the difference is less than 0.001, it is considered that the arrangement pattern of the 6 special landmarks is successfully recognized and the cross-ratio matching is successful.

[0116] After the matching is successful, the ellipse fitting results of the six special landmark points are used to calculate the mean of the major axis of the small circle, the mean of the circumference of the small circle, and the mean of the circumference of the large circle, which are recorded as 、 、 In this step, the circumference of the ellipse is replaced by the number of points that make up the contour of the ellipse in actual calculations.

[0117] S60: Solve the local coordinates of special landmark points Mapped to the perspective transformation matrix W; based on local coordinates , use W to calculate the projection points of each local coordinate in the image coordinate system .

[0118]

[0119] in ∈[0 ,1 ,…,8], ∈[0 ,1 ,…,11].

[0120] S70: Traverse , in the element contour number set N for each projection point Find the corresponding meta-contour, if the following two conditions are met at the same time: ①, the center of the meta-contour fitting ellipse is The spacing is less than ②, the perimeter of the element contour fitting ellipse is between 、 between; at this time it is considered The corresponding meta-contour is matched, and the center of the ellipse fitted by the meta-contour is recorded to form the final transparent calibration plate 7-point matrix recognition result set.

[0121] S80: Run the transparent calibration plate 7 calibration method of S20 to S70 for each calibration image obtained in S10, summarize the recognition results of each calibration image, and use the Zhang Zhengyou calibration method and stereo calibration principle to calibrate the intrinsic parameters of each camera and the extrinsic parameters of the posture between the left and right binocular cameras.

[0122] It should be specifically explained that the self-luminous target described in the first part of the present invention is achieved by assembling an aluminum plate with extremely high reflectivity to microwave radar signals into a corner reflector 1 and then installing a transparent calibration plate 7 thereon, so that the target can be used for both microwave radar and computer vision technology to measure displacement. With respect to the second part of the present invention, the transparent calibration plate 7 and the calibration method thereof are mainly used for industrial camera calibration. The main reason is that the pictures collected by industrial cameras have the characteristic of low distortion, and the calibration method included in the present invention can be implemented efficiently and stably, and the cross-ratio parameters can be maintained almost unchanged. It should be added that when recognizing highly distorted images, the dot matrix transparent calibration plate 7 can still be recognized by adjusting the relevant thresholds in the calibration method.

[0123] Figure 9 The images before and after recognition in a test scenario are compared to show that the recognition results of special landmarks are accurate.

[0124] Figure 10This is a schematic diagram of the effect of the installation of the measurement system of the present invention. The measurement system includes a microwave radar, an industrial camera, a target of the present invention, and a computer for processing data. The microwave radar is used to transmit microwave signals, the industrial camera is used to collect images of the target, and the computer is used to process the microwave radar signal data and image data. By comparing the microwave radar signals and image data at different times, the three-dimensional displacement information of the target at different times is obtained, wherein the target is set at the observation point. An infrared lamp outside the target can also be set to further enhance the recognition ability. The infrared self-luminous device 2 inside the target is connected to an external power supply via an electric wire.

[0125] The measurement system process is as follows:

[0126] Obtaining the radial displacement of the target through microwave radar: obtaining the radial position information of the target in the initial state and the current radial position information, and obtaining the radial displacement information of the target by comparing the current position information with the position information in the initial state.

[0127] Obtain the in-plane displacement of the target through computer vision: Use an industrial camera to obtain the image information of the target in its initial state and the image information of the target at the current moment, and summarize the recognition results of each image.

[0128] Finally, the radial and in-plane displacement information at the same moment is fed into a computer, where it is calibrated using the Zhang Zhengyou calibration method and stereo calibration principles to perform internal and external calibrations. The displacement change information for the current observation point is then calculated. This summary reveals the motion patterns of the observation point over a specific time period.

[0129] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A target for displacement measurement by integrating microwave radar and computer vision, characterized in that: It includes a corner reflector, a transparent calibration plate, and a uniform light diffuse reflector. The specific positions and connection relationships are as follows: A transparent calibration plate is provided on the open surface of the corner reflector. A self-luminous component is provided in the cavity between the transparent calibration plate and the corner reflector. A uniform light diffuse reflection plate is provided between the transparent calibration plate and the self-luminous component. The power cord of the self-luminous component is connected to an external power supply through a reserved hole in the corner reflector. The transparent calibration plate is a dot matrix transparent calibration plate, and a calibration pattern is etched on the transparent calibration plate. The calibration pattern is a mark point, and the mark points are divided into ordinary mark points and special mark points. The special mark points and the ordinary mark points are arranged in an array form under known local coordinates; there are at least 6 special mark points, and among all the special mark points, one of the special mark points is selected as the intersection point of two groups of collinear special mark points with different slopes. The remaining special mark points extend outward according to the selected slope with the intersection point as the reference point, and the interval distance between all special mark points is different.

2. The target for displacement measurement by integrating microwave radar and computer vision according to claim 1, characterized in that: The transparent calibration plate is made of PVC board; the transparent calibration plate has a pattern with rich texture information in the area other than the calibration pattern.

3. The target for displacement measurement by integrating microwave radar and computer vision according to claim 1, characterized in that: The uniform light diffuse reflection plate is attached to the inner surface of the transparent calibration plate, the self-luminous component is an infrared light source, and an infrared filter is attached to the outer surface of the transparent calibration plate.

4. The target for displacement measurement by integrating microwave radar and computer vision according to claim 1, characterized in that: The special marking point is a circular double concentric circle; the ordinary marking point is a solid circle; the background color of the transparent calibration plate and the color of the marking point have an obvious contrast.

5. The target for displacement measurement by integrating microwave radar and computer vision according to claim 1, characterized in that: The corner reflector is composed of three equal-sized right-angled isosceles triangular aluminum plates, one of which is provided with two holes for fixing the self-luminous component, and a radiator is provided on the outer surface of the aluminum plate for fixing the self-luminous component.

6. The target calibration method according to claim 1, characterized in that: The following steps are involved: (1) Image acquisition: Use an industrial camera to capture the calibration image containing the dot matrix transparent calibration plate; (2) Image processing: converting the calibration image into a single-channel 8-byte grayscale image and performing binarization processing to obtain a binary image; performing contour extraction on the binary image and establishing a complete contour hierarchical topological structure; (3) Based on the contour hierarchical topological structure, the image coordinates of all special landmarks in the calibration image are extracted, and the linear relationship between the special landmarks is calculated. The groups are summarized according to the slope, the intersection points are solved, and the intersection ratio and the simple ratio are verified. If the calculated intersection ratio is equal to the target intersection ratio, the target Equality, simple ratio and target simple ratio If they are equal, the match is successful; (4) Projection matrix calculation: Calculate the perspective transformation projection matrix based on the local coordinates of special landmarks and their corresponding image coordinates; (5) Projection result acquisition: Based on the obtained perspective transformation projection matrix, all the marker points in the dot matrix are projected to the image coordinate system according to the local coordinates to obtain the projection results of all the marker points in the dot matrix in the image coordinate system; (6) Recognition result formation: Based on the projection results, the real image coordinates of all landmark points are obtained to form the final transparent calibration plate dot matrix recognition result set.

7. The target calibration method according to claim 6, characterized in that: The specific operation of step (3) is as follows: first, extract the contour from the original image and construct a hierarchical topological structure of the contour; then, traverse the structure to find all contours without internal sub-contours, and store the numbers of these contours in a set N; traverse each element in the set N, assuming that the current element is the i-th meta-contour; if the meta-contour has a parent contour, and the first sub-contour number of the parent contour is equal to i, then perform ellipse fitting on the i-th meta-contour and its parent contour; determine whether it is a special landmark point from the fitting result; repeat the above process until all special landmark points are identified, and record the image coordinates of each special landmark point; match: (1) Taking one of the special landmark points as the reference, calculate the linear relationship between the remaining image coordinates and the line connecting this point; (2) The image coordinates with the same linear relationship slope are combined and summarized, and the image coordinates with four collinear images are defined as a group. , with three images with collinear coordinates defined as a group The common image coordinates of the two sets of image coordinates are the intersection of the lines connecting the two sets of special markers. ; (3) Comparison of intersection points The absolute distance to the pixel coordinates of other images in the two groups, from small to large Rearrange the image coordinate order; compare The absolute distance between the intersection point and the image coordinates, from small to large Rearrange the order of image coordinates; (4) For the four collinear image coordinates on the same straight line, solve the cross ratio ; For the three collinear image coordinates on the same straight line, solve the simple ratio .

8. A microwave radar and computer vision fusion displacement measurement system based on the target of claim 1, comprising: Microwave radar, used to measure the radial displacement of the target; Industrial cameras, used to capture images of targets and measure in-plane displacements; A computer for processing radar signals and image data and calculating three-dimensional displacement, characterized in that the target according to claim 1 is used and the target is set at an observation point.

9. The measurement system according to claim 8, characterized in that The measuring system realizes three-dimensional displacement measurement by the following method: Measuring the radial displacement of the target by microwave radar; Capturing multiple target images with an industrial camera, and identifying the transparent calibration plate using the calibration method described in claim 6; summarizing the recognition results of each calibration image, and performing internal reference calibration and external reference calibration using the Zhang Zhengyou calibration method and stereo calibration principle; Calculate the three-dimensional displacement of the target.

Images