Monocular camera and laser range finder three-dimensional fusion measurement and target center extraction method

Abstract

The invention relates to the technical field of precision measurement and positioning, and discloses a monocular camera and laser range finder three-dimensional fusion measurement and target center extraction method, which comprises the following steps: S1, calibrating an internal reference of a monocular camera, and obtaining a principal point pixel coordinate and a focal length; s2, calibrating external parameters of the laser range finder, and obtaining coordinates of a light source point of the laser range finder in a camera coordinate system and a laser direction vector; s3, designing a common target; s4, shooting the target through a monocular camera, and extracting a two-dimensional image plane coordinate of a return light reflection point; s5, designing a common target, determining the geometric constraint of the common target according to the geometric characteristics of the common target, designing a target recognition algorithm, eliminating interference points, extracting return light reflection points on the common target, solving the center image plane coordinates of the common target, calculating the geometric constraint between the return light reflection points according to the geometric characteristics of the target, and recognizing the target. Screening candidate point groups, and extracting two-dimensional image plane coordinates of the center of the target through a fitting algorithm; and S6, measuring the distance information of the center of the target by using a laser range finder. By fusing the two-dimensional image plane coordinates and the distance information, the three-dimensional coordinates of the center of the target are accurately calculated, and the efficient and reliable space positioning effect is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of precision measurement and positioning, and in particular to a method for three-dimensional fusion measurement of a monocular camera and a laser rangefinder and for extracting a target center. Background Art

[0002] With the continuous development of aerospace technology, large antennas of spacecraft are gradually developing in the direction of ultra-large and ultra-high precision. After the antenna is deployed in orbit, it is large in scale and flexibility, and operates in a complex space environment for a long time. It is easily deformed and vibrated by factors such as high and low temperature changes in space, micro-vibrations, and solar light pressure. This dynamic change directly affects the surface maintenance accuracy of the antenna panel, thereby reducing the working performance of the antenna. As the size of the antenna continues to increase, the accuracy requirements for its on-orbit surface maintenance have exceeded the limits of existing material performance. Therefore, studying a high-precision on-orbit antenna surface measurement method that can monitor the antenna surface in real time and compensate for data has become the key to solving this problem.

[0003] At present, the commonly used methods for the three-dimensional measurement of large structures include structured light measurement, microwave measurement, photogrammetry and laser measurement. These methods have their own advantages and disadvantages, but they all have certain limitations in the high-precision on-orbit measurement of large spacecraft antennas:

[0004] Structured light measurement: By projecting a structured light pattern onto the surface of an object and capturing the deformation of the pattern, the three-dimensional topography of the object surface is calculated. However, the range of structured light measurement is limited, and the complex reflection characteristics of large antenna surfaces will interfere with the capture of structured light patterns, thereby affecting measurement accuracy.

[0005] Microwave measurement: widely used in deformation measurement of large structures, such as bridges, slopes, etc., with good measurement range and accuracy (sub-millimeter level). However, microwave measurement devices are large in size, heavy in weight, high in power consumption, and cannot achieve absolute measurement. Three-dimensional solution is difficult, so it is difficult to apply to on-orbit measurement of large antennas.

[0006] Photogrammetry: It is currently widely used in ground and on-orbit measurement tasks of large space structures, such as 3D measurement of space telescopes, solar sails, and antennas. Binocular vision measurement can accurately obtain 3D information of objects without contact, but its measurement accuracy and range are limited by the baseline distance, and it is difficult to meet the ground measurement needs of large antennas. Monocular vision measurement has the characteristics of small size and strong flexibility, and is suitable for various scenes from small objects to large buildings, but due to the lack of direct depth information, it cannot independently achieve high-precision 3D spatial positioning.

[0007] Laser ranging: As an optical measurement method, laser ranging can provide absolute measurement information, with high measurement frequency and high accuracy, especially in the line of sight. However, laser ranging cannot achieve full-field synchronous measurement and can only provide single-dimensional distance data. It needs to be combined with other sensors to complete the three-dimensional measurement of objects.

[0008] In summary, it is difficult for existing technologies to simultaneously meet the needs of high-precision on-orbit surface measurement of large spacecraft antennas. Summary of the invention

[0009] In view of the shortcomings of the prior art, the present invention provides a method for three-dimensional fusion measurement of a monocular camera and a laser rangefinder and extraction of a target center, which solves the problem of difficulty in real-time monitoring and compensation of antenna surfaces of large spacecraft caused by insufficient measurement accuracy, unstable data fusion and poor equipment applicability in the on-orbit measurement of antenna surfaces of large spacecraft in the prior art.

[0010] To achieve the above objectives, the present invention is implemented by the following technical scheme: a method for three-dimensional fusion measurement of a monocular camera and a laser rangefinder and extraction of a target center, comprising the following steps:

[0011] S1, calibrate the internal parameters of the monocular camera to obtain the pixel coordinates and focal length of the principal point;

[0012] S2, calibrate the external parameters of the laser rangefinder, and obtain the coordinates of the laser rangefinder light source point in the camera coordinate system and the laser direction vector;

[0013] S3. Design shared targets;

[0014] S4, photographing the target with a monocular camera and extracting the two-dimensional image coordinates of the reflected light point;

[0015] S5. Perform a first screening based on the geometric features of the common target, then use the least squares method to fit an ellipse, analyze the error of the fitted ellipse, and perform a second screening on the point group to eliminate interference points and extract the return light reflection points on the common target, and then calculate the coordinates of the central image plane of the common target;

[0016] S6. Using a laser rangefinder to measure the distance information of the target center;

[0017] S7, combining the laser ranging value and the two-dimensional image plane coordinates of the target center, constructing a fusion measurement model, and calculating the three-dimensional coordinates of the target center in the camera coordinate system;

[0018] S8. Convert the three-dimensional coordinates of the target center from the camera coordinate system to the world coordinate system.

[0019] Preferably, the internal parameters for calibrating the monocular camera in S1 include:

[0020] Multiple sets of images are obtained through the calibration board, and the camera intrinsic parameter matrix is ​​calculated. The intrinsic parameter matrix includes the focal length and the pixel coordinates of the principal point.

[0021] Preferably, the calibrating the external parameters of the laser rangefinder in S2 includes:

[0022] Determine the three-dimensional coordinates of the laser rangefinder light source point in the camera coordinate system;

[0023] Get the laser direction vector;

[0024] The laser beam equation is represented by the coordinates of the source point and the direction vector.

[0025] Preferably, the targets in S2 include:

[0026] Multiple light reflection points are used to extract the two-dimensional image plane coordinates of the monocular camera;

[0027] A corner cube prism for distance measurement in a laser rangefinder;

[0028] The reflected light points are distributed at the vertices of the regular polygon, and the vertex of the corner cube coincides with the center of the regular polygon.

[0029] Preferably, extracting the two-dimensional image plane coordinates of the return light reflection point in S4 includes:

[0030] The weighted grayscale centroid method is used to calculate the two-dimensional image coordinates of the reflected light point;

[0031] The pixel coordinates of each reflected light point are extracted according to the gray value distribution.

[0032] Preferably, the step of calculating the two-dimensional image plane coordinates of the target center in S5 includes:

[0033] By analyzing the geometric constraints between the return light reflection points, the candidate point group is screened;

[0034] The least squares fitting algorithm is used to extract the two-dimensional image coordinates of the target center.

[0035] Preferably, the three-dimensional coordinates of the target center in S7 are calculated by a fusion measurement model, and the fusion measurement model includes:

[0036] The laser rangefinder provides distance information to the center of the target;

[0037] Construct the direction equation of the laser beam and the direction equation of the camera light;

[0038] The three-dimensional coordinates of the target center are solved by the shortest distance point method.

[0039] Preferably, the fused measurement model in S7 satisfies the following geometric constraints:

[0040] The laser direction vector and the camera light direction vector at the target center have a relationship with the minimum vertical error;

[0041] The intersection point of the laser beam and the camera light is taken as the three-dimensional coordinate point of the target center.

[0042] Preferably, when the three-dimensional coordinates of the target center in S8 are converted from the camera coordinate system to the world coordinate system, an external parameter matrix conversion is adopted, including a rotation matrix and a translation matrix.

[0043] A measuring device, comprising:

[0044] A monocular camera is used to obtain the two-dimensional image coordinates of the target's reflected light point;

[0045] Laser rangefinder, used to measure the distance information of the target center;

[0046] A common target including a plurality of retro-reflection points and a corner cube prism;

[0047] The calculation module is used to fuse the laser ranging data and the two-dimensional image plane coordinate data to calculate the three-dimensional coordinates of the target center.

[0048] The invention provides a method for three-dimensional fusion measurement of a monocular camera and a laser rangefinder and for extracting a target center.

[0049] It has the following beneficial effects:

[0050] 1. The present invention achieves efficient and reliable spatial positioning by integrating the two-dimensional image plane coordinates and distance information to accurately calculate the three-dimensional coordinates of the target center. Compared with the existing technology that relies only on a single device for measurement, it solves the problem that the measurement accuracy is limited by a single device and the measurement results are easily affected by the environment, and at the same time improves the stability and adaptability of the measurement data.

[0051] 2. The present invention realizes the collaborative measurement of the monocular camera and the laser rangefinder by designing a common target with a backlight reflection point and a corner cube prism, achieving the technical effect of equipment data sharing and target feature unification. Compared with the prior art solution that requires designing a dedicated target for each device, it solves the shortcomings of complex target design, cumbersome calibration, and inability to efficiently fuse measurement data, simplifies system design, and improves measurement efficiency.

[0052] 3. The present invention constructs a fusion measurement model through a calculation module, combines laser ranging data and image plane coordinates, calculates the three-dimensional coordinates of the target center, and achieves the effect of improving measurement accuracy. Compared with the traditional monocular camera solution based on feature point extraction and calibration to achieve three-dimensional measurement, it solves the technical problems of large depth error and insufficient reliability of measurement results caused by image resolution limitations, especially showing significant accuracy advantages in medium and long-distance measurement scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the step structure of the present invention;

[0054] Figure 2 It is a schematic diagram of the device components of the present invention;

[0055] Figure 3 This is a schematic diagram of the vision and laser ranging fusion measurement principle of the present invention;

[0056] Figure 4 Schematic diagram of the common target of the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Embodiment 1:

[0059] Please see attached Figure 1 -Attached Figure 2 The embodiment of the present invention provides a method for three-dimensional fusion measurement of a monocular camera and a laser rangefinder and extraction of a target center, comprising the following steps:

[0060] S1, calibrate the internal parameters of the monocular camera to obtain the pixel coordinates and focal length of the principal point;

[0061] Specifically, the acquisition of internal parameters is the basis for the subsequent 3D space point calculation. The camera's internal parameters include the principal point pixel coordinates and focal length, which directly affect the accuracy of the camera imaging model.

[0062] Monocular camera imaging follows the perspective projection model. After perspective projection, a spatial point forms an image point on the imaging plane. The geometric relationship of the imaging depends on the intrinsic parameters of the camera. The intrinsic parameters include:

[0063] 1. The pixel coordinates of the principal point (u 0 ,v 0 ): The intersection point of the camera optical axis on the image plane.

[0064] 2. Focal length f: The optical focal length of the camera lens determines the proportional relationship of the projection.

[0065] Through calibration, the imaging model of the camera can be accurately established.

[0066] S2, calibrate the external parameters of the laser rangefinder, and obtain the coordinates of the laser rangefinder light source point in the camera coordinate system and the laser direction vector;

[0067] Specifically, the laser rangefinder external parameter calibration requires aligning the laser rangefinder coordinate system with the camera coordinate system to ensure that the laser distance measurement data and the image coordinates can be seamlessly integrated. Specifically, the goal of this step is to obtain the spatial position of the laser rangefinder light source point in the camera coordinate system and determine the direction vector of the laser beam.

[0068] In general, in order to achieve coordinated operation of the laser rangefinder and the camera system, it is necessary to map the relative geometric position and direction between the two devices through calibration scenes and specific common targets. As an option, the method of calibrating the external parameters of the laser rangefinder should combine the actual characteristics of the laser rangefinder and the geometric design of the target to ensure the stability and high accuracy of the measurement. In some embodiments, the laser rangefinder and the camera remain relatively fixed during the calibration process, while the target provides multiple sets of reference data by being placed at different positions and angles, and finally the external parameter calibration is completed through fitting calculation.

[0069] In this embodiment, the direction vector d of the laser beam L It is obtained by measuring the coordinate position of the target center multiple times and fitting it. During the calibration process, the target is placed at different positions in the calibration scene, and the three-dimensional coordinates of the target center in the camera coordinate system are recorded. As an option, the direction vector of the laser beam can be fitted in the following way: Assume that the laser beam satisfies the straight line equation r L =p L +t·d L , where p L is the coordinate of the light source point, d L is the direction vector, and t is the parameter. By fitting the measured data using the least squares method, the coordinates of the light source point and the specific value of the direction vector can be obtained simultaneously.

[0070] S3. Design shared targets;

[0071] Specifically, in this embodiment, the design of the common target adopts a regular hexagonal distribution scheme to meet the collaborative measurement requirements of the monocular camera and the laser rangefinder.

[0072] Specifically, the target contains six backlight reflection points and a corner cube prism. Among them, the six backlight reflection points are evenly distributed at the vertices of a regular hexagon, which are used to provide two-dimensional feature points when the camera captures the image. These reflection points are made of highly reflective material, and their surface is coated with a nonlinear reflective coating, which has a prominent brightness difference during camera shooting, which is convenient for subsequent image processing. The vertex of the corner cube prism coincides with the center of the regular hexagon. Its main function is to accurately reflect the laser beam to the rangefinder to ensure the accuracy of the laser ranging data.

[0073] In one possible implementation, the size of the target and the size of the reflection point are adjusted according to the specific measurement scenario. For example, in a small scenario, the diameter of the backlight reflection point can be set to 5 mm to 10 mm; while in a large-scale measurement scenario, the diameter of the reflection point can be extended to 20 mm to 50 mm. In addition, the overall size of the target should be compatible with the field of view of the measurement device to avoid insufficient measurement range or reduced accuracy due to the target being too large or too small.

[0074] In general, the pixel coordinates of the six reflected light points can be obtained by shooting the target image with a camera. In the image processing stage, the grayscale weighted method is used to extract the features of the pixel area of ​​each reflection point and calculate its two-dimensional image plane coordinates. The principle of the grayscale weighted method is based on the distribution of pixel grayscale values ​​in the reflection point area, and the grayscale value of the pixel point is used as a weight to improve the accuracy of coordinate calculation. The calculation formula is as follows:

[0075]

[0076] Where (x, y) is the pixel coordinate, and I(x, y) is the grayscale value of the corresponding pixel. As an option, in the image processing stage, a threshold range can be set for the size and brightness distribution of the reflection point area to eliminate false points caused by ambient light or reflection anomalies. In some embodiments, in order to improve the extraction efficiency, the noise area around the reflection point can be filtered out by binarization, thereby further enhancing the prominence of the reflection point feature. After the two-dimensional image plane coordinates of the six reflection points are extracted, after affine transformation, the geometric constraint relationship of the regular hexagon is used to screen out the point group that meets the parallelism. Specifically, the relative sides of the regular hexagon are parallel to each other. By analyzing the geometric distribution between the reflection points and combining the above characteristics, abnormal points or interference points can be effectively removed.

[0077] S4, photographing the target with a monocular camera and extracting the two-dimensional image coordinates of the reflected light point;

[0078] Specifically, after completing the design and layout of the target, collecting the target image through the camera and extracting the two-dimensional image plane coordinates of the return light reflection point is one of the key basic steps in the present invention. The purpose of this step is to provide accurate two-dimensional image plane data for subsequent target center extraction and three-dimensional coordinate calculation. The accuracy of the extraction result directly affects the calculation reliability of the subsequent fusion measurement model.

[0079] In general, the extraction of the reflected light points needs to be combined with the geometric characteristics of the target, the internal parameters of the camera, and the image processing algorithm to ensure that the image plane coordinates of each reflection point can be accurately obtained under complex lighting conditions. As an option, the sub-pixel centroid of the reflection point area can be calculated by a high grayscale weighted method to improve the resolution and accuracy of the extraction. In some embodiments, the stability of point extraction can also be enhanced by combining the regional segmentation algorithm with the distribution characteristics of different target reflection points.

[0080] S5. According to the geometric characteristics of the target, the geometric constraints between the return light reflection points are calculated, the candidate point group is screened, and the least squares ellipse fitting is used. The identified candidate point group is screened again according to the fitting ellipse error, and finally the two-dimensional image plane coordinates of the target center are extracted by but not limited to the grayscale weighted method.

[0081] Specifically, after completing the extraction of the two-dimensional image plane coordinates of the return light reflection point, it is necessary to further calculate the two-dimensional image plane coordinates of the target center. The accuracy of the target center directly affects the subsequent fusion calculation of the three-dimensional coordinates. This step combines the geometric design of the target, and through the geometric constraint analysis of the extracted reflection point coordinates, finds the candidate point group, and then uses the least squares method to fit the ellipse. The candidate point group is screened again according to the error size of the fitted ellipse, and finally the target point group is obtained, but not limited to the grayscale weighted method to extract the two-dimensional image plane coordinates of the target center.

[0082] In general, the geometric properties of the target are the basis for high-precision center calculation. The reflection points of the target are usually arranged in a symmetrical structure, which can provide multi-point geometric distribution characteristics. For example, in the present invention, the target reflection points are arranged in a regular hexagon, and the distance between any adjacent points is equal and symmetrical to the center of the hexagon. This geometric property provides an effective constraint for screening interference points and abnormal points. As an option, the least squares fitting method can be combined to further optimize the calculation results and improve the robustness of the center coordinates. In some embodiments, weighted processing can also be performed in combination with data collected from multiple frames to eliminate measurement errors.

[0083] S6. Using a laser rangefinder to measure the distance information of the target center;

[0084] Specifically, after completing the extraction of the two-dimensional image coordinates of the target center, using a laser rangefinder to measure the laser distance value of the target center is one of the key steps to achieve three-dimensional coordinate fusion. The distance value L of the target center directly affects the accuracy of the three-dimensional coordinate calculation and is an important input parameter for the subsequent construction of the fusion measurement model.

[0085] Generally, a laser rangefinder measures the straight-line distance between the center of a target and the rangefinder by emitting a laser beam and receiving a reflected signal. In order to improve the measurement accuracy and stability, the measurement angle and position of the rangefinder need to be adjusted in combination with the geometric characteristics of the target and the directionality of the laser beam. As an option, a corner cube prism in the target can be used to reflect the laser beam so that its return path is consistent with the incident path, thereby reducing the impact of environmental interference. In some embodiments, the optical center of the laser rangefinder needs to be aligned with the center of the target to ensure consistency between the distance measurement data and the image plane coordinates.

[0086] S7, combining the laser ranging value and the two-dimensional image plane coordinates of the target center, constructing a fusion measurement model, and calculating the three-dimensional coordinates of the target center in the camera coordinate system;

[0087] Specifically, after completing the extraction of the two-dimensional image coordinates of the target center, using a laser rangefinder to measure the laser distance value of the target center is one of the key steps to achieve three-dimensional coordinate fusion. The distance value L of the target center directly affects the accuracy of the three-dimensional coordinate calculation and is an important input parameter for the subsequent construction of the fusion measurement model.

[0088] Generally, a laser rangefinder measures the straight-line distance between the center of a target and the rangefinder by emitting a laser beam and receiving a reflected signal. In order to improve the measurement accuracy and stability, the measurement angle and position of the rangefinder need to be adjusted in combination with the geometric characteristics of the target and the directionality of the laser beam. As an option, a corner cube prism in the target can be used to reflect the laser beam so that its return path is consistent with the incident path, thereby reducing the impact of environmental interference. In some embodiments, the optical center of the laser rangefinder needs to be aligned with the center of the target to ensure consistency between the distance measurement data and the image plane coordinates.

[0089] S8. Convert the three-dimensional coordinates of the target center from the camera coordinate system to the world coordinate system.

[0090] Specifically, after obtaining the three-dimensional coordinates (X c ,Y c ,Z c ), converting it from the camera coordinate system to the world coordinate system is the key to achieving spatial positioning and subsequent measurement applications. The camera coordinate system is a local coordinate system with the camera optical center as the origin, while the world coordinate system is usually a reference coordinate system defined relative to the global environment. The accuracy of the conversion process directly determines the consistency of the 3D measurement results in space.

[0091] In general, the transformation of the coordinate system needs to be achieved through an extrinsic matrix. The extrinsic matrix consists of a rotation matrix R and a translation matrix T, which is used to describe the rotation relationship and position offset between the camera coordinate system and the world coordinate system. As an option, a calibration plate and a multi-view measurement method can be used to determine the extrinsic matrix of the camera. In some embodiments, the noise data can also be filtered during the coordinate transformation process to improve the stability of the transformation result.

[0092] A measuring device, comprising:

[0093] A monocular camera is used to obtain the two-dimensional image coordinates of the target's reflected light point;

[0094] Laser rangefinder, used to measure the distance information of the target center;

[0095] A common target including a plurality of retro-reflection points and a corner cube prism;

[0096] The calculation module is used to fuse the laser ranging data and the two-dimensional image plane coordinate data to calculate the three-dimensional coordinates of the target center.

[0097] Specifically, the monocular camera in the measuring device is mainly used to obtain the two-dimensional image coordinates of the backlight reflection point on the target. Generally, the monocular camera has high-resolution imaging capabilities and can extract the feature points of the target in complex lighting environments.

[0098] Specifically, the monocular camera is mounted on a fixed frame of the measuring device, with its optical axis pointing to the center of the common target. The intrinsic and extrinsic parameters of the camera are predetermined by a calibration method to associate the two-dimensional image plane coordinates (u, v) of the target with the actual three-dimensional space point. As an option, the resolution of the monocular camera can be above 1920×1080 pixels, and the focal length can be adjusted according to the measurement scene, for example, it can be selected in the range of 8 mm to 50 mm.

[0099] In some embodiments, in order to enhance the measurement accuracy, a filter or a light shield may be installed at the front end of the camera to reduce the interference of external ambient light on the image quality. Specifically, the filter may be a filter with narrow-band light transmission characteristics to match the spectral characteristics of the reflected light point, thereby improving the signal strength of the reflected point in the image.

[0100] The laser rangefinder emits a laser beam, locks the corner cube prism on the target and receives the reflected signal, thereby calculating the spatial distance L between the center of the target and the rangefinder.

[0101] The beam direction of the laser rangefinder is adjusted by the mounting bracket, and its optical center maintains a certain angle with the optical axis of the monocular camera to ensure that the laser beam can accurately illuminate the corner cube prism at the center of the target. In some embodiments, the laser rangefinder supports multiple ranging modes, such as pulse laser ranging and continuous wave phase difference ranging, and the measurement accuracy can reach ±1 mm.

[0102] As a possible design solution, the laser rangefinder can cover a measurement range of 1 meter to 100 meters, which is suitable for medium and long distance measurement scenarios. In addition, in order to improve the measurement stability, the laser rangefinder is equipped with a signal strength analysis module to judge the quality of the reflected signal. When the reflected signal is insufficient, the user can be prompted to adjust the measuring device or target position.

[0103] The target's backlight reflection points are arranged in a regular polygon, such as a regular hexagon or a regular octagon, and the diameter of each reflection point can be designed to be 5 mm to 20 mm. The material of the reflection point is a highly reflective coating, and its surface can maintain a stable reflection intensity under a variety of lighting conditions, thereby ensuring that the monocular camera accurately extracts the two-dimensional image coordinates of the reflection point during shooting.

[0104] The corner cube is located at the center of the target, and its vertex coincides with the center of the regular polygon. The function of the corner cube is to reflect the laser beam back to the laser rangefinder. Generally, the design of the corner cube ensures that the incident path and the reflection path of the laser beam are consistent, thereby reducing measurement errors. In some embodiments, in order to adapt to the outdoor measurement environment, a dustproof coating or protective cover is added to the target surface.

[0105] The calculation module includes a data receiving unit, a geometric calculation unit and an output unit, and each part works together to complete data processing and result output.

[0106] Generally, the calculation module first receives the measurement data from the monocular camera and the laser rangefinder. The two-dimensional image plane coordinates (u, v) and the laser rangefinder value L are transmitted to the calculation module via wired or wireless means. In the data receiving unit, the measurement data is preprocessed, such as removing invalid data or smoothing noise signals.

[0107] The optical axes of the monocular camera and the laser rangefinder are relatively fixed, ensuring the consistency of the measurement data. The position of the common target is determined by the measurement scene, but the geometric relationship between its reflection point and the corner cube prism is fixed during the design. By integrating the measurement results of the two devices through the calculation module, the three-dimensional coordinates of the target center can be accurately restored.

[0108] As an extension, the measuring device can also be combined with multiple monocular cameras or multiple laser rangefinders for synchronous measurement to meet the needs of larger range or higher precision scenarios. For example, in industrial automation measurement, multiple measuring devices can be used to jointly measure the same target, thereby constructing a global 3D model of the target.

[0109] Embodiment 2:

[0110] Based on Example 1, the present invention provides a vision and laser ranging fusion measurement of a monocular camera and a laser rangefinder for three-dimensional fusion measurement and a target center extraction method, including:

[0111] Fusion measurement principle:

[0112] Figure 3 Medium C -X C Y C Z C is the camera coordinate system; point P is the point to be measured in space, and its coordinates in the camera coordinate system are (X P ,Y P ,Z P ); point p is the image point of the measured point in space, and its coordinates in the camera coordinate system are (xx 0 ,yy 0 ,f), where (x,y) is the image plane coordinate of point P, (x 0 ,y 0 ) is the coordinate of the principal point on the image plane, and f is the principal distance of the camera. L is the light source point of the laser rangefinder, and its coordinates in the camera coordinate system are (X L ,Y L ,Z L ); L is the laser ranging distance; point P L is the measuring point of P measured by the laser rangefinder. Due to the aiming error, the two points cannot coincide. Set point P L The coordinates in the camera coordinate system are

[0113] In the camera coordinate system, let the direction vector of the laser beam d = (a, b, c), then the laser measurement point P L The parametric equation of the coordinates is:

[0114]

[0115] In the camera coordinate system, the direction vector O of the light C P=(xx 0 ,yy 0 ,f), the parametric equation of the light can be obtained as:

[0116]

[0117] Among them, t is a parameter.

[0118] The measured point P is located on the camera light, so its coordinates are expressed by the parametric equation:

[0119]

[0120] Among them, t P as a parameter.

[0121] In actual measurement, due to the aiming error, the laser beam does not intersect with the camera light, and the laser measurement point P L If it is not on the camera light, it can be considered that the camera light is on the distance from the laser measuring point P L The nearest point is taken as the measured point P, so the vector PP L The direction vector O of the camera light C P is vertical, we can get:

[0122] PP L ·O C P=0(4)

[0123] Right now:

[0124]

[0125] Combining formula (3) and formula (6) can calculate the parameter t P :

[0126]

[0127] Combining formula (3) and formula (8), we get:

[0128]

[0129] Combining formula (1) and formula (9) we get the coordinates of the measured point P:

[0130]

[0131] In the above formula (x 0 ,y 0 ) is the coordinate of the principal point; f is the principal distance; (x, y) is the image plane coordinate of the measured point P in space; (a, b, c) is the direction vector of the laser beam; L is the laser ranging distance; (X L ,Y L ,Z L ) is the light source point O of the laser rangefinder L The three-dimensional coordinates of the measured point P in the camera coordinate system. Therefore, the three-dimensional coordinates of the measured point P in the camera coordinate system can be calculated by formula (10).

[0132] Common target design and calculation of its center coordinates

[0133] Shared target design:

[0134] In actual measurement, a corner cube is placed at the measured point. The corner cube can be measured by a laser rangefinder, but the camera cannot obtain the vertex position of the corner cube. The camera can accurately obtain the circular coordinates of the plane return light reflection point, so the following shared target is designed. Figure 4 The target used by the Chinese Communist Party consists of six circular reflection points and one corner cube prism. The vertex of the corner cube prism coincides with the center of the circle formed by the six reflection points, thereby achieving the physical unification of the measurement objects of the two sensors, the monocular camera and the laser rangefinder.

[0135] The six backlight reflection points are located at the six vertices of the regular hexagon. The six vertices form a line segment in pairs, with a total of 15 line segments. Take any point, and the line segment it forms has 8 sets of parallel relationships among all the line segments. Taking vertex A as an example, the line segments composed of point A are line segments AB, AC, AD, AE, and AF, among which AB / / CD, AB / / EF, AC / / BE, AC / / DF, AD / / CF, AE / / BF, AF / / BD, and AF / / CE, that is, 8 sets of parallel relationships. According to the characteristics of affine transformation, using this as a constraint, the target point can be screened in the candidate point group to obtain the six vertices of the common target, and then the image plane coordinates of the six vertices can be obtained. The central image plane coordinates can be obtained by the following formula:

[0136]

[0137] After obtaining the image plane coordinates (x, y) of the common target center, the direction vector of the camera light in the camera coordinate system can be obtained according to the above principle.

[0138] The process of shared target identification and center coordinate extraction is as follows:

[0139] Perform image processing and use the weighted square grayscale centroid method to solve the image plane coordinates of the reflected light point;

[0140] Search with each point as the center and a certain search radius to determine the candidate point group;

[0141] Save all possible line segments in the candidate point group and filter them to find all parallel line segments;

[0142] Count the frequency of occurrence of points that make up the parallel line segments, sort them according to the frequency of occurrence, and select the first 6 points with the highest frequency as the target point group;

[0143] The least square method is used to fit an ellipse to the target point group and the ellipse fitting error is solved. The ellipse fitting error is used to determine whether it is the target point group.

[0144] Solve the average coordinates of each target point group after screening, that is, the image plane coordinates of the center of each shared target.

[0145] Basic measurement process:

[0146] The measuring device consists of an AVT camera and a small commercial laser rangefinder, and both are installed on a fixed structure. First, the designed calibration field is used to complete the calibration of the camera's internal and external parameters. The external parameters are the relationship between the camera coordinate system and the world coordinate system. Secondly, according to the method proposed by NguyenT, the external orientation parameters of the laser rangefinder relative to the camera are calibrated, that is, the position of the laser rangefinder's light point and the direction vector of the laser beam in the camera coordinate system.

[0147] Then, a fusion measurement experiment is conducted. Before the experiment, the common target is placed in the calibration field to form a measured field. The laser is aligned with the corner cube prism by adjusting the turntable to obtain distance information and the measured field is photographed using a camera. At the same time, a laser rangefinder is used to obtain distance measurement data and combined with the target recognition algorithm to identify the backlight reflection point on the common target and calculate the image plane coordinates of the center of the common target. Finally, the three-dimensional coordinates of the center of the common target in the camera coordinate system are solved according to formula (10).

[0148] In order to compare the measured values ​​of the target center coordinates with the values ​​measured by the V-STARS system, it is necessary to solve the exterior orientation parameters of the camera in the world coordinate system, that is, the transformation matrix between the camera coordinate system and the world coordinate system, and then solve the three-dimensional coordinates of the shared target center in the world coordinate system.

[0149] Finally, in the world coordinate system, the measured three-dimensional coordinates of the common target center obtained by formula (10) are compared with the measured values ​​of the V-STARS photogrammetry system, and the distance deviation between the two is calculated, thereby verifying the feasibility of the fusion of the above-mentioned camera and laser rangefinder to achieve the measurement of three-dimensional points in space.

[0150] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for three-dimensional fusion measurement of a monocular camera and a laser rangefinder and extraction of a target center, characterized in that: The following steps are involved: S1, calibrate the internal parameters of the monocular camera to obtain the pixel coordinates and focal length of the principal point; S2, calibrate the external parameters of the laser rangefinder, and obtain the coordinates of the laser rangefinder light source point in the camera coordinate system and the laser direction vector; S3. Design shared targets; S4, photographing the target with a monocular camera and extracting the two-dimensional image coordinates of the reflected light point; S5. Perform a first screening based on the geometric features of the common target, then use the least squares method to fit an ellipse, analyze the error of the fitted ellipse, and perform a second screening on the point group to eliminate interference points and extract the return light reflection points on the common target, and then calculate the coordinates of the central image plane of the common target; S6. Using a laser rangefinder to measure the distance information of the target center; S7, combining the laser ranging value and the two-dimensional image plane coordinates of the target center, constructing a fusion measurement model, and calculating the three-dimensional coordinates of the target center in the camera coordinate system; S8. Convert the three-dimensional coordinates of the target center from the camera coordinate system to the world coordinate system.

2. The method for three-dimensional fusion measurement and target center extraction of a monocular camera and a laser rangefinder according to claim 1 is characterized in that: The internal parameters for calibrating the monocular camera in S1 include: Multiple sets of images are obtained through the calibration board, and the camera intrinsic parameter matrix is ​​calculated. The intrinsic parameter matrix includes the focal length and the pixel coordinates of the principal point.

3. The method for three-dimensional fusion measurement and target center extraction of a monocular camera and a laser rangefinder according to claim 1 is characterized in that: The external parameters of the laser rangefinder calibrated in S2 include: Determine the three-dimensional coordinates of the laser rangefinder light source point in the camera coordinate system; Get the laser direction vector; The laser beam equation is represented by the coordinates of the source point and the direction vector.

4. The method for three-dimensional fusion measurement and target center extraction of a monocular camera and a laser rangefinder according to claim 1, characterized in that: The targets in S2 include: Multiple light reflection points are used to extract the two-dimensional image plane coordinates of the monocular camera; A corner cube prism for distance measurement in a laser rangefinder; The reflected light points are distributed at the vertices of the regular polygon, and the vertex of the corner cube coincides with the center of the regular polygon.

5. The method for three-dimensional fusion measurement and target center extraction of a monocular camera and a laser rangefinder according to claim 1, characterized in that: Extracting the two-dimensional image plane coordinates of the return light reflection point in S4 includes: The weighted grayscale centroid method is used to calculate the two-dimensional image coordinates of the reflected light point; The pixel coordinates of each reflected light point are extracted according to the gray value distribution.

6. The method for three-dimensional fusion measurement and target center extraction of a monocular camera and a laser rangefinder according to claim 1, characterized in that: Calculating the two-dimensional image plane coordinates of the target center in S5 includes: By analyzing the geometric constraints between the return light reflection points, the candidate point group is screened; The least squares fitting algorithm is used to extract the two-dimensional image coordinates of the target center.

7. The method for three-dimensional fusion measurement and target center extraction of a monocular camera and a laser rangefinder according to claim 1, characterized in that: The three-dimensional coordinates of the target center in S7 are calculated by a fusion measurement model, and the fusion measurement model includes: The laser rangefinder provides distance information to the center of the target; Construct the direction equation of the laser beam and the direction equation of the camera light; The three-dimensional coordinates of the target center are solved by the shortest distance point method.

8. The method for three-dimensional fusion measurement and target center extraction of a monocular camera and a laser rangefinder according to claim 1, characterized in that: The fused measurement model in S7 satisfies the following geometric constraints: The laser direction vector and the camera light direction vector at the target center have a relationship with the minimum vertical error; The intersection point of the laser beam and the camera light is taken as the three-dimensional coordinate point of the target center.

9. The method for three-dimensional fusion measurement and target center extraction of a monocular camera and a laser rangefinder according to claim 1, characterized in that: When the three-dimensional coordinates of the target center in S8 are converted from the camera coordinate system to the world coordinate system, an external parameter matrix conversion is adopted, including a rotation matrix and a translation matrix.

10. A measuring device, according to the method for three-dimensional fusion measurement of a monocular camera and a laser rangefinder and extraction of a target center according to any one of claims 1 to 9, characterized in that: include: A monocular camera is used to obtain the two-dimensional image coordinates of the target's reflected light point; Laser rangefinder, used to measure the distance information of the target center; A common target including a plurality of retro-reflection points and a corner cube prism; The calculation module is used to fuse the laser ranging data and the two-dimensional image plane coordinate data to calculate the three-dimensional coordinates of the target center.

Images