A vision-assisted laser measurement method and system

Abstract

The present invention proposes a visually assisted laser measurement method and system, which obtains a first ranging value, antenna phase center position coordinates and attitude information from a laser ranging point to a laser point to be measured; based on the first ranging value, antenna phase center position coordinates and attitude information, calculates the laser point coordinates; based on the first ranging value and the image taken by a visually assisted camera, calculates the coordinates of the laser point on the image by mapping parameters; based on the first ranging value, antenna phase center position coordinates, attitude information, laser point coordinates and the coordinates of the laser point on the image, generates an auxiliary information message; based on the auxiliary information message, generates an auxiliary information layer and superimposes the auxiliary information layer on the preview image of the visually assisted camera, so that the laser measurement result is presented on the preview image of the visually assisted camera. The present invention improves the accuracy of laser measurement without the need for manual calibration of equipment and calibration of internal and external parameters of the camera.

Description

Technical Field

[0001] The present invention relates to the field of laser distance measurement, and in particular to a vision-assisted laser measurement method and system. Background Art

[0002] When a measuring instrument equipped with a laser measurement function performs laser measurement operations, if the point to be measured is in a strong light environment or at a long distance, it is difficult for the human eye to capture the position of the laser point, which makes it difficult for the laser to accurately aim at the target point. To solve this problem, it is usually considered to use a camera as an auxiliary tool to locate the laser point. However, due to differences in module installation angles of different laser devices and changes in the distance between the laser point and the camera module, the specific position of the laser point on the image captured by the camera will also be different. Therefore, even with the assistance of a camera, it may not be possible to immediately determine the exact position of the laser point.

[0003] At present, the commonly used method in the prior art is to assist in obtaining the position of the laser point on the image by calculating the internal and external parameters of the camera. However, this method cannot guarantee the structural error of the installation during manual installation and adjustment. In addition, the actual position deviation of the laser point of the laser ranging module is measured using a calibrated camera. First, the internal and external parameters of the camera are calibrated. The steps of the prior art solution are relatively complicated, and the problem of poor laser measurement accuracy may easily occur. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a vision-assisted laser measurement method and system, which can improve the accuracy of laser measurement without the need for manual adjustment of equipment and calibration of internal and external parameters of the camera.

[0005] To achieve the above-mentioned purpose, an embodiment of the present invention provides a visually assisted laser measurement method, including: obtaining a first ranging value from a laser ranging point to a laser point to be measured; obtaining antenna phase center position coordinates and attitude information; calculating the laser point coordinates based on the first ranging value, the antenna phase center position coordinates and the attitude information; calculating the coordinates of the laser point on the image through mapping parameters based on the first ranging value and an image taken by a visually assisted camera; generating an auxiliary information message based on the first ranging value, the antenna phase center position coordinates, the attitude information, the laser point coordinates and the coordinates of the laser point on the image; generating an auxiliary information layer based on the auxiliary information message and superimposing the auxiliary information layer on the preview image of the visually assisted camera, so that the laser measurement result is presented on the preview image of the visually assisted camera.

[0006] The embodiment of the present invention proposes a vision-assisted laser measurement method, which calculates the coordinates of the laser point in space by collecting multiple data such as the first ranging value from the laser ranging point to the laser point to be measured, the antenna phase center position coordinates and posture information. The collected data can be automatically measured by the equipment, and there is no need to manually adjust the equipment, which simplifies the equipment installation steps, thereby improving the overall accuracy of the laser measurement. The laser ranging value is mapped with the image taken by the auxiliary camera, and the coordinates of the laser point on the image are calculated using the mapping parameters. Although the auxiliary camera is used, its internal and external parameters are not used, so that it can adapt to more camera models, simplify the calibration steps, and improve the accuracy of laser measurement. Finally, all the collected laser point related information is generated into an auxiliary information message, and it is converted into a layer and superimposed on the preview image of the visual auxiliary camera, so that the position of the laser point can be more intuitively obtained. Therefore, it is achieved that there is no need to manually adjust the equipment and there is no need to calibrate the internal and external parameters of the camera, thereby improving the accuracy of laser measurement.

[0007] Furthermore, based on the first ranging value, the antenna phase center position coordinates and the attitude information, the laser point coordinates are calculated, including: based on the attitude information, the laser direction vector is calculated; the first ranging value is multiplied by the laser direction vector to obtain the position vector; the antenna phase center position coordinates are added to the position vector to obtain the laser point coordinates.

[0008] Through the above scheme, the measurement environment is comprehensively reflected through the first ranging value, the antenna phase center position coordinates and the attitude information, so as to solve the problem that the traditional measurement method may cause error accumulation due to factors such as improper equipment calibration and environmental changes. In addition, by automatically acquiring and processing a variety of data sources to obtain the direction vector of the laser point, in addition to being able to dynamically adjust the measurement parameters, it is also possible to achieve the unity of the laser point and the antenna center position coordinate system by adding the antenna phase center position coordinates to the position vector, thereby reducing the cumulative effect of errors and thus improving the accuracy of laser measurement.

[0009] Furthermore, based on the first ranging value and the image taken by the visual-assistance camera, the coordinates of the laser point on the image are calculated by mapping parameters, including: calibrating the mapping parameters of the laser point on the image taken by the visual-assistance camera; including: obtaining a plurality of second ranging values ​​from the laser ranging point to a plurality of laser points; identifying the corresponding positions of a plurality of laser points in the image by a preset image processing algorithm, and obtaining the position coordinates of the plurality of laser points corresponding to the image taken by the visual-assistance camera; constructing the three-dimensional coordinates of each laser point based on the plurality of second ranging values ​​of the plurality of laser points and the position coordinates of the plurality of laser points corresponding to the image taken by the visual-assistance camera; and mapping the plurality of laser points to the corresponding positions by a preset mapping algorithm. The three-dimensional coordinates of the light spot are nonlinearly optimized to obtain mapping parameters of the laser point on the image taken by the visual assistance camera, including: constructing a target mapping function and initializing mapping parameters; inputting the three-dimensional coordinates of each laser point into the target mapping function, performing gradient calculation on the mapping parameters, and obtaining gradient values ​​of each mapping parameter; updating the gradient value of each mapping parameter through a preset gradient descent algorithm until the preset update requirements are met, and obtaining the mapping parameters of the laser point on the image taken by the visual assistance camera; based on the first ranging value and the mapping parameters, calculating the coordinates of the laser point to be measured mapped to the image taken by the visual assistance camera through a transformation formula, and obtaining the coordinates of the laser point on the image.

[0010] Through the above scheme, mapping parameters are obtained by adopting a nonlinear optimization method, and then the three-dimensional laser point coordinates are mapped to the coordinates on the two-dimensional image using the mapping parameters. It can automatically adapt to different measurement environments and conditions, and can avoid the need for manual calibration of equipment and internal and external parameters of the camera. Through gradient learning of a large amount of data, the accuracy of the mapping results is guaranteed, thereby improving the accuracy of laser measurement.

[0011] Furthermore, based on the first ranging value, the antenna phase center position coordinates, the posture information, the laser point coordinates and the coordinates of the laser point on the image, an auxiliary information message is generated, including: encoding the first ranging value, the antenna phase center position coordinates, the posture information, the laser point coordinates and the coordinates of the laser point on the image to obtain a plurality of data segments that conform to a preset message structure; and combining and encapsulating the plurality of data segments that conform to the preset message structure to obtain an auxiliary information message.

[0012] Through the above scheme, by encoding data from multiple sources and combining the encoded data segments of the multi-source data into auxiliary information messages, the data format is unified and the data is combined, thereby improving data transmission efficiency and parsing efficiency.

[0013] Furthermore, based on the auxiliary information message, an auxiliary information layer is generated and superimposed on the preview image of the visual assist camera, so that the laser measurement result is presented on the preview image of the visual assist camera, including: decoding the auxiliary information message to obtain an auxiliary information combination; constructing an auxiliary information layer based on the auxiliary information combination; adjusting the position and transparency of the auxiliary information layer, superimposing the auxiliary information layer that meets the preview requirements on the preview image of the visual assist camera, so that the laser measurement result is presented on the preview image of the visual assist camera.

[0014] Through the above scheme, the auxiliary information combination is extracted by decoding the auxiliary information message, and the auxiliary information combination is superimposed on the preview image of the visual auxiliary camera. The auxiliary information layer provides additional reference information, which helps to reduce measurement errors caused by improper operation or equipment errors. Without repeated measurements, the position of the current laser point can be quickly found according to the generated image, thereby improving the efficiency of aligning the laser point with the point to be measured. At the same time, the auxiliary information combination can also be used to quickly obtain information such as the current position and posture of the host and the measurement results, thereby improving the accuracy of the laser measurement results.

[0015] Furthermore, an embodiment of the present invention provides a visually assisted laser measurement method which also includes: real-time updating of the first ranging value of the laser point to be measured, the antenna phase center position coordinates and the attitude information, recalculating the laser point coordinates and verifying the laser measurement results; and updating the mapping parameters based on the preview image of the visually assisted camera corresponding to the real-time updated laser point coordinates and the laser measurement results.

[0016] Through the above scheme, by real-time updating of data such as ranging values, antenna phase center position coordinates and attitude information, it is helpful to reduce measurement errors caused by environmental changes or equipment status changes, and real-time updating of laser point coordinates and verification of laser measurement results enable the system to immediately discover measurement errors and correct them. The instant feedback mechanism helps to ensure the accuracy and reliability of measurement results. By updating the preview image of the visual auxiliary camera corresponding to the updated laser point coordinates and the laser measurement results, the mapping parameters are updated, which can improve the accuracy of the mapping results. The mapping position of the laser point on the image is continuously learned and updated, thereby optimizing the mapping parameters and improving the accuracy of the laser measurement results.

[0017] The embodiment of the present invention also provides a visually assisted laser measurement system, comprising: a laser ranging module, a navigation module, a first coordinate acquisition module, a second coordinate acquisition module, an auxiliary information message generation module and a layer overlay module; the laser ranging module is used to obtain a first ranging value from a laser ranging point to a laser point to be measured; the navigation module obtains the antenna phase center position coordinates and attitude information; the first coordinate acquisition module is used to calculate the laser point coordinates based on the first ranging value, the antenna phase center position coordinates and the attitude information; the second coordinate acquisition module is used to calculate the coordinates of the laser point on the image through mapping parameters based on the first ranging value and the image taken by the visually assisted camera; the auxiliary information message generation module is used to generate an auxiliary information message based on the first ranging value, the antenna phase center position coordinates, the attitude information, the laser point coordinates and the coordinates of the laser point on the image; the layer overlay module is used to generate an auxiliary information layer based on the auxiliary information message and overlay the auxiliary information layer on the preview image of the visually assisted camera, so that the laser measurement result is presented on the preview image of the visually assisted camera.

[0018] The embodiment of the present invention proposes a visually assisted laser measurement system. The laser ranging module and the navigation module are used to collect a plurality of data such as a first ranging value from a laser ranging point to a laser point to be measured, the coordinates of the antenna phase center position and the attitude information. The first coordinate acquisition module is then used to calculate the coordinates of the laser point in space. The collected data can be automatically measured by the device without manual adjustment of the device, simplifying the device installation steps, thereby improving the overall accuracy of the laser measurement. The second coordinate acquisition module is used to map the laser ranging value with the image taken by the auxiliary camera, and then the mapping parameters are used to calculate the coordinates of the laser point on the image. Although the auxiliary camera is used, its internal and external parameters are not used, so that it can adapt to more camera models and improve the accuracy of laser measurement. Finally, the auxiliary information message generation module generates an auxiliary information message for all the collected laser point related information, and converts it into a layer through the layer overlay module and overlays it on the preview image of the visual auxiliary camera, so that the position of the laser point can be more intuitively obtained. Therefore, it is achieved that there is no need to manually adjust the device and calibrate the internal and external parameters of the camera, thereby improving the accuracy of laser measurement.

[0019] Furthermore, the second coordinate acquisition module is used to calculate the coordinates of the laser point on the image through mapping parameters based on the first ranging value and the image taken by the visual assistance camera, including: a mapping parameter calibration module and a laser point mapping module; the mapping parameter calibration module is used to calibrate the mapping parameters of the laser point on the image taken by the visual assistance camera; the laser point mapping module is used to calculate the coordinates of the laser point to be measured mapped to the image taken by the visual assistance camera through a transformation formula based on the first ranging value and the mapping parameters, so as to obtain the coordinates of the laser point on the image.

[0020] Through the above scheme, the mapping parameters are obtained by using a large number of data points for nonlinear optimization through the mapping parameter calibration module, and then the three-dimensional laser point coordinates are mapped to the coordinates on the two-dimensional image through the laser point mapping module using the mapping parameters. It can automatically adapt to different measurement environments and conditions, avoid the need for manual calibration of equipment and internal and external parameters of the camera, and ensure the accuracy of the mapping results through gradient learning of a large amount of data, thereby improving the accuracy of laser measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic flow chart of the steps of a visually assisted laser measurement method provided by an embodiment of the present invention;

[0022] Figure 2 A schematic flow chart of mapping parameter calibration steps of a visually assisted laser measurement method provided in one embodiment of the present invention;

[0023] Figure 3 A schematic diagram of the module structure of a visually assisted laser measurement system provided in one embodiment of the present invention;

[0024] Among them, 301, laser ranging module; 302, navigation module; 303, first coordinate acquisition module; 304, second coordinate acquisition module; 305, auxiliary information message generation module; 306, layer overlay module; 3041, mapping parameter calibration module; 3042, laser point mapping module. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Example 1

[0027] See also Figure 1 , Figure 1 A schematic flow chart of the steps of a visually assisted laser measurement method provided by an embodiment of the present invention; Figure 1 As shown, the embodiment of the present invention provides a visually assisted laser measurement method, including steps 101 to 106, each of which is specifically as follows:

[0028] Step 101, obtaining a first distance measurement value from a laser distance measurement point to a laser point to be measured;

[0029] A specific implementation method is to turn on the laser measuring device and obtain the first distance measurement value from the current laser ranging point to the laser point to be measured returned by the laser measuring device, which is recorded as d. In this embodiment, the laser ranging point is equivalent to the laser emission point.

[0030] Step 102, obtaining antenna phase center position coordinates and attitude information;

[0031] A specific implementation method is to output the antenna phase center position coordinates through the satellite navigation of the laser measurement device, wherein the antenna phase center position coordinates are three-dimensional coordinates, and the antenna phase center position coordinates can be set in the geodetic coordinate system, the geocentric fixed coordinate system or other applicable coordinate systems according to the actual working environment; output the attitude information through the inertial navigation of the laser measurement device, and the attitude information generally includes: pitch angle, roll angle and heading angle, etc.;

[0032] Step 103, calculating the laser point coordinates based on the first ranging value, the antenna phase center position coordinates and the attitude information;

[0033] As an example of this embodiment, based on the posture information, the laser direction vector is calculated; the first ranging value is multiplied by the laser direction vector to obtain a position vector; the antenna phase center position coordinates are added to the position vector to obtain the laser point coordinates.

[0034] In order to solve the problem of error accumulation caused by improper equipment calibration and environmental changes in the prior art, the laser point coordinates are calculated by obtaining the ranging value, the antenna phase center position coordinates and the attitude information, thereby being able to comprehensively reflect the measurement environment, and by realizing the unity of the laser point and the antenna center position coordinate system, the cumulative effect of errors can be reduced, thereby improving the accuracy of laser measurement. A specific implementation method is to construct a laser direction vector of the laser beam relative to the antenna phase center according to the attitude information output by the inertial navigation, calculate the product of the laser direction vector and the first ranging value, and obtain the position vector of the laser point, and then unify the position vector of the laser point and the antenna phase center position coordinates in the same coordinate system to obtain the laser point coordinates. The specific implementation method can be achieved by using mature means of the prior art.

[0035] Step 104, based on the first distance measurement value and the image taken by the visual auxiliary camera, calculate the coordinates of the laser point on the image through mapping parameters;

[0036] As an example of this embodiment, the mapping parameters of the laser point on the image taken by the visual assistance camera are calibrated; based on the first ranging value and the mapping parameters, the coordinates of the laser point to be measured mapped to the image taken by the visual assistance camera are calculated by a transformation formula to obtain the coordinates of the laser point on the image.

[0037] In order to avoid the need for manual calibration of equipment and camera internal and external parameters, the embodiment of the present invention proposes to use mapping parameters to map the three-dimensional laser point coordinates to the coordinates on the two-dimensional image. A specific implementation method is to calculate the coordinates of the laser point on the image through the mapping parameters and the transformation formula. The specific calculation formula is:

[0038]

[0039] Wherein, [x, y] is the coordinate of the laser point on the image; d is the first distance measurement value from the laser ranging point to the laser point to be measured obtained in step 101; k1, k2, u and v are mapping parameters; wherein k1, k2, u and v are mapping parameters obtained through the parameter calibration step.

[0040] In order to make the mapping results more reliable, the mapping parameters k1, k2, u and v are calibrated by nonlinear optimization to ensure the accuracy of the mapping results and thus improve the accuracy of laser measurement. For a specific implementation method of parameter calibration, see Figure 2 , Figure 2 A flowchart of mapping parameter calibration steps of a visually assisted laser measurement method provided in an embodiment of the present invention; Figure 2 As shown, it includes steps S201 to S204,

[0041] Step S201, obtaining a plurality of second distance measurement values ​​from the laser ranging point to a plurality of laser points; a specific explanation is to turn on the laser measurement device, aim the laser point at any laser ranging point observable by the auxiliary camera, the auxiliary camera takes the current photo, and records the distance measurement value d0 returned by the current laser ranging module, and repeat the above steps to obtain a plurality of second distance measurement values ​​d i ,i=0,1,2,3,...,i;

[0042] Step S202, identifying the corresponding positions of several laser points in the picture by a preset image processing algorithm, and obtaining the position coordinates of several laser points corresponding to the image taken by the visual auxiliary camera; a specific explanation is to identify the position A0 (x0, y0) of the laser point on the photo by a preset image processing algorithm, repeat the above steps, and obtain the position coordinates A0 (x0, y0) of several laser points corresponding to the image taken by the visual auxiliary camera. i (x i ,y i ), i=0, 1, 2, 3,..., i;

[0043] Step S203, based on the second ranging values ​​of the laser points and the position coordinates of the laser points corresponding to the image captured by the visual auxiliary camera, construct the three-dimensional coordinates of each laser point; a specific explanation is to combine the second ranging value of the laser point with the position coordinates of the corresponding laser point corresponding to the image captured by the visual auxiliary camera to obtain the three-dimensional coordinates P0 (x0, y0, d0) of the laser point, repeat the above steps to obtain the three-dimensional coordinates P0 (x0, y0, d0) of the laser point corresponding to the laser points. i (x i ,y i , d i ), i=0, 1, 2, 3, ..., i; It is worth mentioning that, under normal circumstances, step S204 can be executed to perform nonlinear optimization calculation after obtaining at least two three-dimensional coordinates. In addition, if the position of the laser point is known and is a certain value, step S204 can be directly executed through the three-dimensional coordinates of the laser point to perform nonlinear optimization calculation.

[0044] Step S204, nonlinearly optimize the three-dimensional coordinates of each laser point through a preset mapping algorithm to obtain mapping parameters of the laser point on the image captured by the visual auxiliary camera. Specifically, construct a target mapping function and initialize the mapping parameters; taking the least squares error as an example, the target function is as follows:

[0045]

[0046] [x i ,y i ] is the position coordinate of the laser point corresponding to the image captured by the visual auxiliary camera; n is the number of data points; i indicates the i-th measurement; d i The second distance measurement value is obtained in step S201; k1, k2, u and v are mapping parameters;

[0047] Construct the objective function and give the initial values ​​of the initialization parameters k1, k2, u and v based on previous test experience;

[0048] The three-dimensional coordinates of each laser point are input into the target mapping function, and the mapping parameters are gradient calculated to obtain the gradient values ​​of each mapping parameter; specifically, a nonlinear optimization algorithm is selected, and the optimization algorithm can also be a gradient descent, Newton method, quasi-Newton method, global optimization algorithm, least squares method, conjugate gradient method, simulated annealing algorithm and genetic algorithm. In this embodiment, gradient descent is used as an example to explain, and the gradient of the target function with respect to each parameter is calculated. For the constructed target function, the calculation of the gradient involves finding partial derivatives for each parameter, and the calculus rules and the chain rule can usually be used to directly find the derivative;

[0049] By using a preset gradient descent algorithm, the gradient value of each mapping parameter is updated until the preset update requirement is met, and the mapping parameter of the laser point on the image taken by the visual auxiliary camera is obtained; specifically, the parameter value is continuously updated according to the gradient descent algorithm until a certain stopping criterion is met, and the stopping criterion may be a situation where the set maximum number of iterations is reached, the change in the objective function value is less than a certain threshold, or the change in the parameter is less than a certain threshold, etc. The gradient descent algorithm is a mature optimization algorithm in the prior art, and the optimization process thereof is not described in detail here;

[0050] When the optimization algorithm stops updating the parameters, the obtained values ​​of k1, k2, u and v are the optimal mapping parameter values.

[0051] Step 105, generating an auxiliary information message based on the first ranging value, the antenna phase center position coordinates, the attitude information, the laser point coordinates and the coordinates of the laser point on the image;

[0052] As an example of this embodiment, the first ranging value, the antenna phase center position coordinates, the posture information, the laser point coordinates and the coordinates of the laser point on the image are encoded to obtain a number of data segments that conform to a preset message structure; the several data segments that conform to the preset message structure are combined and encapsulated to obtain an auxiliary information message.

[0053] The laser measurement result is usually multiple data. The existing technology is difficult to unify the data format through the technical solution of manually adjusting the equipment and calibrating the internal and external parameters of the camera, which not only affects the accuracy of laser measurement, but also affects the efficiency of data transmission. In order to improve the efficiency of data transmission, the embodiment of the present invention proposes to combine the encoded data segments of multi-source data into auxiliary information messages to achieve data format unification and data combination. A specific implementation method is to design a message structure containing all necessary information, including a message header, a data segment and a check code. The message header contains basic information such as the message type, version number and sending time. The data segment contains specific data such as the first ranging value, the antenna phase center position coordinates, attitude information, the laser point coordinates and the coordinates of the laser point on the image. The check code is used to verify the integrity and correctness of the message. The data is encoded, such as by using binary encoding or hexadecimal encoding, to improve the efficiency and reliability of data transmission. The data is then encapsulated according to the message structure, and an auxiliary information message is generated according to the encapsulated and encoded data; the auxiliary information message can be output to the user device through channels including but not limited to serial ports, USB and WiFi.

[0054] Step 106: Generate an auxiliary information layer based on the auxiliary information message and overlay the auxiliary information layer on the preview image of the visual assistance camera, so that the laser measurement result is presented on the preview image of the visual assistance camera.

[0055] As an example of this embodiment, the auxiliary information message is decoded to obtain an auxiliary information combination; based on the auxiliary information combination, an auxiliary information layer is constructed; the position and transparency of the auxiliary information layer are adjusted, and the auxiliary information layer that meets the preview requirements is superimposed on the preview image of the visual assist camera, so that the laser measurement result is presented on the preview image of the visual assist camera.

[0056] In order to obtain the position of the laser point more intuitively, the auxiliary information message is parsed into an auxiliary information combination and then superimposed on the preview image of the visual auxiliary camera. The auxiliary information layer provides additional reference information, which helps to reduce measurement errors caused by improper operation or equipment errors. The position of the current laser point can be quickly found according to the generated image without repeated measurements, thereby improving the efficiency of aligning the laser point with the point to be measured. At the same time, the current position and posture of the host and the measurement results can be quickly obtained through the auxiliary information combination, thereby improving the accuracy of the laser measurement results. A specific implementation method is to parse the auxiliary information message through the existing mature parsing algorithm to obtain the auxiliary information message. A plurality of groups of auxiliary information combinations, each combination usually includes specific data such as a first ranging value, antenna phase center position coordinates, attitude information, laser point coordinates and coordinates of the laser point on the image, etc., the decoded auxiliary information combination is subjected to noise filtering and coordinate calibration and other processing, the auxiliary information combination is converted into a format suitable for display on the image, an empty layer is created, the auxiliary information combination is imported into the empty layer, and then aligned with the preview image of the visual assistance camera, the layer transparency is adjusted as needed, and the pixels of the layer are merged with the pixels of the preview image, so that the auxiliary information combination is clearly displayed on the preview image of the visual assistance camera, so that the laser measurement result is presented on the preview image of the visual assistance camera.

[0057] As another example of an embodiment of the present invention, in order to make the laser measurement data more reliable, the embodiment of the present invention also proposes a feedback update mechanism, which updates the ranging value, the antenna phase center position coordinates and the attitude information and other data in real time, and then updates the laser point coordinates and the preview image of the visual auxiliary camera corresponding to the laser measurement result, and repeats steps S201 to S204 to update the mapping parameters, continuously learns and updates the mapping position of the laser point on the image, and then optimizes the mapping parameters and improves the accuracy of the laser measurement result. A specific implementation method is to update the first ranging value, the antenna phase center position coordinates and the attitude information of the laser point to be measured in real time, recalculate the laser point coordinates and verify the laser measurement result, for example: use the current position, attitude, and ranging value of the laser measurement device to recalculate the coordinates of the laser point and the coordinates of the laser point on the image, and verify the laser measurement result; based on the real-time updated laser point coordinates and the preview image of the visual auxiliary camera corresponding to the laser measurement result, update the mapping parameters, for example: use multiple measurement results to perform gradient optimization on the mapping parameters, continuously learn and update the mapping parameter values, and improve the accuracy of the laser measurement result.

[0058] The embodiment of the present invention proposes a vision-assisted laser measurement method, which calculates the coordinates of the laser point in space by collecting multiple data such as the first ranging value from the laser ranging point to the laser point to be measured, the antenna phase center position coordinates and posture information. The collected data can be automatically measured by the equipment, and there is no need to manually adjust the equipment, which simplifies the equipment installation steps, thereby improving the overall accuracy of the laser measurement. The laser ranging value is mapped with the image taken by the auxiliary camera, and the coordinates of the laser point on the image are calculated using the mapping parameters. Although the auxiliary camera is used, its internal and external parameters are not used, so that it can adapt to more camera models, simplify the calibration steps, and improve the accuracy of laser measurement. Finally, all the collected laser point related information is generated into an auxiliary information message, and it is converted into a layer and superimposed on the preview image of the visual auxiliary camera, so that the position of the laser point can be more intuitively obtained. Therefore, it is achieved that there is no need to manually adjust the equipment and there is no need to calibrate the internal and external parameters of the camera, thereby improving the accuracy of laser measurement.

[0059] Example 2

[0060] See also Figure 3 , Figure 3 A schematic diagram of the module structure of a visually assisted laser measurement system provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides a visually assisted laser measurement system, comprising:

[0061] Laser ranging module 301, navigation module 302, first coordinate acquisition module 303, second coordinate acquisition module 304, auxiliary information message generation module 305 and layer overlay module 306; the laser ranging module 301 is used to obtain a first ranging value from a laser ranging point to a laser point to be measured; the navigation module 302 obtains the antenna phase center position coordinates and attitude information; the first coordinate acquisition module 303 is used to calculate the laser point coordinates based on the first ranging value, the antenna phase center position coordinates and the attitude information; the second coordinate acquisition module 304 is used to calculate the coordinates of the laser point on the image through mapping parameters based on the first ranging value and the image taken by the visual assistance camera; the auxiliary information message generation module 305 is used to generate an auxiliary information message based on the first ranging value, the antenna phase center position coordinates, the attitude information, the laser point coordinates and the coordinates of the laser point on the image; the layer overlay module 306 is used to generate an auxiliary information layer based on the auxiliary information message and overlay the auxiliary information layer on the preview image of the visual assistance camera, so that the laser measurement result is presented on the preview image of the visual assistance camera.

[0062] As an example of an embodiment of the present invention, the second coordinate acquisition module 304 is used to calculate the coordinates of the laser point on the image through mapping parameters based on the first ranging value and the image taken by the visual assistance camera, including: a mapping parameter calibration module 3041 and a laser point mapping module 3042; the mapping parameter calibration module 3041 is used to calibrate the mapping parameters of the laser point on the image taken by the visual assistance camera; the laser point mapping module 3042 is used to calculate the coordinates of the laser point to be measured mapped to the image taken by the visual assistance camera through a transformation formula based on the first ranging value and the mapping parameters, so as to obtain the coordinates of the laser point on the image.

[0063] The embodiment of the present invention proposes a visually assisted laser measurement system. The laser ranging module and the navigation module are used to collect a plurality of data such as a first ranging value from a laser ranging point to a laser point to be measured, the coordinates of the antenna phase center position and the attitude information. The first coordinate acquisition module is then used to calculate the coordinates of the laser point in space. The collected data can be automatically measured by the device without manual adjustment of the device, simplifying the device installation steps, thereby improving the overall accuracy of the laser measurement. The second coordinate acquisition module is used to map the laser ranging value with the image taken by the auxiliary camera, and then the mapping parameters are used to calculate the coordinates of the laser point on the image. Although the auxiliary camera is used, its internal and external parameters are not used, so that it can adapt to more camera models and improve the accuracy of laser measurement. Finally, the auxiliary information message generation module generates an auxiliary information message for all the collected laser point related information, and converts it into a layer through the layer overlay module and overlays it on the preview image of the visual auxiliary camera, so that the position of the laser point can be more intuitively obtained. Therefore, it is achieved that there is no need to manually adjust the device and calibrate the internal and external parameters of the camera, thereby improving the accuracy of laser measurement.

[0064] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

[0065] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0066] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

Claims

1. A visually assisted laser measurement method, characterized in that: include: Obtain a first distance measurement value from a laser distance measurement point to a laser point to be measured; Obtain antenna phase center position coordinates and attitude information; Calculate the laser point coordinates based on the first ranging value, the antenna phase center position coordinates and the attitude information; Based on the first distance measurement value and the image captured by the vision-assisted camera, the coordinates of the laser point on the image are calculated by mapping parameters; Generate an auxiliary information message based on the first ranging value, the antenna phase center position coordinates, the attitude information, the laser point coordinates and the coordinates of the laser point on the image; Based on the auxiliary information message, an auxiliary information layer is generated and superimposed on the preview image of the visual assistance camera, so that the laser measurement result is presented on the preview image of the visual assistance camera.

2. A visually assisted laser measurement method as claimed in claim 1, characterized in that: Calculating the laser point coordinates based on the first ranging value, the antenna phase center position coordinates and the attitude information includes: Based on the posture information, calculating the laser direction vector; Multiplying the first distance measurement value by the laser direction vector to obtain a position vector; The antenna phase center position coordinates are added to the position vector to obtain the laser point coordinates.

3. A visually assisted laser measurement method as claimed in claim 1, characterized in that: Based on the first distance measurement value and the image captured by the visual auxiliary camera, the coordinates of the laser point on the image are calculated by mapping parameters, including: Calibrate mapping parameters of laser points on the image captured by the vision-assisted camera; Based on the first ranging value and the mapping parameter, the coordinates of the laser point to be measured mapped to the image taken by the vision-assisted camera are calculated by a transformation formula to obtain the coordinates of the laser point on the image.

4. A visually assisted laser measurement method as claimed in claim 3, characterized in that: The mapping parameters of the calibration laser point on the image taken by the visual auxiliary camera include: Acquire a plurality of second ranging values ​​from the laser ranging point to a plurality of laser points; Identify corresponding positions of a plurality of laser points in the image by a preset image processing algorithm, and obtain position coordinates of the plurality of laser points corresponding to the image captured by the visual auxiliary camera; Constructing the three-dimensional coordinates of each laser point based on the second distance measurement values ​​of the laser points and the position coordinates of the laser points corresponding to the image captured by the visual auxiliary camera; The three-dimensional coordinates of each laser point are nonlinearly optimized using a preset mapping algorithm to obtain mapping parameters of the laser point on the image taken by the vision-assisted camera.

5. A visually assisted laser measurement method as claimed in claim 4, characterized in that: The nonlinear optimization of the three-dimensional coordinates of each laser point by a preset mapping algorithm to obtain mapping parameters of the laser point on the image captured by the visual auxiliary camera includes: Construct the target mapping function and initialize the mapping parameters; Inputting the three-dimensional coordinates of each laser point into the target mapping function, performing gradient calculation on the mapping parameters, and obtaining the gradient value of each mapping parameter; By using a preset gradient descent algorithm, the gradient value of each mapping parameter is updated until a preset update requirement is met, thereby obtaining the mapping parameter of the laser point on the image taken by the vision-assisted camera.

6. A visually assisted laser measurement method as claimed in claim 1, characterized in that: Based on the first ranging value, the antenna phase center position coordinates, the attitude information, the laser point coordinates and the coordinates of the laser point on the image, an auxiliary information message is generated, including: Encoding the first ranging value, the antenna phase center position coordinates, the attitude information, the laser point coordinates, and the coordinates of the laser point on the image to obtain a plurality of data segments conforming to a preset message structure; The plurality of data segments conforming to the preset message structure are combined and encapsulated to obtain an auxiliary information message.

7. A visually assisted laser measurement method as claimed in claim 1, characterized in that: Based on the auxiliary information message, an auxiliary information layer is generated and the auxiliary information layer is superimposed on the preview image of the visual auxiliary camera, so that the laser measurement result is presented on the preview image of the visual auxiliary camera, including: Decoding the auxiliary information message to obtain an auxiliary information combination; Based on the auxiliary information combination, construct an auxiliary information layer; The position and transparency of the auxiliary information layer are adjusted, and the auxiliary information layer that meets the preview requirements is superimposed on the preview image of the visual auxiliary camera, so that the laser measurement result is presented on the preview image of the visual auxiliary camera.

8. A visually assisted laser measurement method according to any one of claims 1 to 7, characterized in that: Also includes: Update the first distance measurement value, antenna phase center position coordinates and attitude information of the laser point to be measured in real time, recalculate the laser point coordinates and verify the laser measurement results; The mapping parameters are updated based on the real-time updated laser point coordinates and the preview image of the vision-assisted camera corresponding to the laser measurement result.

9. A vision-assisted laser measurement system, characterized in that: include: Laser ranging module, navigation module, first coordinate acquisition module, second coordinate acquisition module, auxiliary information message generation module and layer overlay module; The laser ranging module is used to obtain a first ranging value from a laser ranging point to a laser point to be measured; The navigation module obtains the antenna phase center position coordinates and attitude information; The first coordinate acquisition module is used to calculate the laser point coordinates based on the first ranging value, the antenna phase center position coordinates and the attitude information; The second coordinate acquisition module is used to calculate the coordinates of the laser point on the image through mapping parameters based on the first distance measurement value and the image taken by the visual auxiliary camera; The auxiliary information message generation module is used to generate an auxiliary information message based on the first ranging value, the antenna phase center position coordinates, the posture information, the laser point coordinates and the coordinates of the laser point on the image; The layer overlay module is used to generate an auxiliary information layer based on the auxiliary information message and overlay the auxiliary information layer on the preview image of the visual auxiliary camera, so that the laser measurement result is presented on the preview image of the visual auxiliary camera.

10. A visually assisted laser measurement system as claimed in claim 9, characterized in that: The second coordinate acquisition module is used to calculate the coordinates of the laser point on the image through mapping parameters based on the first distance measurement value and the image taken by the visual auxiliary camera, including: Mapping parameter calibration module and laser point mapping module; The mapping parameter calibration module is used to calibrate the mapping parameters of the laser point on the image taken by the visual auxiliary camera; The laser point mapping module is used to calculate the coordinates of the laser point to be measured mapped to the image taken by the visual auxiliary camera through a transformation formula based on the first ranging value and the mapping parameters, so as to obtain the coordinates of the laser point on the image.

Images