Visual assistance method, system and storage medium

Abstract

Provided in the present application are a visual assistance method, a system, a device and a storage medium. A visual assistance measurement system comprises a laser ranging module, a navigation module, a calculation module and a visual assistance module. The method is characterized by comprising: acquiring coordinates of a specified point in a first coordinate system, and calculating pixel coordinates of the specified point; adding a location marker of the specified point into an environmental image acquired by means of the visual assistance module; acquiring a distance parameter between the laser ranging module and a laser irradiation point; on the basis of pixel coordinates of the laser irradiation point, adding a location marker of the laser irradiation point into the environmental image acquired by means of the visual assistance module; and, on the basis of the location marker of the laser irradiation point in the environmental image and the location marker of the specified point in the environmental image, adjusting the location of the laser irradiation point of the laser ranging module. The present application adds a visual assistance function during the layout or measurement process, thereby improving the working performance of measurement systems.

Description

Visual aids, systems and storage media Technical Field

[0001] This application relates to the field of satellite navigation technology, and in particular to a visual assistance method, system and storage medium. Background Technology

[0002] In the field of engineering surveying, construction layout is the process of marking the coordinates of points on design drawings or other known points in the actual working environment. Inertial navigation RTK (Real-time kinematic) receivers hold an irreplaceable position in construction layout applications.

[0003] Patent application CN116972818A discloses a stakeout method and a multifunctional measuring device. The problem with this device and method is that, in bright sunlight or other environments with poor laser visibility, it is difficult to determine the location of the laser illumination point using only the human eye. Due to these limitations, the applicability of this device and method is somewhat restricted, as strong ambient light can affect stakeout and measurement work. Summary of the Invention

[0004] To address the problems in the prior art, the purpose of this application is to provide a visual assistance method, system, and storage medium that adds visual assistance functions and improves the performance of the measurement system during the layout or measurement process.

[0005] This application provides a vision-assisted method for a vision-assisted measurement system, the vision-assisted measurement system including a laser ranging module, a navigation module, a calculation module, and a vision-assisted module, the method including the following steps:

[0006] The visual assistance module acquires an environmental image, and a camera coordinate system and a pixel coordinate system are established based on the environmental image.

[0007] Obtain the coordinates of a specified point in a first coordinate system, and calculate the pixel coordinates of the specified point using the calculation module. The first coordinate system is the coordinate system corresponding to the navigation module.

[0008] Based on the pixel coordinates of the specified point, add a position marker for the specified point to the environmental image obtained by the visual assistance module;

[0009] The laser ranging module acquires the laser illumination point and the distance parameter between the laser ranging module and the laser illumination point.

[0010] Based on the distance parameter between the laser ranging module and the laser irradiation point, the pixel coordinates of the laser irradiation point in the pixel coordinate system are calculated by the calculation module.

[0011] Based on the pixel coordinates of the laser irradiation point, a position marker for the laser irradiation point is added to the environmental image obtained by the visual assistance module.

[0012] The position of the laser irradiation point of the laser ranging module is adjusted according to the position marker of the laser irradiation point in the environmental image and the position marker of the designated point in the environmental image.

[0013] In some embodiments, adding a location marker for the specified point to the environmental image acquired by the visual assistance module based on the pixel coordinates of the specified point includes the following steps:

[0014] Determine whether the pixel coordinates of the specified point are within the range of the environmental image acquired by the visual assistance module;

[0015] If so, add the location marker of the specified point to the environmental image obtained through the visual assistance module;

[0016] If not, the pixel coordinate system is extended to include the pixel coordinates of the specified point, and a guiding direction marker is added according to the relative direction of the extended pixel coordinates of the specified point relative to the environment image.

[0017] In some embodiments, the pixel coordinates of the laser illumination point in the pixel coordinate system are calculated by the calculation module based on the distance parameter between the laser ranging module and the laser illumination point, including:

[0018] Based on the angle parameter between the laser ranging module and the visual assistance module, the coordinates of the laser ranging module in the camera coordinate system, and the distance parameter between the laser ranging module and the laser illumination point, the pixel coordinates of the laser illumination point in the pixel coordinate system are calculated by the calculation module based on the transformation matrix between the camera coordinate system and the pixel coordinate system.

[0019] In some embodiments, before the calculation module calculates the pixel coordinates of the laser illumination point in the pixel coordinate system, the following step is further included: calibrating the angle parameter between the laser ranging module and the vision assistance module:

[0020] Obtain the intrinsic parameters of the visual assistance module, and establish the transformation matrix between the camera coordinate system and the pixel coordinate system based on the intrinsic parameters of the visual assistance module;

[0021] Obtain the coordinates of the laser ranging module in the camera coordinate system;

[0022] Establish an equation relating the pixel coordinates of the laser illumination point to the intrinsic parameters of the visual assistance module, the angle parameter between the laser ranging module and the visual assistance module, and the coordinates of the laser ranging module in the camera coordinate system. In the equation, the angle parameter is an undetermined coefficient.

[0023] The laser ranging module emits multiple laser illumination points for calibration and acquires environmental images for calibration through the vision assistance module, thereby obtaining the coordinates of the laser ranging module in the camera coordinate system and the distance parameters between each laser illumination point for calibration and the laser ranging module.

[0024] Based on the coordinates of the laser ranging module in the camera coordinate system, the distance parameters between each laser illumination point used for calibration and the laser ranging module, and the relationship equation, the angle parameter between the laser ranging module and the visual assistance module is obtained by least squares fitting.

[0025] In some embodiments, the designated point is a point to be staked out; adjusting the position of the laser illumination point of the laser ranging module according to the position marker of the laser illumination point in the environmental image and the position marker of the designated point in the environmental image includes the following steps:

[0026] Based on the coordinates of the navigation module in the first coordinate system, the distance parameters between the laser ranging module and the laser illumination point, and the vector information between the laser ranging module and the navigation module, the coordinates of the laser illumination point in the first coordinate system are obtained.

[0027] The layout error is calculated based on the coordinates of the specified point in the first coordinate system and the coordinates of the laser irradiation point in the first coordinate system.

[0028] Based on the layout error, adjust the position of the laser irradiation point marked in the environmental image to obtain the actual position mark of the laser irradiation point in the environmental image;

[0029] The position of the laser irradiation point of the laser ranging module is adjusted according to the position marker of the actual laser irradiation point in the environmental image and the position marker of the designated point in the environmental image, until the position marker of the actual laser irradiation point coincides with the position marker of the designated point.

[0030] In some embodiments, after obtaining the location marker of the actual laser irradiation point in the environmental image, the following steps are further included:

[0031] Based on the location markers of the actual laser irradiation point and the designated point in the environmental image, directional markers are added to the environmental image.

[0032] In some embodiments, the lofting error includes the rotation angle and sway angle of the vision-assisted measurement system;

[0033] The error is calculated based on the coordinates of the specified point in the first coordinate system and the coordinates of the laser irradiation point in the first coordinate system, including the following steps:

[0034] Calculate the first heading angle of the vector from the specified point to the laser irradiation point based on the coordinates of the specified point in the first coordinate system and the coordinates of the laser irradiation point in the first coordinate system;

[0035] The second heading angle of the laser ray direction is calculated by adding the heading angle of the vision assistance system obtained in real time by the navigation module to the fixed deviation between the heading angle of the laser ranging module and the heading angle of the vision assistance system.

[0036] Calculate the first angle between the designated point and the horizontal plane of the first coordinate system, and calculate the second angle between the laser irradiation point and the horizontal plane of the first coordinate system;

[0037] Calculate the rotation angle of the visual aid measurement system based on the first heading angle and the second heading angle;

[0038] The sway angle of the visual aid measurement system is calculated based on the first included angle and the second included angle.

[0039] This application embodiment also provides a visual assistance measurement system for implementing the aforementioned visual assistance method, the system comprising:

[0040] Navigation module;

[0041] A laser ranging module is used to emit a laser and acquire the distance parameters between the laser ranging module and the laser illumination point.

[0042] A visual assistance module is used to acquire and display an environmental image, and add a location marker for the specified point and a location marker for the laser irradiation point to the environmental image;

[0043] The calculation module is used to calculate the pixel coordinates of the specified point, wherein the first coordinate system is the coordinate system corresponding to the navigation module and the calculated pixel coordinates of the laser irradiation point in the pixel coordinate system.

[0044] In some embodiments, the navigation module includes an inertial measurement unit and a GNSS positioning unit, the laser ranging module includes a laser emitting unit, a laser ranging module input unit and a laser ranging module control unit, and the vision assistance module includes a camera unit and a display unit.

[0045] This application also provides a computer-readable storage medium for storing a program that, when executed by a processor, implements the steps of the vision assistance method.

[0046] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0047] The visual assistance method, system, and storage medium of this application have the following beneficial effects:

[0048] The visual assistance method of this application is a visual assistance layout method or a visual assistance measurement method, which is implemented based on a visual assistance measurement system. During the layout or measurement process through the visual assistance measurement system, the coordinates of the specified point and the laser irradiation point in the pixel coordinate system are calculated by the calculation module, and the corresponding position marks are displayed in the environmental image to assist the user in adjusting the position of the laser irradiation point. This increases the visual assistance function, and even in the case of strong sunlight or other conditions with low laser visibility, accurate layout and measurement can be achieved, thus improving the working performance of the measurement system. Attached Figure Description

[0049] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0050] Figure 1 is a flowchart of a visual assistance method according to an embodiment of this application;

[0051] Figure 2 is a schematic diagram of the structure of a vision-assisted measurement system according to an embodiment of this application;

[0052] Figure 3 is a flowchart of a visual-assisted lofting method according to an embodiment of this application;

[0053] Figure 4 is a flowchart of a visual aid measurement method according to an embodiment of this application;

[0054] Figure 5 is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of this application. Detailed Implementation

[0055] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0056] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices. Although the terms "first" or "second," etc., are used in this specification to denote certain features, they are only for indicating function and not as a limitation on the number or importance of specific features.

[0057] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined. Therefore, the actual execution order may change depending on the specific circumstances.

[0058] As shown in Figure 1, this application provides a visual assistance method applied to a visual assistance measurement system. The visual assistance method includes the following steps:

[0059] S100: Obtain an environmental image from the visual assistance module, and establish a camera coordinate system and a pixel coordinate system based on the environmental image;

[0060] S200: Obtain the coordinates of a specified point in a first coordinate system, and calculate the pixel coordinates of the specified point through the calculation module. The first coordinate system is the coordinate system corresponding to the navigation module.

[0061] S300: Based on the pixel coordinates of the specified point, add a position marker for the specified point to the environmental image obtained by the visual assistance module;

[0062] S400: The laser irradiation point is obtained through the laser ranging module, and the distance parameter between the laser ranging module and the laser irradiation point is obtained;

[0063] S500: Based on the distance parameter between the laser ranging module and the laser irradiation point, the calculation module calculates the pixel coordinates of the laser irradiation point in the pixel coordinate system.

[0064] S600: Based on the pixel coordinates of the laser irradiation point, add a position marker of the laser irradiation point to the environmental image obtained by the visual assistance module;

[0065] S700: Adjust the position of the laser irradiation point of the laser ranging module according to the position marker of the laser irradiation point in the environmental image and the position marker of the designated point in the environmental image.

[0066] As shown in Figure 2, this application embodiment also provides a visual assistance measurement system for implementing the aforementioned visual assistance method, including:

[0067] Navigation module M100;

[0068] The laser ranging module M200 is used to emit a laser and acquire the distance parameters between the laser ranging module and the laser illumination point.

[0069] The visual assistance module M300 is used to acquire and display an environmental image, and add the location markers of the specified point and the laser irradiation point to the environmental image;

[0070] The calculation module M400 is used to calculate the pixel coordinates of the specified point, wherein the first coordinate system is the coordinate system corresponding to the navigation module and the calculated pixel coordinates of the laser irradiation point in the pixel coordinate system.

[0071] Specifically, in this embodiment, the visual assistance system further includes an antenna. The navigation module, laser ranging module, and visual assistance module are connected to the computing module. The navigation module includes an inertial measurement unit (IMU), a GNSS (Global Navigation Satellite System) positioning unit, and a Kalman filter. The GNSS positioning unit can acquire high-precision position information from the multi-functional measurement device, and the Kalman filter can acquire high-frequency position and attitude information from the IMU. The antenna works in conjunction with the navigation module to transmit and receive data. The laser ranging module includes a laser emitting unit, a laser ranging module input unit, and a laser ranging module control unit. The laser ranging module control unit is a CPU (Central Processing Unit), and the laser emitting unit is a single-line laser rangefinder that illuminates a point in space with a laser beam. By acquiring phase observation values ​​and processing them through the CPU, the distance information between the laser illumination point and the laser emitting point can be obtained, i.e., the distance information between the laser illumination point and the laser ranging module. The laser ranging module input unit is an input keyboard that controls laser emission and deactivation. The vision assistance module includes a camera unit and a display unit. The camera unit is a camera used to collect environmental images and transmit them to the vision assistance measurement system. The display unit is used to display the environmental images collected by the camera. In the displayed environmental images, corresponding position markers are added according to the pixel coordinates of the specified point and the pixel coordinates of the laser irradiation point calculated by the calculation module. It can also be used to display data collected by the navigation module and the laser ranging module.

[0072] The visual assistance method of this application is a visual assistance layout method or a visual assistance measurement method, which is implemented based on the aforementioned visual assistance measurement system. A visual assistance module consisting of a camera unit and a display unit is configured in the measurement system and related devices. Visual sensing is combined with laser ranging, inertial sensing and satellite positioning. During the layout or measurement process through the visual assistance measurement system, the coordinates of the specified point and the laser illumination point in the pixel coordinate system are calculated by the calculation module, and the corresponding position marks are displayed in the environmental image to assist the user in adjusting the position of the laser illumination point. This increases the visual assistance function, and even in the case of strong sunlight or other conditions with low laser visibility, accurate layout and measurement can be achieved, thus improving the working performance of the measurement system.

[0073] The following sections will use visual-assisted layout and visual-assisted measurement methods as examples to illustrate their implementation in specific applications.

[0074] Figure 3 shows a flowchart of a visually assisted lofting method according to an embodiment of this application. The designated point is the lofting point.

[0075] S11: Obtain the coordinates of the navigation module in the first coordinate system through the navigation module;

[0076] Specifically, the position vector of the antenna phase center is obtained by the GNSS positioning unit and projected into the first coordinate system. This coordinate is used as the coordinates of the navigation module in the first coordinate system. The origin of the first coordinate system is located at the center of the inertial measurement unit. The x-axis is parallel to the local horizontal plane and points north, the y-axis is parallel to the local horizontal plane and points east, and the z-axis, x-axis, and y-axis form a right-handed coordinate system.

[0077] S12: (corresponding to step S100 above) Obtain an environmental image from the visual assistance module, and establish a camera coordinate system and a pixel coordinate system based on the environmental image.

[0078] The origin of the camera coordinate system is the optical center of the visual aid module. The x-axis and y-axis are parallel to the X and Y axes of the acquired environmental image. The z-axis is the optical axis of the visual aid module and is perpendicular to the image plane. The pixel coordinate system is a rectangular coordinate system with the upper left corner of the image as the origin and pixels as the unit. The horizontal axis is u, with the positive direction to the right; the vertical axis is v, with the positive direction downward. u and v represent the number of rows and columns of pixels.

[0079] S13: (corresponding to step S200 above) Obtain the coordinates of the point to be lofted in the first coordinate system, and calculate the pixel coordinates of the point to be lofted through the calculation module;

[0080] Where u,v are the pixel coordinates of the point to be lofted; X,Y,Z are the coordinates of the point to be lofted in the camera coordinate system; K is the camera intrinsic parameter matrix of the visual assistance module; The external parameter rotation matrix between the vision assistance module and the inertial sensor; This is the rotation matrix between the first coordinate system and the inertial sensor coordinate system; Let be the coordinates of the point to be laid out in the first coordinate system; Let be the coordinates of the optical center of the vision assist module in the first coordinate system.

[0081] Specifically, obtaining the coordinates of the point to be laid out in the first coordinate system includes: manually inputting the projected coordinates of the point to be laid out in the first coordinate system via a keyboard, or selecting the point to be laid out from a point database and transmitting the projected coordinates of the point to be laid out in the first coordinate system to a vision-assisted measurement system, for example, via Bluetooth, WIFI or other communication methods.

[0082] S14: (corresponding to step S300 above) Based on the pixel coordinates of the point to be lofted, add the position marker of the point to be lofted to the environmental image obtained by the visual assistance module.

[0083] Specifically, step S14 includes the following steps:

[0084] Determine whether the pixel coordinates of the point to be lofted are within the range of the environmental image acquired by the visual assistance module;

[0085] If so, the location marker of the lofting point is added to the environmental image obtained by the visual assistance module; that is, a location marker is added to the pixel point in the environmental image corresponding to the pixel coordinates of the lofting point. The location marker is, for example, a specific graphic or a dot of a specific color.

[0086] If not, i.e., the pixel coordinates of the point to be lofted are outside the range of the environmental image acquired by the visual assistance module, then the pixel coordinate system is extended to include the pixel coordinates of the point to be lofted. Based on the relative direction of the extended pixel coordinates of the point to be lofted relative to the environmental image, a guiding direction mark is added according to the relative direction. This guiding direction mark can be, for example, an arrow pointing from the current environmental image view to the pixel coordinates of the point to be lofted, or other guiding graphic shapes.

[0087] S15: Calibrate the laser ranging module, which includes the following steps:

[0088] Obtain the intrinsic parameters of the visual assistance module, and establish a transformation matrix between the camera coordinate system and the pixel coordinate system based on the intrinsic parameters of the visual assistance module; the transformation matrix is ​​specifically formulated as follows:

[0089] Where fx, fy, cx, and cy are in pixels and are camera intrinsic parameters. X, Y, and Z are coordinates in the camera coordinate system, and u and v are coordinates in the pixel coordinate system.

[0090] Obtain the coordinates of the laser ranging module in the camera coordinate system;

[0091] Specifically, the coordinates of the laser emission point in the camera coordinate system are taken as the coordinates of the laser ranging module in the camera coordinate system, with the initial position being (X0, Y0, Z0).

[0092] Establish an equation relating the pixel coordinates of the laser illumination point to the intrinsic parameters of the visual assistance module, the angle parameter between the laser ranging module and the visual assistance module, and the coordinates of the laser ranging module in the camera coordinate system. In the equation, the angle parameter is an undetermined coefficient.

[0093] Specifically, let α be the angle between the laser emitting unit and the camera unit in the ZOX plane, β be the angle between the laser emitting unit and the camera unit in the ZOY plane, and the angle between the laser emitting unit and the Z-axis be... The distance from the laser emission point to the laser illumination point is d, and the laser illumination point is P. Then, the coordinates of point P projected onto the X-axis in the camera coordinate system are: Xp = X0 + d*sinα

[0094] Similarly, we can obtain the coordinates of point P projected onto the Y-axis in the camera coordinate system as: Yp=Y0+d*sinβ

[0095] The coordinates of point P projected onto the Z-axis in the camera coordinate system are:

[0096] Therefore, the transformation matrix between the camera coordinate system and the pixel coordinate system can be expressed as:

[0097] From the above formula, we can obtain:

[0098] Further transformation yields the following relational equation:

[0099] The laser ranging module emits multiple laser illumination points for calibration and acquires environmental images for calibration through the vision assistance module. The coordinates of the laser ranging module in the camera coordinate system are obtained, and the distance parameters between each laser illumination point for calibration and the laser ranging module are obtained. Each distance parameter is then substituted into the above relational equation.

[0100] Based on the coordinates of the laser ranging module in the camera coordinate system, the distance parameters between each laser illumination point used for calibration and the laser ranging module, and the relationship equation, the angle parameters α and β between the laser ranging module and the visual assistance module are obtained by least squares fitting.

[0101] S16: Obtain the coordinates of the laser ranging module in the camera coordinate system;

[0102] S17: (corresponding to step S400) Obtain the laser illumination point through the laser ranging module, and obtain the distance parameter between the laser ranging module and the laser illumination point.

[0103] S18: (corresponding to step S500): Based on the distance parameter between the laser ranging module and the laser irradiation point, the pixel coordinates of the laser irradiation point in the pixel coordinate system are calculated by the calculation module.

[0104] Specifically, based on the angle parameter between the laser ranging module and the visual assistance module, the coordinates of the laser ranging module in the camera coordinate system, and the distance parameter between the laser ranging module and the laser illumination point, the pixel coordinates of the laser illumination point in the pixel coordinate system are calculated by the calculation module based on the transformation matrix between the camera coordinate system and the pixel coordinate system, as shown in the following formula:

[0105] S19: (corresponding to step S600) Based on the pixel coordinates of the laser irradiation point, add a position marker of the laser irradiation point to the environmental image obtained by the visual assistance module.

[0106] S110: (corresponding to step S700) Adjust the position of the laser irradiation point of the laser ranging module according to the position mark of the laser irradiation point in the environmental image and the position mark of the point to be staked out in the environmental image.

[0107] Specifically, step S110 includes the following steps:

[0108] Initialize the vision-assisted measurement system;

[0109] Obtain vector information between the laser emitting unit of the laser ranging module and the navigation module;

[0110] Based on the coordinates of the navigation module in the first coordinate system, the distance parameters between the laser ranging module and the laser illumination point, and the vector information between the laser ranging module and the navigation module, the coordinates of the laser illumination point in the first coordinate system are obtained.

[0111] The layout error is calculated based on the coordinates of the point to be laid out in the first coordinate system and the coordinates of the laser irradiation point in the first coordinate system. In this embodiment, the layout error includes the rotation angle and sway angle of the visual aid measurement system. Specifically, the layout error is calculated using the following steps:

[0112] The first heading angle α1 of the vector from the point to be laid out to the laser illumination point is calculated based on the coordinates of the point to be laid out in the first coordinate system and the coordinates of the laser illumination point in the first coordinate system. The formula for calculating the first heading angle α1 is as follows:

[0113] Among them, T E and T N L represents the east and north components of the coordinates of the point to be staked out in the first coordinate system, respectively. E and L N These represent the eastward and northward components of the coordinates of the laser irradiation point in the first coordinate system, respectively.

[0114] The second heading angle α2 of the laser ray direction is obtained by adding the heading angle of the vision-assisted measurement system acquired in real time by the navigation module to the fixed deviation between the heading angle of the laser ranging module and the heading angle of the vision-assisted measurement system.

[0115] Calculate the first angle β1 between the point to be laid out and the horizontal plane of the first coordinate system (the angle between the vector pointing from the origin of the first coordinate system to the point to be laid out and the horizontal plane of the first coordinate system), and calculate the second angle β2 between the laser irradiation point and the horizontal plane of the first coordinate system (the angle between the vector pointing from the origin of the first coordinate system to the laser irradiation point and the horizontal plane of the first coordinate system).

[0116] The rotation angle Δα of the visual aid measurement system is calculated based on the first heading angle and the second heading angle.

[0117] The sway angle Δβ = β1 - β2 of the visual aid measurement system is calculated based on the first included angle and the second included angle.

[0118] Based on the layout error, the position of the laser irradiation point marked in the environmental image is adjusted to obtain the actual position mark of the laser irradiation point in the environmental image. Specifically, the layout error can be displayed on the screen. After the operator adjusts the laser irradiation point according to the layout error, the calculation module adjusts the position of the laser irradiation point marked in the environmental image to obtain the actual position mark of the laser irradiation point in the environmental image.

[0119] Based on the location markers of the actual laser irradiation point and the location markers of the point to be lofted in the environmental image, a directional marker is added to the environmental image; the directional marker is, for example, an arrow pointing from the actual laser irradiation point to the directional point, or other graphic with a guiding effect;

[0120] The position of the laser irradiation point of the laser ranging module is adjusted according to the position mark of the actual laser irradiation point in the environmental image and the position mark of the point to be staked out in the environmental image, until the position mark of the actual laser irradiation point coincides with the position mark of the point to be staked out.

[0121] In this embodiment, the visual-assisted lofting method further includes an initialization process for the parameters of the visual-assisted measurement system before step S11. This initialization may include, for example, initializing the error of the inertial measurement unit.

[0122] Therefore, in this embodiment, the calculated pixel coordinates of the point to be lofted and the calculated pixel coordinates of the laser irradiation point are highlighted on the display unit to indicate their locations. Users can use the displayed image to assist in adjusting the position of the laser irradiation point, thus enabling accurate point location and normal lofting even in environments with strong ambient light and low visibility of the laser irradiation point. Furthermore, directional guide markers can be displayed when showing the point to be lofted and the laser irradiation point, allowing users to quickly adjust the position of the laser irradiation point based on these markers.

[0123] Figure 4 shows a flowchart of a vision-assisted measurement method according to an embodiment of this application. The designated point is the point to be measured.

[0124] S21: (corresponding to step S100 above) Obtain an environmental image from the visual assistance module, and establish a camera coordinate system and a pixel coordinate system based on the environmental image.

[0125] The origin of the camera coordinate system is the optical center of the visual aid module. The x-axis and y-axis are parallel to the X and Y axes of the acquired environmental image. The z-axis is the optical axis of the visual aid module and is perpendicular to the image plane. The pixel coordinate system is a rectangular coordinate system with the upper left corner of the image as the origin and pixels as the unit. The horizontal axis is u, with the positive direction to the right; the vertical axis is v, with the positive direction downward. u and v represent the number of rows and columns of pixels.

[0126] S22: (corresponding to step S200 above) Obtain the coordinates of the point to be measured in the first coordinate system, and calculate the pixel coordinates of the point to be measured through the calculation module. The calculation method of the pixel coordinates of the point to be measured can refer to the calculation method of the pixel coordinates of the point to be lofted above.

[0127] Specifically, obtaining the coordinates of the point to be measured in the first coordinate system includes: manually inputting the projected coordinates of the point to be measured in the first coordinate system via a keyboard, or selecting the point to be measured from a point database, and transmitting the projected coordinates of the point to be measured in the first coordinate system to the vision-assisted measurement system, for example, via Bluetooth, WIFI or other communication means.

[0128] S23: (corresponding to step S300 above) Based on the pixel coordinates of the point to be measured, add a position marker of the point to be measured to the environmental image obtained by the visual assistance module.

[0129] Specifically, step S23 includes the following steps:

[0130] Determine whether the pixel coordinates of the point to be measured are within the range of the environmental image acquired by the visual assistance module;

[0131] If so, a position marker for the point to be measured is added to the environmental image obtained by the visual assistance module; that is, a position marker is added to the pixel point in the environmental image corresponding to the pixel coordinates of the point to be measured. This position marker is, for example, a specific graphic or a dot of a specific color.

[0132] If not, i.e., the pixel coordinates of the point to be measured are outside the range of the environmental image acquired by the vision assistance module, then the pixel coordinate system is extended to include the pixel coordinates of the point to be measured. Based on the relative direction of the extended pixel coordinates of the point to be measured relative to the environmental image, a guiding direction mark is added according to the relative direction. This guiding direction mark can be, for example, an arrow pointing from the current environmental image view to the pixel coordinates of the point to be measured, or other guiding graphic shapes.

[0133] S24: (corresponding to step S400) Obtain the laser illumination point through the laser ranging module, and obtain the distance parameter between the laser ranging module and the laser illumination point.

[0134] S25: (corresponding to step S500): Based on the distance parameter between the laser ranging module and the laser irradiation point, the pixel coordinates of the laser irradiation point in the pixel coordinate system are calculated by the calculation module.

[0135] Specifically, based on the angle parameter between the laser ranging module and the visual assistance module, the coordinates of the laser ranging module in the camera coordinate system, and the distance parameter between the laser ranging module and the laser illumination point, the pixel coordinates of the laser illumination point in the pixel coordinate system are calculated by the calculation module based on the transformation matrix between the camera coordinate system and the pixel coordinate system, as shown in the following formula:

[0136] S26: (corresponding to step S600) Based on the pixel coordinates of the laser irradiation point, add a position marker of the laser irradiation point to the environmental image obtained by the visual assistance module.

[0137] S27: (corresponding to step S700) Adjust the position of the laser irradiation point of the laser ranging module according to the position mark of the laser irradiation point in the environmental image and the position mark of the point to be measured in the environmental image.

[0138] Specifically, step S27 includes the following steps:

[0139] Based on the location markers of the laser irradiation point and the measurement point in the environmental image, a directional marker is added to the environmental image; the directional marker is, for example, an arrow pointing from the laser irradiation point to the directional point, or other graphic with a guiding effect.

[0140] The position of the laser irradiation point of the laser ranging module is adjusted according to the position mark of the laser irradiation point in the environmental image and the position mark of the point to be measured in the environmental image, until the position mark of the laser irradiation point coincides with the position mark of the point to be measured;

[0141] S28: Perform measurement work based on the vision-assisted measurement system.

[0142] In this embodiment, the visual-assisted measurement method further includes a process of initializing the parameters of the visual-assisted measurement system before step S21.

[0143] Therefore, in this embodiment, the calculated pixel coordinates of the point to be measured and the calculated pixel coordinates of the laser illumination point are highlighted on the display unit to indicate their locations. Users can use the displayed image to assist in adjusting the position of the laser illumination point, thus enabling accurate point location and normal measurement even in environments with strong ambient light and low visibility of the laser illumination point. In addition to displaying the point to be measured and the laser illumination point, directional guide markers can also be displayed to facilitate faster adjustment of the laser illumination point's position.

[0144] This application also provides a computer-readable storage medium for storing a program that, when executed by a processor, implements the steps of the described vision-assisted method. In some possible implementations, various aspects of this application can also be implemented as a program product comprising program code that, when executed on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of this application described in the foregoing vision-assisted method section of this specification.

[0145] Referring to Figure 5, a program product 800 for implementing the above-described method according to an embodiment of this application is depicted. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and can be executed on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0146] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0147] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0148] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0149] When the program in the computer storage medium is executed by the processor, it implements the steps of the visual assistance method. Therefore, the computer storage medium can also achieve the technical effects of the visual assistance method.

[0150] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of this application and should not be construed as limiting the specific implementation of this application to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of this application, and all such modifications or substitutions should be considered within the scope of protection of this application.

Claims

1. A vision-assisted method for a vision-assisted measurement system, the vision-assisted measurement system comprising a laser ranging module, a navigation module, a calculation module, and a vision-assisted module, characterized in that, The method includes the following steps: The visual assistance module acquires an environmental image, and a camera coordinate system and a pixel coordinate system are established based on the environmental image. Obtain the coordinates of a specified point in a first coordinate system, and calculate the pixel coordinates of the specified point using the calculation module. The first coordinate system is the coordinate system corresponding to the navigation module. Based on the pixel coordinates of the specified point, add a position marker for the specified point to the environmental image obtained by the visual assistance module; The laser ranging module acquires the laser illumination point and the distance parameter between the laser ranging module and the laser illumination point. Based on the distance parameter between the laser ranging module and the laser irradiation point, the pixel coordinates of the laser irradiation point in the pixel coordinate system are calculated by the calculation module. Based on the pixel coordinates of the laser irradiation point, a position marker for the laser irradiation point is added to the environmental image obtained by the visual assistance module. The position of the laser irradiation point of the laser ranging module is adjusted according to the position marker of the laser irradiation point in the environmental image and the position marker of the designated point in the environmental image.

2. The visual assistance method according to claim 1, characterized in that, Based on the pixel coordinates of the specified point, a location marker for the specified point is added to the environmental image acquired by the visual assistance module, including the following steps: Determine whether the pixel coordinates of the specified point are within the range of the environmental image acquired by the visual assistance module; If so, add the location marker of the specified point to the environmental image obtained through the visual assistance module; If not, the pixel coordinate system is extended to include the pixel coordinates of the specified point, and a guiding direction marker is added according to the relative direction of the extended pixel coordinates of the specified point relative to the environment image.

3. The visual assistance method according to claim 1, characterized in that, Based on the distance parameter between the laser ranging module and the laser illumination point, the pixel coordinates of the laser illumination point in the pixel coordinate system are calculated by the calculation module, including: Based on the angle parameter between the laser ranging module and the visual assistance module, the coordinates of the laser ranging module in the camera coordinate system, and the distance parameter between the laser ranging module and the laser illumination point, the pixel coordinates of the laser illumination point in the pixel coordinate system are calculated by the calculation module based on the transformation matrix between the camera coordinate system and the pixel coordinate system.

4. The visual assistance method according to claim 3, characterized in that, Before the calculation module calculates the pixel coordinates of the laser illumination point in the pixel coordinate system, the following step is also included: calibrating the angle parameter between the laser ranging module and the vision assistance module: Obtain the intrinsic parameters of the visual assistance module, and establish the transformation matrix between the camera coordinate system and the pixel coordinate system based on the intrinsic parameters of the visual assistance module; Obtain the coordinates of the laser ranging module in the camera coordinate system; Establish an equation relating the pixel coordinates of the laser illumination point to the intrinsic parameters of the visual assistance module, the angle parameter between the laser ranging module and the visual assistance module, and the coordinates of the laser ranging module in the camera coordinate system. In the equation, the angle parameter is an undetermined coefficient. The laser ranging module emits multiple laser illumination points for calibration and acquires environmental images for calibration through the vision assistance module, thereby obtaining the coordinates of the laser ranging module in the camera coordinate system and the distance parameters between each laser illumination point for calibration and the laser ranging module. Based on the coordinates of the laser ranging module in the camera coordinate system, the distance parameters between each laser illumination point used for calibration and the laser ranging module, and the relationship equation, the angle parameter between the laser ranging module and the visual assistance module is obtained by least squares fitting.

5. The visual assistance method according to claim 1, characterized in that, The designated point is the point to be staked out. The position of the laser illumination point of the laser ranging module is adjusted according to the position markers of the laser illumination point and the designated point in the environmental image. This includes the following steps: Based on the coordinates of the navigation module in the first coordinate system, the distance parameters between the laser ranging module and the laser illumination point, and the vector information between the laser ranging module and the navigation module, the coordinates of the laser illumination point in the first coordinate system are obtained. The layout error is calculated based on the coordinates of the specified point in the first coordinate system and the coordinates of the laser irradiation point in the first coordinate system. Based on the stakeout error, the calculation module adjusts the position of the laser irradiation point marked in the environmental image to obtain the actual position mark of the laser irradiation point in the environmental image; The position of the laser irradiation point of the laser ranging module is adjusted according to the position marker of the actual laser irradiation point in the environmental image and the position marker of the designated point in the environmental image, until the position marker of the actual laser irradiation point coincides with the position marker of the designated point.

6. The visual assistance method according to claim 5, characterized in that, After obtaining the location marker of the actual laser irradiation point in the environmental image, the following steps are also included: Based on the location markers of the actual laser irradiation point and the designated point in the environmental image, directional markers are added to the environmental image.

7. The visual assistance method according to claim 5, characterized in that, The layout error includes the rotation angle and sway angle of the visual aid measurement system; The error is calculated based on the coordinates of the specified point in the first coordinate system and the coordinates of the laser irradiation point in the first coordinate system, including the following steps: Calculate the first heading angle of the vector from the specified point to the laser irradiation point based on the coordinates of the specified point in the first coordinate system and the coordinates of the laser irradiation point in the first coordinate system; The second heading angle of the laser ray direction is calculated by adding the heading angle of the vision assistance system obtained in real time by the navigation module to the fixed deviation between the heading angle of the laser ranging module and the heading angle of the vision assistance system. Calculate the first angle between the designated point and the horizontal plane of the first coordinate system, and calculate the second angle between the laser irradiation point and the horizontal plane of the first coordinate system; Calculate the rotation angle of the visual aid measurement system based on the first heading angle and the second heading angle; The sway angle of the visual aid measurement system is calculated based on the first included angle and the second included angle.

8. A vision-assisted measurement system, characterized in that, The system for implementing the visual assistance method according to any one of claims 1 to 7, the system comprising: Navigation module; A laser ranging module is used to emit a laser and acquire the distance parameters between the laser ranging module and the laser illumination point. A visual assistance module is used to acquire and display an environmental image, and add a location marker for the specified point and a location marker for the laser irradiation point to the environmental image; The calculation module is used to calculate the pixel coordinates of the specified point, wherein the first coordinate system is the coordinate system corresponding to the navigation module and the calculated pixel coordinates of the laser irradiation point in the pixel coordinate system.

9. The visual-assisted measurement system according to claim 8, characterized in that, The navigation module includes an inertial measurement unit and a GNSS positioning unit; the laser ranging module includes a laser emitting unit, a laser ranging module input unit, and a laser ranging module control unit; and the vision assistance module includes a camera unit and a display unit.

10. A computer-readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps of the visual assistance method according to any one of claims 1 to 7.

Images