A camera measurement method and system

Abstract

The invention discloses a video measurement method, which aims to measure the world coordinates of a point to be measured in real time, realize the measurement of the point where the object is pointed, and improve the measurement efficiency; at the same time, a video measurement system is provided, and its technical scheme is as follows: a video measurement method, involving a host, wherein the host is provided with a GNSS positioning module, an IMU module, a laser distance measurement module and a camera; the laser distance measurement module obtains the laser distance of the point to be measured; a real-time picture is obtained by the camera, and the pixel position of the point to be measured in the real-time picture is obtained according to the laser distance, and an identifier is displayed in the real-time picture, and the identifier marks the point to be measured; according to the laser distance, the coordinates of the point to be measured in the laser coordinate system are obtained, and the coordinates of the point to be measured in the laser coordinate system are first coordinates; the IMU module obtains the attitude angle of the host, and the GNSS positioning module obtains the world coordinates of the host, and according to the attitude angle of the host and the world coordinates, the first coordinates are converted to obtain the world coordinates of the point to be measured, and the invention belongs to the field of measurement technology.

Description

Technical Field

[0001] The present invention belongs to the field of measurement technology, and more specifically, relates to a video measurement method and system. Background Art

[0002] Existing photogrammetry technologies all involve taking multiple photos or recording videos with a high degree of overlap that contain the measured points, marking the measured points to be measured in the photos or videos, and then using the epipolar geometry and nonlinear optimization techniques to solve the problem based on the principle of 3D reconstruction to obtain the accurate coordinates of the measured points.

[0003] This method is sensitive to the texture requirements of the marked points. If there is no obvious texture, the modeling will fail. At the same time, it is sensitive to noise. If there is noise in the image, it may affect the reconstruction accuracy. Moreover, this method requires solving and cannot quickly obtain the accurate coordinates of the measured points. If the computing power is insufficient, the solving time will be longer. Summary of the invention

[0004] The main purpose of the present invention is to provide a video measurement method, which aims to measure the world coordinates of the point to be measured in real time, achieve measurement wherever pointed, and improve measurement efficiency; at the same time, a video measurement system is provided.

[0005] According to a first aspect of the present invention, there is provided a video measurement method, which involves a host, wherein the host is provided with a GNSS positioning module, an IMU module, a laser ranging module and a camera;

[0006] Step 1: The laser distance measurement module obtains the laser distance to the point to be measured;

[0007] Step 2: Obtain the real-time picture through the camera, obtain the pixel position of the point to be measured in the real-time picture according to the laser distance, and display the identifier in the real-time picture. The identifier will mark the point to be measured;

[0008] Step 3: According to the laser distance, the coordinates of the point to be measured in the laser coordinate system are obtained, and the coordinates of the point to be measured in the laser coordinate system are the first coordinates;

[0009] The IMU module obtains the attitude angle of the host, and the GNSS positioning module obtains the world coordinates of the host. According to the attitude angle and world coordinates of the host, the first coordinate is converted to obtain the world coordinates of the point to be measured.

[0010] In the above-mentioned video measurement method, in step 3, the first coordinate is converted into the body coordinate system of the host to obtain the second coordinate;

[0011] According to the attitude angle and world coordinates of the host, the second coordinate is converted to obtain the world coordinates of the point to be measured.

[0012] In the above-mentioned video measurement method, in step 3, the second coordinate is converted into the navigation coordinate system according to the attitude angle of the host to obtain the third coordinate;

[0013] The third coordinate is converted into the world coordinate system according to the world coordinate of the host computer to obtain the world coordinate of the point to be measured.

[0014] In the above-mentioned video measurement method, the second coordinate is P B (X B , Y B , Z B ), the third coordinate is P N (X N , Y N , Z N ), the transformation relationship between the second coordinate and the third coordinate is:

[0015]

[0016] Among them, R BN is the rotation matrix of the host's body coordinate system to the world coordinate system, T BN The translation matrix for transforming the host body coordinate system to the world coordinate system.

[0017] In the above-mentioned video measurement method, the compensation value of the point to be measured in the world coordinate is calculated according to the world coordinate and the third coordinate of the host;

[0018] Obtain the real world coordinates of the host according to the world coordinates of the host;

[0019] The compensation value is combined with the real world coordinates of the host to obtain the world coordinates of the point to be measured.

[0020] In the above-mentioned video measurement method, the latitude B, longitude L and altitude H are obtained according to the world coordinates of the host, and the third coordinate is P N (X N , Y N , Z N ), compensation value P dW (X dW , Y dW , Z dW ) is calculated as follows:

[0021] X dW =-sin(B)*cos(L)*X N -sin(L)*Y N +cos(B)*cos(L)*Z N

[0022] Y dW =-sin(B)*sin(L)*XN +cos(L)*Y N +cos(B)*sin(L)*Z N

[0023] Z dW =cos(B)*X N +sin(B)*Z N .

[0024] The above-mentioned video measurement method also includes a mobile terminal, which is communicatively connected with the host and on which a real-time image can be displayed.

[0025] In the above-mentioned video measurement method, the mobile terminal is a mobile phone terminal, a handheld terminal or a PC terminal.

[0026] According to a second aspect of the present invention, a video measurement system is provided, including a host, on which a GNSS positioning module, an IMU module, a laser ranging module and a camera are provided to implement the video measurement method as described in the first aspect.

[0027] One of the above technical solutions of the present invention has at least one of the following advantages or beneficial effects:

[0028] The present invention combines the camera with the laser ranging technology. The camera can obtain the real-time picture during measurement. The calibration relationship between the camera and the laser ranging module is used to display the identifier in the real-time picture, and the identifier is used to mark the point to be measured in real time. The measurement personnel can intuitively see the position of the point to be measured, which is convenient for alignment and confirmation of the point to be measured.

[0029] At the same time, the laser distance measurement module is used to obtain the laser distance of the measured point, the IMU module is used to obtain the attitude angle of the host, and the GNSS positioning module is used to obtain the world coordinates of the host. In this way, the coordinates of the measured point in the laser coordinate system are gradually converted to the world coordinate system, and the world coordinates of the measured point are obtained in real time, so that the measurement can be achieved wherever it is pointed, thereby improving the measurement efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention is further described below in conjunction with the accompanying drawings and embodiments;

[0031] Figure 1 It is a flow chart of the video measurement method of the present invention. DETAILED DESCRIPTION

[0032] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0033] Existing photogrammetry technologies all involve taking multiple photos or recording videos with a high degree of overlap that contain the measured points, marking the measured points to be measured in the photos or videos, and then using the epipolar geometry and nonlinear optimization techniques to solve the problem based on the principle of 3D reconstruction to obtain the accurate coordinates of the measured points.

[0034] This method is sensitive to the texture requirements of the marked points. If there is no obvious texture, the modeling will fail. At the same time, it is sensitive to noise. If there is noise in the image, it may affect the reconstruction accuracy. Moreover, this method requires solving and cannot quickly obtain the accurate coordinates of the measured points. If the computing power is insufficient, the solving time will be longer.

[0035] Reference Figure 1 As shown, the present invention provides a video measurement method, which combines a camera with a laser ranging technology, uses a camera to mark a point to be measured, and uses the laser ranging technology to obtain the precise distance of the point to be measured. Through coordinate conversion, the accurate coordinates of the point to be measured can be obtained in real time.

[0036] The video measurement method involves a host computer, on which a GNSS positioning module, an IMU module, a laser ranging module and a camera are arranged. The GNSS positioning module, the IMU module, the laser ranging module and the camera are fixed to the host computer in a rigid connection manner. The relative position relationship of each part will be calibrated before leaving the factory. Once the calibration is completed, the relative position relationship will be permanently determined and cannot be changed.

[0037] The GNSS positioning module is used to calculate the world coordinates of the host. It uses the base station and the mobile station to receive satellite signals at the same time. The base station provides high-precision location information through real-time observation and calculation, and then transmits this information to the mobile terminal through wireless communication. After the mobile terminal receives the data from the base station, it uses the base station data and the received satellite signals for differential calculation, thereby eliminating most errors and achieving high-precision real-time positioning. The mobile station calculates the world coordinates of the mobile station based on the coordinates of the base station. The mobile station is located on the host, so the world coordinates of the host can be calculated.

[0038] The IMU module is a device for measuring the three-axis attitude angle and acceleration of an object. Generally speaking, an IMU module contains three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the independent three-axis acceleration signals of the host in the carrier coordinate system, while the gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system, measures the angular velocity and acceleration of the host in three-dimensional space, and uses this to calculate the attitude of the host.

[0039] Before the host performs measurement, it is necessary to calibrate the IMU module, GNSS positioning module and laser ranging module. The calibration is performed by laying out several reference points with known coordinates. Specifically, several reference points with known coordinates are laid out first, and then the coordinates of these reference points are measured by the GNSS positioning module in a centering manner. Then, the distance from the host to these reference points and the attitude of the host are measured by the IMU module and the laser ranging module using the tilt measurement method. Finally, the coordinates measured by the GNSS positioning module are compared with the distance measured by the laser ranging module and the attitude of the host, so as to determine the calibration parameters between the IMU module, the GNSS positioning module and the laser ranging module.

[0040] At the same time, it is necessary to calibrate the external parameters between the IMU module and the camera. The external parameters between the IMU module and the camera are calibrated using the hand-eye calibration method. The hand-eye calibration method is a calibration method based on a calibration plate. Specifically, the calibration plate is fixed and cannot be moved during calibration. The image of the calibration plate is taken at multiple angles, and the coordinates and posture of the calibration plate in the world coordinate system are measured using tilt measurement. The position of the calibration plate in the image is used to determine the external parameter parameters between the IMU module and the camera.

[0041] Step 1: The laser distance measurement module obtains the laser distance to the point to be measured;

[0042] The laser distance measurement module can obtain the laser distance through the phase method or TOF time of flight method:

[0043] The phase method is a method of measuring distance by using the phase difference of light waves. Its basic principle is: the transmitter emits a beam of modulated light, which is reflected by the object to be measured and then received by the receiver. The transmitter and receiver measure the phase difference between the emitted light and the received light through digital signal processing technology (DSP), and then calculate the distance based on the speed of light and the wavelength of the modulated light.

[0044] The TOF time-of-flight method uses the flight time of light waves to measure distance. Its basic principle is: the transmitter emits a beam of light pulses, which are reflected by the object to be measured and then received by the receiver. The receiver records the time from the emission to the reception of the light pulse, and then calculates the distance based on the speed of light.

[0045] In this embodiment, the laser ranging module uses a phase method to measure distance, which has high measurement accuracy and a wide measurement range.

[0046] Step 2: Obtain the real-time picture through the camera, obtain the pixel position of the point to be measured in the real-time picture according to the laser distance, and display the identifier in the real-time picture. The identifier will mark the point to be measured;

[0047] Before the host is used for measurement, the laser ranging module and the camera need to be calibrated. The laser and the camera are calibrated by segmented calibration plus least squares fitting. Specifically, several reference points with known coordinates are first arranged, and then the laser points are calibrated at different reference points to correspond to the pixel positions in the camera image.

[0048] Moreover, before measuring, the camera will be calibrated with internal parameters, and the Zhang Zhengyou calibration method will be used to determine the internal parameters of the camera. Specifically, a calibration plate is fixed in front of the camera, and images of the calibration plate are taken at multiple angles. The position of the calibration plate in the image is used to determine the internal parameters of the camera;

[0049] Based on this, after obtaining the laser distance of the point to be measured, the pixel position of the point to be measured in the real-time picture can be obtained by measuring the previously calibrated data; then an identifier is set, and according to the pixel position, the identifier will be displayed in the real-time picture to mark the point to be measured. The measurement personnel can intuitively see the position of the point to be measured, which is convenient for alignment and confirmation of the point to be measured;

[0050] In practice, the laser used in the laser ranging module is not all visible light, but also invisible light. Moreover, invisible light is more used outdoors and can be used for long-distance ranging. When using invisible light for ranging, it is difficult to distinguish the points to be measured. In the method proposed in this application, a camera is used to obtain real-time images during measurement, and identifiers are set to mark the points to be measured, which is convenient for measurement personnel to perform measurement work.

[0051] In this embodiment, the identifier may be in the shape of a cross, a circle, or a square.

[0052] Step 3: According to the laser distance, the coordinates of the point to be measured in the laser coordinate system are obtained, and the coordinates of the point to be measured in the laser coordinate system are the first coordinates;

[0053] The laser coordinate system is a three-dimensional rectangular coordinate system. The origin is located at the laser emission point of the laser sensor. x and y are related to the structural characteristics of the laser sensor itself. x points to the long side, y points to the short side, and z points to the laser emission direction, which coincides with the laser beam. According to the laser distance, the first coordinate is P L (X L , Y L , Z L ), where X L =Y L =0, Z L =Laser distance;

[0054] Convert the first coordinate into the host machine's body coordinate system to obtain a second coordinate;

[0055] The host's body coordinate system is a three-dimensional rectangular coordinate system. The origin is located at the phase center of the GNSS positioning module on the host. The x, y, and z axes coincide with the three axes of the IMU module. The second coordinate is P B (X B , Y B , Z B ):

[0056]

[0057] R LB The rotation matrix for transforming the laser coordinate system to the host's body coordinate system is 3×3. LB is the translation matrix from the laser coordinate system to the host body coordinate system, and the matrix size is 3×1; R LB , T LB This includes rotation and translation parameters, which can be obtained through the previous calibration process;

[0058] The IMU module obtains the attitude angle of the host, and transforms the second coordinate into the navigation coordinate system according to the attitude angle of the host to obtain the third coordinate;

[0059] There are two commonly used navigation coordinate systems: North East Earth and Northeast Sky. The North East Earth is used here. The X axis points to the north direction of the host location, the Y axis points to the east, and the Z axis points to the ground. It is usually called the NED coordinate system.

[0060] The third coordinate is P N (X N , Y N , Z N ):

[0061]

[0062] R BN The rotation matrix for transforming the host's body coordinate system to the world coordinate system. The matrix size is 3×3, T BN It is the translation matrix for transforming the host body coordinate system to the world coordinate system. The matrix size is 3×1.

[0063] R BN It can be calculated from the attitude angle of the host measured by the IMU module; when the attitude of the host is measured by the IMU module, the roll angle roll is α, the pitch angle pitch is β, and the heading angle yaw is γ. According to the definition of the Euler angle coordinate system and the relevant conventions of the IMU attitude angle, the rotation matrix can be obtained as follows:

[0064]

[0065] Because the origin of the host's body coordinate system coincides with the origin of the navigation coordinate system, T BNAll values ​​are 0;

[0066] The GNSS positioning module obtains the world coordinates of the host, converts the third coordinates into the world coordinate system according to the world coordinates of the host, and obtains the world coordinates of the point to be measured;

[0067] When converting the navigation coordinate system to the world coordinate system, the latitude and longitude of the host must be considered. Different latitude and longitude positions will have different conversion results. Therefore, the coordinates of the host in the world coordinate system, namely, latitude, longitude and altitude (latitude, longitude, altitude), must be introduced, which are recorded as B, L, and H respectively. The latitude, longitude and altitude are obtained by the GNSS positioning module on the host.

[0068] The compensation value of the point to be measured in the world coordinate is P dW (X dW , Y dW , Z dW ), which is calculated as follows:

[0069] X dW =-sin(B)*cos(L)*X N -sin(L)*Y N +cos(B)*cos(L)*Z N

[0070] Y dW =-sin(B)*sin(L)*X N +cos(L)*Y N +cos(B)*sin(L)*Z N

[0071] Z dW =cos(B)*X N +sin(B)*Z N .

[0072] The real world coordinates P of the host sW (X sW , Y sW , Z sW ) can be obtained by converting latitude, longitude and height;

[0073] The world coordinates P of the point to be measured tW (X tW , Y tW , Z tW ) can be calculated by the following formula:

[0074]

[0075] Based on this, the world coordinates of the measured point can be measured in real time. When operating the host, the surveyor can use the identifier to visually see the measured point, which is simple and clear. At the same time, the coordinates can be converted to obtain the coordinates of the measured point in real time. The operation is very simple and there is no need to take multiple photos for solution, which greatly improves the measurement efficiency.

[0076] In this embodiment, the real-time picture can be presented on the mobile terminal, and the measurement personnel can use the mobile terminal to view the real-time picture, confirm whether the point to be measured is accurate, and then adjust the host;

[0077] At the same time, the world coordinates of the points to be measured will also be displayed on the mobile terminal for easy viewing by the surveyor. The mobile terminal can name and encode the points to be measured, and then record the world coordinates of the points to be measured.

[0078] Furthermore, the mobile terminal can send the world coordinates of the point to be measured to the cloud for storage.

[0079] According to a second aspect of the present invention, a photogrammetry system is provided, including a host, wherein the host is provided with a GNSS positioning module, an IMU module, a laser ranging module and a camera to implement the above-mentioned video measurement method.

[0080] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A video measurement method, characterized in that: A host is provided with a GNSS positioning module, an IMU module, a laser ranging module and a camera; Step 1: The laser distance measurement module obtains the laser distance to the point to be measured; Step 2: Obtain the real-time picture through the camera, obtain the pixel position of the point to be measured in the real-time picture according to the laser distance, and display the identifier in the real-time picture. The identifier will mark the point to be measured; Step 3: According to the laser distance, the coordinates of the point to be measured in the laser coordinate system are obtained, and the coordinates of the point to be measured in the laser coordinate system are the first coordinates; The IMU module obtains the attitude angle of the host, and the GNSS positioning module obtains the world coordinates of the host. The first coordinate is converted into the body coordinate system of the host to obtain the second coordinate. The second coordinate is converted into the navigation coordinate system according to the attitude angle of the host to obtain the third coordinate. The third coordinate is converted into the world coordinate system according to the world coordinate of the host to obtain the world coordinate of the point to be measured.

2. The videography measurement method according to claim 1, characterized in that: The second coordinate is P B (X B , Y B , Z B ), the third coordinate is P N (X N , Y N , Z N ), the transformation relationship between the second coordinate and the third coordinate is: ， Among them, R BN is the rotation matrix of the host's body coordinate system to the world coordinate system, T BN The translation matrix for transforming the host body coordinate system to the world coordinate system.

3. The videography measurement method according to claim 2, characterized in that: The compensation value of the point to be measured in the world coordinate is calculated according to the world coordinate and the third coordinate of the host; Obtain the real world coordinates of the host according to the world coordinates of the host; The compensation value is combined with the real world coordinates of the host to obtain the world coordinates of the point to be measured.

4. The videography measurement method according to claim 3, characterized in that: According to the host's world coordinates, we get the latitude B, longitude L and altitude H. The third coordinate is P N (X N , Y N , Z N ), compensation value P dW (X dW , Y dW , Z dW ) is calculated as follows: 。 5. The video measurement method according to claim 1, characterized in that: It also includes a mobile terminal, which is communicatively connected with the host and on which real-time images can be displayed.

6. The videography measurement method according to claim 5, characterized in that: The mobile terminal is a mobile phone terminal, a handheld terminal or a PC terminal.

7. A video measurement system, characterized in that: It comprises a host, on which a GNSS positioning module, an IMU module, a laser ranging module and a camera are provided to implement the video measurement method as described in any one of claims 1-6.

Images