High-altitude vision measurement system calibration method and device and readable storage medium

Abstract

The invention discloses a high-altitude vision measurement system calibration method and device and a readable storage medium, and the method comprises the steps: obtaining a first calibration result which is a calibration result based on a target placed on the ground; after the first calibration is completed, a binocular camera is used for collecting a first scene picture; increasing the pitch angle of the camera, and recording an image shot by aiming at the preset high-altitude area as a second scene picture; determining a pitch angle change value according to the difference between the pixel point data of the feature point of the feature object in the first scene picture and the pixel point data of the feature point of the feature object in the second scene picture; based on the first calibration result and the pitch angle change value, a second calibration result is determined, and the second calibration result represents calibration parameters when the binocular camera is aligned with the high-altitude area for shooting. The calibration target is set at a low altitude, visual calibration of a high-altitude scene is achieved, the unmanned aerial vehicle does not need to mount RTK calibration, and the system cost and the operation complexity are effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera calibration, in particular to a high-altitude visual measurement system calibration method, device and readable storage medium. BACKGROUND

[0002] In the fields of computer vision, remote sensing mapping and unmanned aerial vehicle autonomous navigation, the camera as the core perception sensor, the accuracy of its parameters is crucial. Camera calibration is the process of determining the internal parameters (such as focal length, principal point coordinates, distortion coefficients, etc.) and external parameters (such as position and attitude) of the camera, and its accuracy directly determines the success or failure of subsequent visual tasks (such as three-dimensional reconstruction, target positioning, stereo visual measurement). For near-ground or indoor scenes, the calibration technology is quite mature, and it is usually achieved by shooting a high-precision calibration board (such as a checkerboard or a circular dot array) of known size in a controllable environment.

[0003] However, when the application scenario extends to high altitudes of tens of meters or even hundreds of meters, the traditional calibration method faces severe challenges. The high-altitude scene has a wide field of view, and it is difficult to arrange and identify a large-size physical calibration board that meets the accuracy requirements in a natural environment. Even if it can be laid out, the manufacturing cost, transportation and deployment difficulty are extremely high. Therefore, the traditional near-distance calibration board method cannot be directly applied in the high-altitude scene.

[0004] To solve this problem, the prior art uses an unmanned aerial vehicle to mount an RTK for calibration, but this calibration method has high system cost, and high-precision RTK greatly increases the hardware overhead of the entire system. Secondly, the system integration and operation complexity is high, not only requires precise mechanical installation to ensure the stability of the relative relationship between sensors, but also needs to solve the problem of strict time synchronization. SUMMARY

[0005] In view of the above problems existing in the prior art, the present application provides a high-altitude visual measurement system calibration method, device and readable storage medium, which does not use an unmanned aerial vehicle to mount an RTK for calibration, effectively improving the efficiency of high-altitude calibration and reducing the system cost. The technical scheme comprises: In a first aspect, a high-altitude visual measurement system calibration method is provided, comprising the following steps: obtaining a first calibration result, the first calibration result being a calibration result of a binocular camera based on a target placed on the ground, the binocular camera shooting a ground target image; the field of view space when the binocular camera shoots the ground target image is recorded as a ground field of view space; After the first calibration is completed, a scene picture of a ground view space is collected by using the binocular camera, and is recorded as a first scene picture; the binocular camera is kept from being translated and horizontally rotated, and the pitch angle of the binocular camera is increased, so that the shooting range of the binocular camera is aligned with a predetermined high-altitude area, and an image shot by aligning with the predetermined high-altitude area is recorded as a second scene picture; a view space of the binocular camera when shooting the high-altitude area is recorded as a high-altitude view space, a bottom area of the high-altitude view space and a top area of the ground view space are overlapped, and the overlapped area contains a characteristic object; According to a difference between pixel point data of a feature point of the characteristic object in the first scene picture and pixel point data of the feature point in the second scene picture, a change value of the pitch angle of the binocular camera from shooting the first scene picture to shooting the second scene picture is determined. Based on the first calibration result and the change value of the pitch angle, a second calibration result is determined, and the second calibration result represents a calibration parameter of the binocular camera when shooting the high-altitude area.

[0006] In some embodiments, the method for obtaining the first calibration result comprises: Based on four coordinate values of the target point in a world coordinate system, a camera coordinate system, an image coordinate system and a pixel coordinate system, a first model representing a conversion relationship from the pixel coordinate system to the world coordinate system is obtained, the first model includes an intrinsic matrix and an extrinsic matrix of the camera, and the extrinsic matrix includes a rotation matrix and a translation matrix. The camera is calibrated by using the collected target image, and the intrinsic matrix and the extrinsic matrix parameters are obtained. A second model representing a relationship between the extrinsic matrix parameters and a first parameter is obtained, and a value of the first parameter is determined by using the determined extrinsic matrix parameters, the first parameter includes a rotation angle between the world coordinate system and the camera coordinate system, and the rotation angle includes rotation angles of rotation centers of three coordinate axes; the second model includes a first relationship between the rotation matrix and the rotation angle, and a second relationship between a coordinate position of any point in space in the camera coordinate system and the world coordinate system, and between the rotation matrix and the translation matrix.

[0007] In some embodiments, the method for obtaining the first model representing the conversion relationship from the pixel coordinate system to the world coordinate system based on the four coordinate values of the target point in the world coordinate system, the camera coordinate system, the image coordinate system and the pixel coordinate system comprises: , and a matrix is recorded as ; wherein, is a coordinate of the target in the pixel coordinate system; is a coordinate of the target in the world coordinate system; K is an intrinsic matrix, R is a rotation matrix, and T is a translation matrix.

[0008] In some embodiments, the second model representing the relationship between the extrinsic parameter matrix parameter and the first parameter is: Rotation matrix of the camera is: , represents the value of the element in the i-th row and the j-th column of the matrix, is related to the trigonometric function value of the rotation angle between the world coordinate system and the camera coordinate system corresponding to the camera; Translation matrix of the camera is: ; wherein, is the coordinate of the target point in the world coordinate system, is the coordinate of the target point in the camera coordinate system.

[0009] In some embodiments, the second calibration result is determined based on the first calibration result and the pitch angle change value, comprising: replacing the pitch angle in the extrinsic parameter matrix represented by the first parameter in the first calibration result with the sum of the pitch angle in the first calibration result and the pitch angle change value; obtaining the updated extrinsic parameter matrix; based on the intrinsic parameter matrix in the first calibration result and the updated extrinsic parameter matrix as the second calibration result.

[0010] In some embodiments, the pitch angle change value of the binocular camera from shooting the first scene picture to shooting the second scene picture is determined according to the difference between the pixel point data of the feature points of the feature object in the first scene picture and the pixel point data of the feature points of the feature object in the second scene picture, comprising: obtaining the pixel point position of the feature points of the feature object in the first scene picture, denoted as the first pixel point position; obtaining the pixel point position of the feature points in the second scene picture, denoted as the second pixel point position; determining the pitch angle change value based on the first pixel point position and the second pixel point position.

[0011] In some embodiments, the camera is calibrated using the collected target image to obtain the intrinsic parameter matrix and the extrinsic parameter matrix, comprising: after obtaining the coordinate values of the target points in the world coordinate system and the pixel coordinate system, performing normalization transformation on the coordinate values in the world coordinate system and the pixel coordinate system to eliminate the dimensional difference; The normalization method for the pixel coordinates of the target points comprises the following steps: obtaining the pixel coordinates of all target points (x ), i=1, 2,..., n, wherein n is the number of target points; calculating the centroid coordinates of all points, i.e. the average position of all points Subtract the average position from the pixel coordinates of all target points, so that the centroid of all points after translation moves to the origin, to obtain the updated pixel coordinates of the target points Calculate the average Euclidean distance of all points to the origin after translation Calculate the scaling factor When the coordinates of the points are two-dimensional coordinates When the coordinates of the points are three-dimensional coordinates Scale all points after translation to obtain normalized coordinates The normalization method for the world coordinate system of the target points is consistent with the normalization method for the pixel coordinates of the target points described above.

[0012] In some embodiments, the target image is used to calibrate the camera to obtain the intrinsic matrix and extrinsic matrix parameters, including: According to the collected target image, obtain the pixel coordinate values of a plurality of target points, and combine the world coordinate values of the target points and the first model to obtain the H matrix; For the H matrix, the H matrix is decomposed by RQ decomposition to obtain the intrinsic matrix and the extrinsic matrix; Optimize the matrix parameters by L-M algorithm to obtain the optimal solution of the intrinsic matrix and the extrinsic matrix.

[0013] In a second aspect, a high-altitude visual measurement system calibration device is provided, which comprises: A first calibration unit is used to obtain a first calibration result, which is a calibration result of a binocular camera based on a target placed on the ground, and the binocular camera captures the ground target image to obtain the calibration result of the binocular camera; The field of view space when the binocular camera captures the ground target image is recorded as the ground field of view space; A low-altitude high-altitude shooting unit is used to collect a scene picture of the ground field of view space after the first calibration is completed, which is recorded as the first scene picture; The binocular camera is kept from translating and horizontally rotating, and the pitch angle of the camera is increased, so that the shooting range of the binocular camera is aligned with the predetermined high-altitude area, and the image captured by aligning the predetermined high-altitude area is recorded as the second scene picture; The field of view space when the binocular camera captures the high-altitude area is recorded as the high-altitude field of view space, the bottom area of the high-altitude field of view space and the top area of the ground field of view space coincide, and the coincident area contains a characteristic object; ​​​​​​​an angle change determining unit configured to determine a pitch angle change value of the binocular camera from taking the first scene picture to taking the second scene picture according to a difference between pixel point data of the feature point in the first scene picture and pixel point data of the feature point in the second scene picture; a second calibration unit configured to determine a second calibration result based on the first calibration result and the pitch angle change value, the second calibration result representing a calibration parameter of the binocular camera when aiming at a high-altitude area.

[0014] In a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions, and the computer instructions are executed by a processor to implement steps of the calibration method of the high-altitude visual measurement system according to the first aspect.

[0015] The calibration method, the calibration device and the computer readable storage medium of the high-altitude visual measurement system have the following beneficial effects: the calibration target is set at a low altitude to obtain the first calibration result at the low altitude, and the pitch angle is increased based on the first calibration result to obtain the second calibration result facing the high altitude. The visual calibration of the high-altitude scene can be realized by only using the calibration target set at the low altitude, and the RTK calibration mounted on the unmanned aerial vehicle is not needed, so that the system cost and the operation complexity are effectively reduced. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a flowchart of the calibration method of the high-altitude visual measurement system in the embodiments of the present application; Figure 2 is a scene diagram of the calibration method of the high-altitude visual measurement system in the embodiments of the present application; Figure 3 is a structural diagram of the calibration device of the high-altitude visual measurement system in the embodiments of the present application. DETAILED DESCRIPTION

[0017] It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application.

[0018] Referring to Figure 1 In the embodiments of the present application, a calibration method of a high-altitude visual measurement system is provided, and the calibration method comprises the following steps: Step 1: obtaining a first calibration result, the first calibration result being a calibration result of a binocular camera obtained by the binocular camera when shooting a ground target image with the target placed on the ground; and a visual field space when the binocular camera shoots the ground target image being recorded as a ground visual field space; Step 2, after the first calibration is completed, a scene picture of the ground view space is collected by using the binocular camera, and is recorded as a first scene picture; the binocular camera is kept from being translated and horizontally rotated, and the pitch angle of the camera is increased, so that the shooting range of the binocular camera is aligned with the predetermined high-altitude area, and the image shot by aligning with the predetermined high-altitude area is recorded as a second scene picture; the view space when the binocular camera shoots the high-altitude area is recorded as a high-altitude view space, the bottom area of the high-altitude view space and the top area of the ground view space overlap, and the overlapping area contains a feature object, such as a mountain peak or a plant on the mountain, which can be shot by the low-altitude shooting scene and the high-altitude shooting scene at the same time; Step 3, according to the difference between the pixel point data of the feature point of the feature object in the first scene picture and the pixel point data of the feature point of the feature object in the second scene picture, the pitch angle change value of the binocular camera from shooting the first scene picture to shooting the second scene picture is determined. Step 4, based on the first calibration result and the pitch angle change value, a second calibration result is determined, which represents the calibration parameters when the binocular camera shoots the high-altitude area.

[0019] In the embodiment of the application, the low-altitude setting calibration target is used for calibration to obtain the first calibration result at low altitude, and the second calibration result facing the high altitude is obtained by increasing the pitch angle on the basis of the first calibration result. In the embodiment of the application, only the low-altitude setting calibration target is used to realize the visual calibration of the high-altitude scene, and the RTK calibration mounted on the unmanned aerial vehicle is not needed, thereby effectively reducing the system cost and the operation complexity.

[0020] Specifically, referring to Figure 2 wherein ① and ② are turntables under the camera, used to control the camera to raise the pitch angle, ③ and ④ are cameras, and the steps in the embodiment of the application are as follows: First, the binocular camera is aligned to shoot the ground area, the binocular camera forms a visual intersection space as an ABCD area, a calibration target is arranged in the area, camera large field calibration (including internal and external parameters) is performed, and the ground area image is collected; Then, the pitch angle of the camera is raised, the horizontal and roll angles are kept unchanged, the binocular camera forms a visual intersection space as an A1B1C1D1 area, and it can be understood that the top of the ABCD area and the bottom of the A1B1C1D1 area have an intersection, and the intersection area is a DEB1F area in the figure; The static feature points (such as mountain peaks or plants on the mountain) of the feature object in the DEB1F area are used, the pitch change is calculated according to the pixel position change of the same feature point before and after the pitch, the calibration parameter conversion is performed, and then the calibration parameters when shooting at high altitude (after the pitch is raised) are generated.

[0021] In one embodiment, the method for obtaining the first calibration result in step 1 above includes: Step 101: Based on the four coordinate values ​​of the target point in the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system, obtain a first model representing the transformation relationship from the pixel coordinate system to the world coordinate system. The first model includes the camera's intrinsic parameter matrix and extrinsic parameter matrix. The extrinsic parameter matrix includes a rotation matrix and a translation matrix. Step 102: Use the acquired target images to calibrate the camera and obtain the intrinsic and extrinsic parameter matrices. Step 103: Obtain a second model representing the relationship between the extrinsic parameter matrix parameters and the first parameter. Using the determined extrinsic parameter matrix parameters, determine the value of the first parameter. The first parameter includes the rotation angle between the world coordinate system and the camera coordinate system. The rotation angle includes the rotation angle around the three coordinate axes as rotation centers. The second model includes the first relationship between the rotation matrix and the rotation angle, and also includes the second relationship between the coordinates of any point in space in the camera coordinate system and the coordinates in the world coordinate system, and the rotation matrix and the translation matrix.

[0022] In this embodiment of the application, two expression models are used to express the rotation matrix in steps 101-103 above. One is of the form of ,in , ... One is the calibration parameter to be solved; the other is a rotation matrix containing rotation angle parameters. This is determined using acquired target images. , ... Then, the rotation angle values ​​in the rotation matrix expressed as rotation angle parameters can be determined.

[0023] For ease of calculation, in step 103, the origin of the camera coordinate system is selected as "any point in space". The coordinates of the "any point in space" in the camera coordinate system are (0, 0, 0). In the first relationship, the values ​​of each element in the rotation matrix are expressed as a function of the rotation angle parameter.

[0024] In some implementations, step 101 above, based on the four coordinate values ​​of the target point in the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system, obtains a first model representing the transformation relationship from the pixel coordinate system to the world coordinate system, including: Let the matrix be... ;in, These are the target's coordinates in the pixel coordinate system; These are the coordinates of the target in the world coordinate system; K is the intrinsic parameter matrix, R is the rotation matrix, and T is the translation matrix.

[0025] Generally, based on the pinhole imaging model, a conversion matrix from the pixel coordinate system to the world coordinate system can be obtained, assuming that the coordinates of a point in space in the world coordinate system are , the coordinates in the camera coordinate system are , the coordinates in the image coordinate system are , and the coordinates in the pixel coordinate system are , the conversion relationship between the image coordinate system and the pixel coordinate system is ; wherein is the coordinates of the origin of the image coordinate system in the pixel coordinate system; is the width of a single photosensitive element, is the height of a single photosensitive element; The conversion relationship between the camera coordinate system and the image coordinate system is ; The conversion relationship between the world coordinate system and the camera coordinate system is ; that is ; Therefore: The conversion relationship from the pixel coordinate system to the world coordinate system is ; wherein is the focal length of the camera, is the camera intrinsic matrix, , are the rotation matrix and the translation matrix of the extrinsic matrix, respectively.

[0026] In an embodiment, the second model representing the relationship between the extrinsic matrix parameters and the first parameters in the above-mentioned step 103 is The rotation matrix of the camera is : , denotes the element value in the i-th row and j-th column of the matrix, is related to the trigonometric function value of the rotation angle between the world coordinate system and the camera coordinate system corresponding to the camera; The translation matrix of the camera is : ; wherein is the coordinates of the target point in the world coordinate system, is the coordinates of the target point in the camera coordinate system. In an embodiment, the target point adopts the coordinate origin of the camera coordinate system, and the translation matrix of the camera in the second model is : .

[0027] Generally, the origin of the pixel coordinate system is located at the top left corner of the image, and the two coordinate axes of the pixel coordinate system are parallel to the image plane, or even parallel to the two sides of the image plane; the origin of the image coordinate system is located at the intersection of the optical axis and the image plane, and the two coordinate axes of the image coordinate system are parallel to the image plane, or even parallel to the two sides of the image plane; the origin of the camera coordinate system is located at the optical center of the lens, the x-axis and the y-axis are parallel to the two sides of the image plane, and the Z-axis is the optical axis of the lens and is perpendicular to the image plane; the world coordinate system can be freely determined according to actual conditions.

[0028] Suppose the world coordinate system is rotated by an angle A around the Z-axis (counterclockwise), then rotated by an angle B around the Y-axis, and then rotated by an angle E around the X-axis, and the camera coordinate system is obtained after three rotations, then: The rotation matrix P1 around the Z-axis is: ; The rotation matrix P2 around the Y-axis is: ; The rotation matrix P3 around the X-axis is: ; The rotation matrix from the world coordinate system to the camera coordinate system is : ; In an embodiment, if the position of the world coordinate system only needs to be rotated twice or once to obtain the camera coordinate system, the rotation matrix from the world coordinate system to the camera coordinate system can be simplified, for example, if the world coordinate system is rotated by an angle A around the Z-axis and an angle E around the X-axis twice to obtain the camera coordinate system (i.e. angle B is 0 degrees), then the rotation matrix from the world coordinate system to the camera coordinate system is: ; In an embodiment, if one of the two cameras in the binocular camera is denoted as camera C1 and the other camera is denoted as camera C2, then the rotation matrix from the world coordinate system to the camera coordinate system of camera C1 is : wherein , , are the angles of rotation around the Z-axis, the Y-axis, and the X-axis in the camera coordinate system of camera C1 obtained by rotating the world coordinate system, respectively; The rotation matrix from the world coordinate system to the camera coordinate system of camera C2 is , which can be obtained in the same way.

[0029] a rotation matrix of the world coordinate system to the camera coordinate system of camera C1 is obtained a rotation matrix of the world coordinate system to the camera coordinate system of camera C2 is obtained Then, further, a translation matrix of the world coordinate system to the camera coordinate system of camera C1 is obtained wherein is the coordinate of the target point in the world coordinate system, is the coordinate of the target point in the camera coordinate system of camera C1, and a translation matrix of the world coordinate system to the camera coordinate system of camera C2 is obtained

[0030] In an embodiment, step 4, the second calibration result is determined based on the first calibration result and the pitch angle change value, comprising: Step 401, replacing the pitch angle in the extrinsic parameter matrix represented by the first parameter in the first calibration result with the sum of the pitch angle in the first calibration result and the pitch angle change value; Step 402, obtaining the updated extrinsic parameter matrix; Step 403, taking the intrinsic parameter matrix in the first calibration result and the updated extrinsic parameter matrix as the second calibration result.

[0031] It should be noted that: it can be understood that the pitch angle change represents the change of the camera shooting field of view in the height direction, and the pitch angle change is determined based on the rotation angle of the camera coordinate system rotating around three coordinate axes in the extrinsic parameter matrix. The position of the camera has changed the pitch angle, which means that the camera coordinate system of the camera has rotated around one of the coordinate axes, and the pitch angle change value can be obtained based on the rotation angle. The extrinsic parameter matrix contains rotation angle parameters around the Z axis, the Y axis and the X axis. If the position of the camera changes the pitch angle, one of the rotation angles in the extrinsic parameter matrix will change.

[0032] Commonly, the camera coordinate system is set as follows: the origin of the camera coordinate system is located at the optical center of the lens, the x axis and the y axis are respectively parallel to the two sides of the image plane, and the Z axis is the lens optical axis and is perpendicular to the image plane. Therefore, if the direction of the side of the image plane parallel to the ground is taken as the x axis, the pitch angle in the extrinsic parameter matrix is the angle of rotation of the world coordinate system around the x axis, and further, the pitch angle change value represents the rotation angle of the camera coordinate system of the camera after increasing (raising) the pitch angle to the rotation angle of the camera coordinate system of the camera before increasing (raising) the pitch angle.

[0033] The pitch angle in the extrinsic parameter matrix represented by the first parameter in the first calibration result is replaced by the sum of the pitch angle in the first calibration result and the pitch angle change value, that is: Suppose the pitch angle in the extrinsic parameter matrix represented by the first parameter is angle E, and the pitch angle change value​ , then the original angle E in the extrinsic matrix is replaced by (E + E ).

[0034] In an embodiment, the step 3, determining the pitch angle change value of the binocular camera from taking the first scene picture to taking the second scene picture according to the difference between the pixel point data of the feature point of the feature object in the first scene picture and the pixel point data of the feature point in the second scene picture, comprises: obtaining the pixel point position of the feature point of the feature object in the first scene picture as the first pixel point position; obtaining the pixel point position of the feature point in the second scene picture as the second pixel point position; determining the pitch angle change value based on the first pixel point position and the second pixel point position.

[0035] Specifically, determining the pitch angle change value based on the first pixel point position and the second pixel point position can be: calculating the height of the first pixel point position , calculating the height of the second pixel point position , and determining the pitch angle change value based on the height of the first pixel point position . . is the focal length of the camera. It should be noted that the pixel point position is determined in the camera coordinate system, the origin of the camera coordinate system is located at the optical center of the lens, the x-axis and the y-axis are respectively parallel to the two sides of the image plane, and the Z-axis is the optical axis of the lens and is perpendicular to the image plane.

[0036] In an embodiment, the step 102, calibrating the camera by using the collected target image to obtain the intrinsic matrix and extrinsic matrix parameters, comprises: After obtaining the coordinate values of the target point in the world coordinate system and the pixel coordinate system, the coordinate values in the world coordinate system and the pixel coordinate system are normalized to eliminate the dimensional difference; The normalization method comprises: normalizing the pixel coordinate value of the target point to the average distance from the far point ; and normalizing the world coordinate value of the target point to the average distance from the origin . The normalization method for the pixel coordinate of the target point comprises the following steps: obtaining the pixel coordinates of all target points (X , i=1, 2,..., n, where n is the number of target points; calculating the centroid coordinates of all points, i.e., the average position of all points (X ); ​​Subtract the average position from the pixel coordinates of all target points, so that the centroid of all points after translation moves to the origin, to obtain the updated pixel coordinates of the target points Calculate the average Euclidean distance of all points to the origin after translation Calculate the scaling factor When the coordinates of the points are two-dimensional coordinates When the coordinates of the points are three-dimensional coordinates Scale all points after translation to obtain normalized coordinates The normalization method for the world coordinate system of the target points is consistent with the normalization method for the pixel coordinates of the target points described above.

[0037] Further, a normalization matrix for the pixel coordinates of the points can be constructed For two-dimensional coordinates, the original coordinates are converted to Similarly, for three-dimensional world coordinates, a normalization matrix for the world coordinates of the points can be constructed

[0038] In the case of normalization processing, by collecting the pixel coordinates and world coordinates of multiple target points of the target image, the conversion matrix between the pixel coordinates and the world coordinates in the case of normalized coordinates can be obtained Therefore, the conversion matrix between the pixel coordinates and the world coordinates in the case of the original coordinate data before normalization is = This solving process does not consider the parallax, and preferably, the present application will solve the optimal calibration parameters by iteration using the L-M optimization algorithm.

[0039] In an embodiment, the step 102 of calibrating the camera using the collected target image to obtain the intrinsic matrix and extrinsic matrix parameters includes: According to the collected target image, obtain the pixel coordinate values of multiple target points, and combine the world coordinate values of the target points and the first model to solve the H matrix; For the H matrix, the H matrix is decomposed using RQ decomposition to obtain the intrinsic matrix and the extrinsic matrix; Optimize the matrix parameters by L-M algorithm to obtain the optimal solution of the intrinsic matrix and the extrinsic matrix. ​​​​​​​​​​​

[0040] The objective function used in the L-M algorithm is: wherein, represents the coordinate of the spatial point projected onto the image through the intrinsic parameters, extrinsic parameters and distortion coefficients, represents the actual coordinate of the spatial point in the image, P is the coordinate of the spatial point in the world coordinate system, K, R, T and distcoeffs are the intrinsic matrix, the rotation parameter of the extrinsic matrix, the translation parameter of the extrinsic matrix and the distortion coefficients, respectively.

[0041] The embodiment of the present application also provides a high-altitude visual measurement system calibration device, which comprises: a first calibration unit, configured to acquire a first calibration result, wherein the first calibration result is a calibration result of a binocular camera obtained by shooting a target placed on the ground, the shooting of the target on the ground is recorded as a ground field of view space; a low-altitude and high-altitude shooting unit, configured to, after the first calibration is completed, collect a scene picture of the ground field of view space by using the binocular camera, which is recorded as a first scene picture; keep the binocular camera from being translated and horizontally rotated, and increase the pitch angle of the camera, so that the shooting range of the binocular camera is aligned with a predetermined high-altitude area, and the image shot by aligning with the predetermined high-altitude area is recorded as a second scene picture; the field of view space when the binocular camera shoots the high-altitude area is recorded as a high-altitude field of view space, the bottom area of the high-altitude field of view space and the top area of the ground field of view space coincide, and the coincident area contains a feature object; an angle change determination unit, configured to determine a pitch angle change value of the binocular camera from shooting the first scene picture to shooting the second scene picture according to the difference between the pixel point data of the feature points of the feature object in the first scene picture and the pixel point data of the feature points of the feature object in the second scene picture; a second calibration unit, configured to determine a second calibration result based on the first calibration result and the pitch angle change value, wherein the second calibration result represents the calibration parameters of the binocular camera when shooting the high-altitude area.

[0042] The specific limitations of the high-altitude visual measurement system calibration device can be referred to the limitations of the high-altitude visual measurement system calibration method in the foregoing, which will not be described herein. The various units in the high-altitude visual measurement system calibration device described above can be realized by software, hardware and combinations thereof, in whole or in part. The various units described above can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the various units.

[0043] The embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to realize steps of the high-altitude vision measurement system calibration method. The computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage node, etc.

[0044] The present application is not limited to the above specific embodiments, and various modifications made by those skilled in the art based on the above concept without creative labor are all within the protection scope of the present application.

Claims

1. A high-altitude visual measurement system calibration method, characterized in that, The method comprises the following steps: Obtaining a first calibration result, which is a calibration result of a binocular camera based on a target placed on the ground and an image of the target taken by the binocular camera; and recording a field of view space of the binocular camera when taking the image of the target on the ground as a ground field of view space; After the first calibration is completed, a scene picture of the ground field of view space is collected by the binocular camera, which is recorded as a first scene picture; the binocular camera is kept from being translated and horizontally rotated, and the pitch angle of the camera is increased so that the shooting range of the binocular camera is aligned with a predetermined high-altitude area, and an image taken by the binocular camera when aiming at the predetermined high-altitude area is recorded as a second scene picture; Recording a field of view space of the binocular camera when taking the image of the high-altitude area as a high-altitude field of view space, the bottom area of the high-altitude field of view space and the top area of the ground field of view space overlap, and the overlapping area contains a feature object; According to the difference between the pixel point data of the feature point of the feature object in the first scene picture and the pixel point data of the feature point of the feature object in the second scene picture, the change value of the pitch angle of the binocular camera from taking the first scene picture to taking the second scene picture is determined; Based on the first calibration result and the change value of the pitch angle, a second calibration result is determined, which represents the calibration parameters of the binocular camera when aiming at the high-altitude area and taking the image.

2. The calibration method of a high-altitude vision measurement system according to claim 1, wherein, The method for obtaining the first calibration result comprises: Based on the four coordinate values of the target point in the world coordinate system, the camera coordinate system, the image coordinate system and the pixel coordinate system, a first model representing the conversion relationship from the pixel coordinate system to the world coordinate system is obtained, which includes the intrinsic matrix and the extrinsic matrix of the camera, and the extrinsic matrix includes a rotation matrix and a translation matrix; The camera is calibrated by using the collected target image to obtain the intrinsic matrix and the extrinsic matrix parameters; A second model representing the relationship between the extrinsic matrix parameters and the first parameters is obtained, and the value of the first parameters is determined by using the determined extrinsic matrix parameters, the first parameters including the rotation angle between the world coordinate system and the camera coordinate system, and the rotation angle including the rotation angle of rotation around three coordinate axes; the second model includes the first relationship between the rotation matrix and the rotation angle, and the second relationship between the coordinates of any point in space in the camera coordinate system and the world coordinate system, and the rotation matrix and the translation matrix.

3. The calibration method of a high-altitude vision measurement system according to claim 2, wherein, The first model representing the conversion relationship from the pixel coordinate system to the world coordinate system based on the four coordinate values of the target point in the world coordinate system, the camera coordinate system, the image coordinate system and the pixel coordinate system comprises: , the matrix of moments ; wherein, is the coordinate of the target in the pixel coordinate system; is the coordinate of the target in the world coordinate system; K is an intrinsic matrix, R is a rotation matrix, and T is a translation matrix.

4. The calibration method of a high-altitude vision measurement system according to claim 2, wherein, The second model representing the relationship between the extrinsic matrix parameters and the first parameters is: Rotation matrix of a camera is: , denotes the value of the element in the i-th row and j-th column of the matrix, based on the trigonometric value of the rotation angle between the world coordinate system and the camera coordinate system of the corresponding camera; Translation matrix of the camera is: ; wherein, is the coordinate of the target point in the world coordinate system, is the coordinate of the target point in the camera coordinate system.

5. The calibration method of a high-altitude vision measurement system according to claim 2, wherein, Based on the first calibration result and the change value of the pitch angle, the second calibration result is determined, which comprises: The pitch angle in the extrinsic matrix represented by the first parameters in the first calibration result is replaced by the sum of the pitch angle in the first calibration result and the change value of the pitch angle; An updated extrinsic matrix is obtained; The intrinsic matrix in the first calibration result and the updated extrinsic matrix are used as the second calibration result.

6. The calibration method of a high-altitude vision measurement system according to claim 1, wherein, The difference between the pixel point data of the feature point of the feature object in the first scene picture and the pixel point data in the second scene picture is used to determine the pitch angle change value of the binocular camera from shooting the first scene picture to shooting the second scene picture, and the method comprises the following steps: The pixel point position of the feature point of the feature object in the first scene picture is recorded as the first pixel point position; The pixel point position of the feature point in the second scene picture is recorded as the second pixel point position; Based on the first pixel point position and the second pixel point position, the pitch angle change value is determined.

7. The calibration method of a high-altitude vision measurement system according to claim 2, wherein, The method for calibrating the camera by using the collected target image, acquiring the intrinsic matrix and extrinsic matrix parameters, comprises the following steps: After the coordinate values of the target point in the world coordinate system and the pixel coordinate system are acquired, the coordinate values in the world coordinate system and the pixel coordinate system are normalized to eliminate the dimensional difference; The normalization method for the pixel coordinates of the target point comprises the following steps: Obtaining pixel coordinates of all target points ), i = 1, 2, …, n, wherein n is the number of target points; Calculate the centroid coordinates of all points, i.e. the average position of all points ); Subtracting the average position from the pixel coordinates of all target points so that the centroid of all points moves to the origin after translation of each point, obtaining updated pixel coordinates of the target points (x' i, y' i) (4) ) Calculate the average Euclidean distance of all points to the origin after translation ; Computing scaling factors , When the coordinates of the points are two-dimensional coordinates When the coordinates of the points are three-dimensional coordinates ; Scaling all points after translation to get normalized coordinates , ; ; The normalization method of the world coordinate system of the target point is consistent with the normalization method of the pixel coordinates of the target point described above.

8. The calibration method of a high-altitude vision measurement system according to claim 2, wherein, The method for calibrating the camera by using the collected target image, acquiring the intrinsic matrix and extrinsic matrix parameters, comprises the following steps: According to the collected target image, the pixel coordinate values of a plurality of target points are obtained, and the H matrix is solved by combining the world coordinate values of the target points and the first model; The H matrix is decomposed by using RQ decomposition to obtain the intrinsic matrix and the extrinsic matrix; The matrix parameters are optimized by using the L-M algorithm to obtain the optimal solution of the intrinsic matrix and the extrinsic matrix.

9. A high-altitude visual measurement system calibration device, characterized in that, Comprise: The first calibration unit is used for acquiring the first calibration result, and the first calibration result is the calibration result of the binocular camera based on the target placed on the ground, which is obtained by shooting the ground target image by the binocular camera; the field of view space when the binocular camera shoots the ground target image is recorded as the ground field of view space; The low-altitude high-altitude shooting unit is used for collecting the scene picture of the ground field of view space after the first calibration is completed, which is recorded as the first scene picture; the binocular camera is kept from translating and rotating horizontally, and the pitch angle of the camera is increased to make the shooting range of the binocular camera align with the predetermined high-altitude area, and the image shot by aligning with the predetermined high-altitude area is recorded as the second scene picture; The field of view space when the binocular camera shoots the high-altitude area is recorded as the high-altitude field of view space, the bottom area of the high-altitude field of view space and the top area of the ground field of view space coincide, and the feature object is contained in the coincident area; The angle change determination unit is used for determining the pitch angle change value of the binocular camera from shooting the first scene picture to shooting the second scene picture according to the difference between the pixel point data of the feature point of the feature object in the first scene picture and the pixel point data in the second scene picture; The second calibration unit is used for determining the second calibration result based on the first calibration result and the pitch angle change value, and the second calibration result represents the calibration parameters when the binocular camera shoots the high-altitude area.

10. A computer readable storage medium having stored thereon computer instructions, wherein, The instructions are executed by the processor to realize the steps of the high-altitude visual measurement system calibration method in any one of claims 1-8.

Images