Method and device for calibrating field large-view-field binocular camera of small unmanned aerial vehicle

Abstract

The invention relates to the technical field of field binocular vision measurement, in particular to a method and a device for calibrating a field large-field binocular camera of a small unmanned aerial vehicle. The method comprises the following steps: based on an RTK technology, controlling a small unmanned aerial vehicle to move in a binocular camera view field range, and obtaining a space coordinate of a space control point unit in a world coordinate system; extracting image coordinates of image points corresponding to the space control point units by adopting a target identification positioning method; and calculating internal and external parameter initial values of the binocular camera according to a calibration method of the space coordinates of the space control point units and the image coordinates of the corresponding image points, and optimizing the internal and external parameter initial values to obtain calibration parameters. The field large-view-field binocular camera calibration method for the small unmanned aerial vehicle is based on the RTK technology, centimeter-level precision can be achieved, meanwhile, the unmanned aerial vehicle can carry control points to be distributed in the whole view field through track programming, and the field large-view-field binocular camera calibration method is high in calibration precision, simple and practical.

Description

Technical Field

[0001] The present invention relates to the technical field of field binocular vision measurement, and in particular to a method and device for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle. Background Art

[0002] Visual measurement systems are being widely used in scientific research and industrial sites. Their main task is to parse out the three-dimensional information of the corresponding scene from the captured two-dimensional image, such as the key position coordinates that characterize the geometric shape of the object. At present, visual measurement is widely used in weapon test ranges and aerospace test launch sites, playing a role in external ballistic measurement, guidance, reconnaissance, flight live recording, and physical characteristic parameter measurement. It is an important means of external measurement at the range.

[0003] Whether in image measurement or machine vision applications, camera parameter calibration is a very critical link. The accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the camera's work results. Therefore, only accurate camera calibration results can ensure the smooth progress of subsequent work. Camera parameters are divided into external parameters and internal parameters. Among them, the camera's external parameters include the position of the posture, that is, the rotation and translation matrix. The internal parameters of the camera include the image center, off-axis lens distortion, radial lens distortion, camera focal length, and the error of the system itself. The camera calibration method can be divided into three categories: visual active calibration technology, traditional calibration technology, and camera self-calibration technology. The traditional camera calibration method can achieve very high standards in calibration accuracy, but the calibration process is not flexible. It is suitable for situations where the calibration accuracy requirements are very high and the camera parameters do not change frequently. Camera self-calibration is the most flexible calibration method, but its accuracy is difficult to achieve a high standard. Its calibration does not require the use of calibration objects and is generally used in scenarios such as communication and virtual reality. The camera calibration method based on active vision requires some known motion information of the camera. Its calibration accuracy is higher than the self-calibration method, and it does not require the use of calibration objects. To a certain extent, it combines the advantages of the first two methods.

[0004] In the existing technology, the long-distance and large-field-of-view target detection system is based on the principle of binocular vision for three-dimensional reconstruction. An image acquisition device is arranged at a distance of about 100m to 1500m from the detection area to continuously monitor the field of view. The markers required by the traditional camera calibration method are difficult to accurately image at a distance of 100m, and the processing accuracy of large multi-feature point markers is required to be high; large-field-of-view calibration can also be performed with the help of total stations, trackers and other equipment, but in the wild, especially in open environments, there are no suitable control points in the field of view, and the application has limitations; the active vision calibration method must adjust the camera to perform geometric movements, such as translation and rotation, and the operation is relatively complicated. Therefore, in the case of long distances and large fields of view in the wild, camera calibration cannot apply the existing model. To this end, it is necessary to design a new technical solution to solve it. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a method and device for calibrating a small unmanned aerial vehicle's large-field binocular camera in the wild, which is based on RTK technology and has high calibration accuracy and is simple and practical.

[0006] To achieve the above objectives and other related objectives, the present invention provides a method for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle, comprising:

[0007] S1. Based on RTK technology, control the small UAV to move within the field of view of the binocular camera and obtain the spatial coordinates of the spatial control point unit in the world coordinate system;

[0008] S2, extracting the image coordinates of the image points corresponding to the spatial control point units by using a target recognition and positioning method;

[0009] S3. According to the calibration method of the spatial coordinates of the spatial control point unit and the image coordinates of the corresponding image point, the initial values ​​of the internal and external parameters of the binocular camera are calculated, and the calibration parameters are obtained after the initial values ​​of the internal and external parameters are optimized.

[0010] In one embodiment of the present invention, the space control point unit is a searchlight installed on the small UAV.

[0011] In one embodiment of the present invention, the spatial coordinates are obtained based on RTK technology, and the searchlight is in a world coordinate system with the antenna of the reference station as the origin.

[0012] In one embodiment of the present invention, the step S2 of extracting the image coordinates of the image points corresponding to the spatial control point units by using the target recognition and positioning method comprises:

[0013] S21, determining whether the binocular camera has local time;

[0014] S22. If local time exists, the time of the spatial coordinates of the spatial control point unit is aligned with the time of the image coordinates of the image point corresponding to the spatial coordinates; if local time does not exist, after the small drone takes off, the small drone is controlled to hover at a set spatial position to obtain a target that is relatively stationary in both the spatial position and the image position, and the spatial coordinate values ​​and image coordinate values ​​within this period of time are solved using a random sampling consistency algorithm.

[0015] In one embodiment of the present invention, the use of the random sampling consistency algorithm in step S22 to solve the spatial coordinate value and the image coordinate value within this period of time includes:

[0016] S221, randomly select k sample points from the sample set, estimate the current model parameters according to the sample mapping mathematical model, and obtain the current model estimation value;

[0017] S222, setting an error threshold t, calculating matching errors for all sample points according to the current model estimation value, determining sample points less than the error threshold t as inliers, and determining sample points greater than or equal to the error threshold t as outliers, and recording the inliers and the number of inliers under the current model estimation value;

[0018] S223, update the maximum number of inliers, repeat steps S221 and S222 until the number of iterations is greater than M, or the maximum number of inliers is greater than the set inlier number threshold, then exit the loop and enter step S224;

[0019] S224. Estimating model parameters of the interior point set corresponding to the maximum number of interior points to obtain an optimal model estimation value, wherein the optimal model estimation value includes a space coordinate value and an image coordinate value.

[0020] In one embodiment of the present invention, the calibration method in step S3 calculates the initial values ​​of the internal and external parameters of the binocular camera according to the spatial coordinates of the spatial control point unit and the image coordinates of the corresponding image point, and obtains the calibration parameters after optimizing the initial values ​​of the internal and external parameters, including:

[0021] Assume that the spatial coordinate value of the i-th feature point in space is (X, Y, Z), and the corresponding image coordinate value is (x, y). From the imaging model of central perspective projection superimposed with nonlinear lens distortion, it can be seen that the collinear equation described by each element of the projection matrix is:

[0022]

[0023] Among them, m0~m 11 are the elements of the projection matrix M, δ x , δ y is the radial aberration and tangential aberration of the binocular camera. The matrix M describes the central perspective projection relationship from the space point to the image point, which can be expanded into the following formula:

[0024]

[0025] Among them, (C x ,C y ) is the coordinate of the center point of the image, F x and F y is the equivalent focal length of horizontal and vertical imaging, T X , T Y , T Z is the translation vector in the X, Y, and Z directions, r0~r8 are the elements of the binocular camera rotation matrix R, in the form of:

[0026]

[0027] The present invention also provides a small unmanned aerial vehicle outdoor large field binocular camera calibration device, comprising:

[0028] RTK module, which is used to control the movement of small UAV within the field of view of the binocular camera based on RTK technology, and obtain the spatial coordinates of the spatial control point unit in the world coordinate system;

[0029] An image coordinate extraction module, used to extract the image coordinates of the image points corresponding to the spatial control point units by using a target recognition and positioning method;

[0030] The calibration parameter acquisition module is used to calculate the initial values ​​of the internal and external parameters of the binocular camera according to the calibration method of the spatial coordinates of the spatial control point unit and the image coordinates of the corresponding image points, and obtain the calibration parameters after optimizing the initial values ​​of the internal and external parameters.

[0031] The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to implement the above-mentioned small unmanned aerial vehicle outdoor large field binocular camera calibration method.

[0032] As described above, the method and device for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle of the present invention have the following beneficial effects:

[0033] (1) The method for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle in the field of the present invention is based on RTK technology and can achieve centimeter-level accuracy. At the same time, the unmanned aerial vehicle can carry control points throughout the entire field of view through track programming. The calibration method of the present invention has high calibration accuracy and is simple and practical.

[0034] (2) The method for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle in the field of view of the present invention obtains the spatial coordinates of the spatial control point unit in the local world coordinate system by controlling the movement of the unmanned aerial vehicle within the camera field of view, and uses the target recognition and positioning method to extract the image coordinates of the image points corresponding to the control points. Then, the spatial coordinates of the control points and the image coordinates of the corresponding image points are obtained according to the random sampling consistency algorithm, and the initial values ​​of the internal and external parameters of the camera are calculated. The present invention can be used for binocular camera calibration under outdoor large field of view conditions. The equipment is mature and has strong versatility, and has important theoretical research significance and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart of a method for calibrating a small unmanned aerial vehicle's large-field-of-view binocular camera in the wild provided in an embodiment of the present application.

[0036] Figure 2 A flowchart of step S2 of a method for calibrating a small unmanned aerial vehicle's large-field-of-view binocular camera in the wild provided in an embodiment of the present application.

[0037] Figure 3A flowchart of step S22 of a method for calibrating a small unmanned aerial vehicle's large-field-of-view binocular camera in the wild provided in an embodiment of the present application.

[0038] Figure 4 RTK measurement principle diagram of a small UAV outdoor large field of view binocular camera calibration method provided in an embodiment of the present application.

[0039] Figure 5 A drone flight trajectory diagram of a small drone outdoor large-field-of-view binocular camera calibration method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0041] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present invention in a schematic manner, and thus the illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0042] Terms such as first or second can be used to describe various components, but these components are not limited by the above terms. The above terms are used to distinguish one component from another component, for example, without departing from the scope of the concept according to the present disclosure, a first component can be referred to as a second component, and similarly, a second component can be referred to as a first component.

[0043] In addition, "connected / coupled" means that one component is directly electrically coupled to another component or is indirectly electrically coupled through another component. The singular form may include the plural form unless explicitly stated in the sentence. In addition, "include / comprise" or "includes / comprises" used in this specification indicates that one or more components, steps, operations, and elements exist or have been added. The specific structural or functional descriptions of the examples of the implementation of the concepts disclosed in this specification are only illustrated to describe the examples of the implementation of the concepts, and the examples of the implementation of the concepts can be implemented in various forms, but these descriptions are not limited to the examples of the implementation described in this specification.

[0044] According to the concept, various modifications and changes can be applied to the examples of the embodiments, so that the examples of the embodiments will be illustrated in the drawings and described in the specification. However, the examples of the embodiments according to the concept are not limited to the specific embodiments, but include all changes, equivalents or replacements included in the spirit and technical scope of the present disclosure.

[0045] It should be understood that when an element is described as being "coupled" or "connected" to another element, the element may be directly coupled or directly connected to the other element, or may be coupled or connected to the other element through a third element. Conversely, it should be understood that when an element is referred to as being "directly coupled to" or "directly coupled to" another element, no other element is placed between them. Other expressions describing the relationship between components (i.e., "between" and "directly between" or "adjacent to" and "directly adjacent to") need to be interpreted in the same manner.

[0046] The terms used in this specification are only used to describe specific examples of the embodiments and are not intended to limit the present disclosure. If there is no clear opposite meaning in the context, the singular form may include the plural form. In this specification, it should be understood that the term "including" or "having" indicates the presence of the features, quantities, steps, operations, components, parts or combinations thereof described in the specification, but the possibility of the presence or addition of one or more other features, quantities, steps, operations, components, parts or combinations thereof cannot be precluded.

[0047] If there is no contrary definition, all terms (including technical terms or scientific terms) used herein have the same meaning as those generally understood by ordinary technicians in the field. If the terms defined in the commonly used dictionary are not clearly defined in this specification, they should be interpreted as having the same meaning as in the context of the relevant technology, and not interpreted as ideal or overly formal meanings.

[0048] Descriptions of well-known components and processing techniques may be omitted so as not to unnecessarily obscure the embodiments of the disclosure.

[0049] Throughout the specification, the same reference numerals refer to the same elements. Thus, even if a reference numeral is not mentioned or described with reference to one figure, it may be mentioned or described with reference to another figure. Furthermore, even if a reference numeral is not shown in one figure, it may be mentioned or described with reference to another figure.

[0050] In addition, the logic level of the signal may be different or opposite to the described logic level. For example, a signal described as having a logic "high" level may alternatively have a logic "low" level, and a signal described as having a logic "low" level may alternatively have a logic "high" level.

[0051] The following will describe the various embodiments of the present disclosure in detail with reference to the accompanying drawings. However, it will be appreciated by those skilled in the art that in the various embodiments of the present disclosure, many technical details are provided in order to enable the reader to better understand the present disclosure. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present disclosure can be implemented.

[0052] See also Figure 1 , Figure 4 , Figure 1 A flowchart of a method for calibrating a small unmanned aerial vehicle's large-field-of-view binocular camera in the wild provided in an embodiment of the present application. Figure 4 The present invention provides a method for calibrating a small unmanned aerial vehicle (UAV) binocular camera with a large field of view in the field, comprising:

[0053] Step S1: Based on RTK technology, control the small UAV to move within the field of view of the binocular camera to obtain the spatial coordinates of the spatial control point unit in the world coordinate system.

[0054] Specifically, RTK technology, or real-time dynamic carrier phase differential technology, is an efficient technology based on the GNSS system that receives satellite signals and performs differential processing to achieve centimeter-level or even millimeter-level positioning accuracy. Its working principle is that the base station receives and processes satellite signals in real time, and then sends the observation data and station coordinate information to the mobile station through the data link. The mobile station combines the satellite signals it receives and the data from the base station for real-time differential processing to obtain high-precision positioning results.

[0055] The application of RTK technology in drones can calculate the precise position of the drone in real time by receiving satellite signals and combining them with data from ground base stations, thereby significantly improving the safety of the drone in flight and the accuracy of mission execution.

[0056] Specifically, the space control point unit is a searchlight installed on the small UAV.

[0057] Specifically, the spatial coordinates are obtained based on the RTK technology, and are the world coordinate system of the searchlight with the antenna of the reference station as the origin.

[0058] Step S2: extracting the image coordinates of the image points corresponding to the spatial control point units by using a target recognition and positioning method.

[0059] Step S3: Calculate the initial values ​​of the internal and external parameters of the binocular camera according to the calibration method of the spatial coordinates of the spatial control point unit and the image coordinates of the corresponding image points, and obtain calibration parameters after optimizing the initial values ​​of the internal and external parameters.

[0060] See also Figure 2 , Figure 2 A flowchart of step S2 of a method for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle in the field provided in an embodiment of the present application. In one embodiment of the present invention, the step S2 of extracting the image coordinates of the image points corresponding to the spatial control point units by using a target recognition and positioning method includes:

[0061] Step S21, determining whether the binocular camera has local time.

[0062] Step S22: if local time exists, the time of the spatial coordinates of the spatial control point unit is aligned with the time of the image coordinates of the image point corresponding to the spatial coordinates; if local time does not exist, after the small drone takes off, the small drone is controlled to hover at a set spatial position to obtain a target that is relatively stationary in both the spatial position and the image position, and the spatial coordinate values ​​and image coordinate values ​​within this period of time are solved using a random sampling consistency algorithm.

[0063] In one embodiment of the present invention, the method for calibrating a small unmanned aerial vehicle's large field of view binocular camera in the field of the present invention needs to obtain the spatial coordinates of a certain spatial control point at a certain moment and the image coordinates of its corresponding image point, so it is necessary to align the time of the obtained spatial coordinates of the spatial control point with the time of the image coordinates of its corresponding image point, and the image of the spatial control point should be easy to identify and select.

[0064] Selection of spatial control points:

[0065] The present invention adopts a small searchlight installed on the small UAV as a space control point. The searchlight has its own battery as a power source, which can ensure that the small UAV always stays lit during the flight and measures the distance from the center of the antenna to the center of the bulb.

[0066] Image recognition and positioning technology:

[0067] The method for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle in the field of the present invention needs to obtain the spatial coordinates of a certain spatial control point at a certain moment and the image coordinates of its corresponding image point, so it is necessary to unify the time of the spatial coordinates of the obtained spatial control point and the time of the image coordinates of its corresponding image point. If the binocular camera does not have local time, after the small unmanned aerial vehicle takes off, it is controlled to hover at a pre-set spatial point position, so that a period of relatively static targets in both spatial position and image can be obtained, and then the spatial coordinate values ​​and image coordinate values ​​of the control points during this period of time are solved by a random sampling consistency algorithm. In the same way, the small unmanned aerial vehicle is controlled to hover in the field of view of the binocular camera to obtain the spatial coordinates and image coordinates of the control points throughout the field of view, thus solving the problem of time unification of the spatial coordinates of the control points and the image coordinates of the corresponding image points of the binocular camera without local time in the case of no time unity.

[0068] The RTK technology of the small unmanned aerial vehicle outdoor large field binocular camera calibration method of the present invention controls the small unmanned aerial vehicle to move within the field of view of the binocular camera, obtains the spatial coordinates of the spatial control point unit in the local world coordinate system, and uses the target recognition and positioning method to extract the image coordinates of the control point corresponding to the image point. Then, the initial values ​​of the internal and external parameters of the camera are calculated according to the calibration method of the spatial coordinates of the control point and the image coordinates of the corresponding image point, and finally the accurate calibration parameters are obtained through optimization.

[0069] See also Figure 3 , Figure 4 , Figure 3 A flowchart of step S22 of a method for calibrating a small unmanned aerial vehicle's large-field-of-view binocular camera in the wild provided in an embodiment of the present application. Figure 4 The RTK measurement principle diagram of a small unmanned aerial vehicle outdoor large field binocular camera calibration method provided in an embodiment of the present application. In one embodiment of the present invention, the use of a random sampling consistency algorithm in step S22 to solve the spatial coordinate value and image coordinate value within this period of time includes:

[0070] Step S221: randomly select k sample points from the sample set, estimate the current model parameters according to the sample mapping mathematical model, and obtain the current model estimation value.

[0071] Step S222, set an error threshold t, calculate the matching error for all sample points according to the current model estimation value, judge the sample points less than the error threshold t as inliers, judge the sample points greater than or equal to the error threshold t as outliers, and record the inliers and the number of inliers under the current model estimation value.

[0072] Step S223, update the maximum number of inliers, repeat steps S221 and S222, until the number of iterations is greater than M, or the maximum number of inliers is greater than the set inlier number threshold, then exit the loop and enter step S224.

[0073] Step S224: perform model parameter estimation on the inlier point set corresponding to the maximum number of inliers to obtain an optimal model estimation value, wherein the optimal model estimation value includes a space coordinate value and an image coordinate value.

[0074] The random sampling consensus algorithm of the present invention is different from the general parameter estimation method such as least squares, which uses all points to estimate model parameters and then discards points with large errors. The random sampling consensus algorithm of the present invention uses as few points as possible to estimate the model, and then uses the remaining points to test the model, which makes up for the shortcomings of the general parameter estimation method and alleviates the impact of abnormal data on model parameter estimation when serious error data appears.

[0075] In one embodiment of the present invention, the calibration method in step S3 calculates the initial values ​​of the internal and external parameters of the binocular camera according to the spatial coordinates of the spatial control point unit and the image coordinates of the corresponding image point, and obtains the calibration parameters after optimizing the initial values ​​of the internal and external parameters, including:

[0076] Assume that the world coordinate value of the i-th feature point in space is (X, Y, Z), and the corresponding image coordinate value is (x, y). From the imaging model of central perspective projection superimposed with nonlinear lens distortion, it can be seen that the collinear equation described by each element of the projection matrix is:

[0077]

[0078] Among them, m0~m 11 are the elements of the projection matrix M, δ x , δ y is the radial aberration and tangential aberration of the binocular camera. The matrix M describes the central perspective projection relationship from the space point to the image point, which can be expanded as follows:

[0079]

[0080] Among them, (C x ,C y ) is the coordinate of the center point of the image, F x and F y is the equivalent focal length of horizontal and vertical imaging, T X , T Y , T Z is the translation vector in the X, Y, and Z directions, r0~r8 are the elements of the binocular camera rotation matrix R, in the form of:

[0081]

[0082] In order to calibrate more accurately, the method for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle of the present invention must artificially set evenly distributed control points in the binocular camera field of view. The present invention installs a searchlight on the small unmanned aerial vehicle as a control point, and places the reference part at the coordinate origin of the world coordinate system. Through the RTK technology, the precise spatial coordinate value of the searchlight on the small unmanned aerial vehicle in the world coordinate system with the antenna of the ground reference station as the origin can be obtained, and then the image of these control points is collected by the camera to be calibrated, and then the image point coordinates of each control point are extracted by a random sampling consistency algorithm to solve the internal and external parameter values ​​of the camera to be calibrated.

[0083] The present invention provides a method for calibrating a binocular camera with a large field of view in a small unmanned aerial vehicle in the field, and proposes a workflow for calibrating binocular camera parameters under the condition of a large field of view in the field, and the steps include:

[0084] Step 11) Aim the binocular camera to be calibrated in the direction and install a small searchlight as a spatial control point on the small UAV.

[0085] Step 12) Control the small UAV to fly slowly within the field of view of the binocular camera to be calibrated, and try to make the flight trajectory of the UAV cover the entire field of view of the binocular camera to be calibrated, and the distance between the UAV and the binocular camera should change in the depth direction.

[0086] Step 13) The small UAV is controlled to hover at the first control point position set in advance, and the binocular camera also needs to simultaneously record the image of the small searchlight on the small UAV as the control point.

[0087] Step 14) Repeat step 13) to obtain control point images in different depth directions.

[0088] Step 15) On the image with control points taken by the binocular camera, use the target tracking and positioning or manual point extraction method to extract each set of image coordinates of the searchlight on the small drone, and use the random sampling consensus algorithm image coordinate points as calibration control points, so that the internal and external parameters of the camera can be calibrated.

[0089] The present invention also provides a small unmanned aerial vehicle outdoor large field binocular camera calibration device, comprising:

[0090] RTK module, which is used to control the movement of small UAV within the field of view of the binocular camera based on RTK technology, and obtain the spatial coordinates of the spatial control point unit in the world coordinate system;

[0091] An image coordinate extraction module, used to extract the image coordinates of the image points corresponding to the spatial control point units by using a target recognition and positioning method;

[0092] The calibration parameter acquisition module is used to calculate the initial values ​​of the internal and external parameters of the binocular camera according to the calibration method of the spatial coordinates of the spatial control point unit and the image coordinates of the corresponding image points, and obtain the calibration parameters after optimizing the initial values ​​of the internal and external parameters.

[0093] The present invention also proposes an electronic device, which includes a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to implement the above-mentioned small unmanned aerial vehicle outdoor large field binocular camera calibration method. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components; the memory may include a random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The memory can also be an internal memory of the random access memory (RAM) type, and the processor and memory can be integrated into one or more independent circuits or hardware, such as: an application specific integrated circuit (ASIC). It should be noted that the computer program in the above-mentioned memory can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0094] The present invention also proposes a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned small unmanned aerial vehicle outdoor large field binocular camera calibration method. The computer-readable storage medium can be an electronic medium, a magnetic medium, an optical medium, an electromagnetic medium, an infrared medium or a semiconductor system or a propagation medium. The computer-readable storage medium can also include a semiconductor or solid-state memory, a magnetic tape, a removable computer disk, a random access memory (RAM), a read-only memory (ROM), a hard disk, and an optical disk. The optical disk can include a compact disk-read only memory (CD-ROM), a compact disk-read / write (CD-RW), and a DVD.

[0095] In summary, the method for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle of the present invention obtains the spatial coordinates of the spatial control point unit in the local world coordinate system by controlling the movement of the unmanned aerial vehicle within the field of view of the camera, and uses the target recognition and positioning method to extract the image coordinates of the image points corresponding to the control points. Then, the spatial coordinates of the control points and the image coordinates of the corresponding image points are obtained according to the random sampling consistency algorithm, and the initial values ​​of the internal and external parameters of the camera are calculated. The present invention can be used for binocular camera calibration under outdoor large field of view conditions. The equipment is mature and has strong versatility, and has important theoretical research significance and broad application prospects. The method for calibrating a binocular camera with a large field of view for a small unmanned aerial vehicle of the present invention is based on RTK technology, which can achieve centimeter-level accuracy. At the same time, the unmanned aerial vehicle can carry control points throughout the entire field of view through track programming. The calibration of the present invention is high in accuracy, simple and practical.

[0096] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for calibrating a small unmanned aerial vehicle's large field binocular camera in the field, characterized in that: include: S1. Based on RTK technology, control the small UAV to move within the field of view of the binocular camera and obtain the spatial coordinates of the spatial control point unit in the world coordinate system; S2, extracting the image coordinates of the image points corresponding to the spatial control point units by using a target recognition and positioning method; S3. According to the calibration method of the spatial coordinates of the spatial control point unit and the image coordinates of the corresponding image point, the initial values ​​of the internal and external parameters of the binocular camera are calculated, and the calibration parameters are obtained after the initial values ​​of the internal and external parameters are optimized.

2. The method for calibrating a small unmanned aerial vehicle's large field of view binocular camera in the field according to claim 1, characterized in that: The space control point unit is a searchlight installed on the small UAV.

3. The method for calibrating a small unmanned aerial vehicle's large field of view binocular camera in the field according to claim 2, characterized in that: The spatial coordinates are obtained based on RTK technology, and are in a world coordinate system where the antenna of the reference station is the origin.

4. The method for calibrating a small unmanned aerial vehicle's large field of view binocular camera in the field according to claim 1, characterized in that: The step S2 of extracting the image coordinates of the image points corresponding to the spatial control point units by using the target recognition and positioning method comprises: S21, determining whether the binocular camera has local time; S22. If local time exists, the time of the spatial coordinates of the spatial control point unit is aligned with the time of the image coordinates of the image point corresponding to the spatial coordinates; if local time does not exist, after the small drone takes off, the small drone is controlled to hover at a set spatial position to obtain a target that is relatively stationary in both the spatial position and the image position, and the spatial coordinate values ​​and image coordinate values ​​within this period of time are solved using a random sampling consistency algorithm.

5. The method for calibrating a small unmanned aerial vehicle's outdoor large-field binocular camera according to claim 4 is characterized in that: The use of the random sampling consistency algorithm in step S22 to solve the spatial coordinate value and the image coordinate value within this period of time includes: S221, randomly select k sample points from the sample set, estimate the current model parameters according to the sample mapping mathematical model, and obtain the current model estimation value; S222, setting an error threshold t, calculating matching errors for all sample points according to the current model estimation value, determining sample points less than the error threshold t as inliers, and determining sample points greater than or equal to the error threshold t as outliers, and recording the inliers and the number of inliers under the current model estimation value; S223, update the maximum number of inliers, repeat steps S221 and S222 until the number of iterations is greater than M, or the maximum number of inliers is greater than the set inlier number threshold, then exit the loop and enter step S224; S224. Estimating model parameters of the interior point set corresponding to the maximum number of interior points to obtain an optimal model estimation value, wherein the optimal model estimation value includes a space coordinate value and an image coordinate value.

6. The method for calibrating a small unmanned aerial vehicle's large field of view binocular camera in the field according to claim 1, characterized in that: The calibration method in step S3 calculates the initial values ​​of the internal and external parameters of the binocular camera according to the spatial coordinates of the spatial control point unit and the image coordinates of the corresponding image point, and obtains the calibration parameters after optimizing the initial values ​​of the internal and external parameters, including: Assume that the spatial coordinate value of the i-th feature point in space is (X, Y, Z), and the corresponding image coordinate value is (x, y). From the imaging model of central perspective projection superimposed with nonlinear lens distortion, it can be seen that the collinear equation described by each element of the projection matrix is: Among them, m0~m 11 are the elements of the projection matrix M, δ x ,δ y is the radial aberration and tangential aberration of the binocular camera. The matrix M describes the central perspective projection relationship from the space point to the image point, which can be expanded into the following formula: Among them, (C x ,C y ) is the coordinate of the center point of the image, F x and F y is the equivalent focal length of horizontal and vertical imaging, T X , T Y , T Z is the translation vector in the X, Y, and Z directions, r0~r8 are the elements of the binocular camera rotation matrix R, in the form of:

7. A small unmanned aerial vehicle field large field binocular camera calibration device, characterized in that: include: RTK module, which is used to control the movement of small UAV within the field of view of the binocular camera based on RTK technology, and obtain the spatial coordinates of the spatial control point unit in the world coordinate system; An image coordinate extraction module, used to extract the image coordinates of the image points corresponding to the spatial control point units by using a target recognition and positioning method; The calibration parameter acquisition module is used to calculate the initial values ​​of the internal and external parameters of the binocular camera according to the calibration method of the spatial coordinates of the spatial control point unit and the image coordinates of the corresponding image points, and obtain the calibration parameters after optimizing the initial values ​​of the internal and external parameters.

8. An electronic device, comprising a processor and a memory, wherein the memory stores program instructions, characterized in that: The processor runs program instructions to implement the small unmanned aerial vehicle outdoor large field of view binocular camera calibration method as described in any one of claims 1 to 6.

Images