Method for detecting calibration parameters, binocular vision system and agricultural drone

Abstract

The application provides a method for detecting a calibration parameter, a binocular vision system and an agricultural unmanned aerial vehicle. The method comprises: performing machine vision feature point extraction on a picture to obtain a first feature point pair set, the picture comprising pictures collected under N scenes, N being a positive integer; inputting the picture into a deep neural network to obtain a disparity map and filtering abnormal feature point pairs in the first feature point pair set on the disparity map to obtain a second feature point pair set; determining N rotation matrices in the N scenes according to the feature point pairs in the second feature point pair set; and determining whether the calibration parameter is used according to the number M of unit matrices in the N rotation matrices. In the embodiment of the application, the automatic detection of the calibration parameter can be realized without the help of artificial labor, so as to help update the calibration parameter in time when the calibration parameter cannot be used, and improve the precision of the binocular vision system.

Description

TECHNICAL FIELD

[0001] The present application relates to the binocular vision technical field, and more particularly, to a method for detecting calibration parameters, a binocular vision system and an agricultural unmanned aerial vehicle. BACKGROUND

[0002] The error of the binocular vision system will accumulate over time, so the accuracy of the calibration parameters is the key to ensuring the high-precision navigation of the binocular vision system. Before leaving the factory, the binocular vision system needs to be accurately calibrated to establish a mathematical model between the input and output of the system and compensate for the scale factor error and installation error of the camera. However, due to the influence of the external environment and the characteristics of the camera itself and many other factors, the error parameters of the binocular vision system will change after long-term use. In order to ensure the navigation accuracy of the system, the binocular vision system needs to be calibrated regularly by a calibration reference, which increases the complexity of the calibration process, and if the calibration is not timely, it may lead to a decrease in the precision of the binocular vision system. SUMMARY

[0003] The present application provides a method for detecting calibration parameters, a binocular vision system and an agricultural unmanned aerial vehicle, which can automatically detect the calibration parameters without the help of manual work, thereby helping to update the calibration parameters in time when the calibration parameters cannot be used continuously and improving the precision of the binocular vision system.

[0004] In a first aspect, a method for detecting calibration parameters is provided, which includes: obtaining calibration parameters of a binocular camera; performing stereoscopic rectification on a first picture collected by the binocular camera according to the calibration parameters of the binocular camera to obtain a second picture after rectification, the first picture including pictures collected under N scenes, N being a positive integer; performing feature point extraction of machine vision according to the second picture to obtain a first set of feature point pairs; inputting the second picture into a deep neural network to obtain a disparity map and filtering abnormal feature point pairs in the first set of feature point pairs on the disparity map to obtain a second set of feature point pairs; determining N rotation matrices in the N scenes according to the feature point pairs in the second set of feature point pairs; when M rotation matrices in the N rotation matrices are unit matrices and the ratio of M to N is greater than or equal to a preset threshold, determining that the calibration parameters can be used continuously, M being an integer greater than or equal to 0 and M being less than or equal to N; or when M rotation matrices in the N rotation matrices are unit matrices and the ratio of M to N is less than the preset threshold, determining that the calibration parameters cannot be used.

[0005] Based on the above technical solution, the feature point pairs extracted by the traditional machine vision and the disparity map output by the neural network can be used to screen the feature point pairs. The screened feature point pairs can be used to calculate the rotation vector in multiple scenes, so as to determine whether the calibration parameters are available through the rotation vector. In this way, the user does not need to periodically calibrate the binocular vision system through the calibration reference object, but updates the calibration parameters in time when the calibration parameters are not available, thereby improving the accuracy of the binocular vision system.

[0006] The feature point extraction based on the second picture is performed to obtain a first feature point pair set, including: extracting feature points in the second picture based on a brute force (BF) algorithm based on FAST and BRIEF (Oriented FAST and Rotated BRIEF, ORB) to obtain the first feature point pair set.

[0007] Optionally, whether the picture captured by the left eye in the current frame and the picture captured by the left eye in the previous frame belong to the same scene is determined according to the number of matched feature points on the picture captured by the left eye in the current frame and the picture captured by the left eye in the previous frame. For example, when the number of matched feature points is less than or equal to a preset number, it can be determined that the picture captured by the left eye in the current frame and the picture captured by the left eye in the previous frame do not belong to the same scene; otherwise, the picture captured by the left eye in the current frame and the picture captured by the left eye in the previous frame belong to the same scene.

[0008] In combination with the first aspect, in some implementations of the first aspect, the feature point extraction based on the second picture is performed to obtain a first feature point pair set, including: filtering pictures in which the number of feature points is less than or equal to a second preset number to obtain filtered pictures; performing feature point extraction based on the filtered pictures to obtain a third feature point pair set; and filtering feature point pairs in the third feature point pair set whose disparity values are less than or equal to a preset disparity value to obtain the first feature point pair set.

[0009] Based on the above technical solution, the number of feature point pairs in the picture can be used to screen the picture, and the feature point pairs in the feature point pair set whose disparity values are less than or equal to a preset disparity value (for example, the feature point pairs whose disparity values are less than or equal to 8 pixels, which are far away from the binocular camera, so they can be filtered out) are further screened out, so that the accuracy of the rotation matrix obtained through the feature point pairs can be further improved.

[0010] In some implementations of the first aspect, in the first aspect, the first feature point pair set includes a first feature point pair, the first feature point pair includes a first feature point and a second feature point, the first feature point is a feature point on an image captured by a left eye of the binocular camera, and the second feature point is a feature point on an image captured by a right eye of the binocular camera. The filtering of the abnormal feature point pairs in the first feature point pair set on the disparity map includes: predicting a third feature point of the first feature point on the image captured by the right eye according to the disparity map; and determining that the first feature point pair is an abnormal feature point pair when a depth error between the second feature point and the third feature point is greater than or equal to a preset error.

[0011] According to the above technical solution, the depth error between the second feature point obtained by the traditional machine vision and the third feature point obtained by the disparity map can be used to determine the abnormal feature point pairs in the feature point pair set. In this way, the accuracy of the rotation matrix obtained by the feature point pair can be further improved.

[0012] In some implementations of the first aspect, before the determining of the N rotation matrices from the feature point pairs in the second feature point pair set, the method further includes: determining that the number of the feature point pairs in the second feature point pair set is greater than or equal to a third preset number.

[0013] In some implementations of the first aspect, before the determining of the N rotation matrices from the feature point pairs in the second feature point pair set, the method further includes: determining that the number of the feature point pairs in the second feature point pair set is greater than or equal to a third preset number.

[0014] With reference to the second aspect, in some implementations of the second aspect, the processor is configured to: filter the second image to obtain a filtered image, wherein the filtering is performed on the second image based on a number of feature points in the second image being less than or equal to a second preset number; perform feature point extraction based on the filtered image to obtain a third set of feature point pairs; filter the third set of feature point pairs to obtain the first set of feature point pairs, wherein the filtering is performed on the third set of feature point pairs based on a disparity value of each feature point pair in the third set of feature point pairs being less than or equal to a preset disparity value.

[0015] With reference to the second aspect, in some implementations of the second aspect, the first set of feature point pairs includes a first feature point pair, the first feature point pair includes a first feature point and a second feature point, the first feature point is a feature point on an image captured by a left camera of the binocular camera, and the second feature point is a feature point on an image captured by a right camera of the binocular camera. The processor is configured to: predict, based on the disparity map, a third feature point of the first feature point on the image captured by the right camera; and determine that the first feature point pair is an abnormal feature point pair when a depth value error between the second feature point and the third feature point is greater than or equal to a preset error.

[0016] With reference to the second aspect, in some implementations of the second aspect, the processor is further configured to: determine that a number of feature point pairs in the second set of feature point pairs is greater than or equal to a third preset number before determining the N rotation matrices in the N scenes based on the feature point pairs in the second set of feature point pairs.

[0017] A third aspect provides an agricultural unmanned aerial vehicle, which includes a memory configured to store computer instructions and a processor configured to execute the computer instructions stored in the memory to cause the device to perform the method of any one of the first aspect.

[0018] A fourth aspect provides an agricultural unmanned aerial vehicle, which includes the binocular vision system of any one of the second aspect.

[0019] A fifth aspect provides a computer program product, which includes computer program codes configured to cause a computer to perform the method of any one of the first aspect when the computer program codes are executed on the computer.

[0020] The computer program codes can be stored on a first storage medium in whole or in part, where the first storage medium can be packaged together with the processor or packaged separately from the processor, and the present application does not make a specific limitation in this regard.

[0021] A sixth aspect provides a computer readable medium, which stores program codes configured to cause a computer to perform the method of any one of the first aspect when the computer program codes are executed on the computer.

[0022] In a seventh aspect, there is provided a chip comprising circuitry configured to perform the method of any one of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a schematic block diagram of a binocular vision system provided by an embodiment of the present application.

[0024] Figure 2 is a schematic flow chart of a calibration method of a binocular camera provided by an embodiment of the present application.

[0025] Figure 3 is another schematic flow chart of a calibration method of a binocular camera provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" herein is a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean that there are three cases of A alone, A and B together, and B alone. In the present application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or the like means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can mean a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0027] In the embodiments of the present application, the prefix words such as "first", "second" are only used to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of ordinal words such as ordinal words in the embodiments of the present application does not limit the described objects, and the description of the described objects should be referred to the description of the context in the claims or embodiments, and should not be construed as redundant limitation because of the use of such prefix words.

[0028] The technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0029] As mentioned earlier, errors in binocular vision systems accumulate over time, making the accuracy of calibration parameters crucial for ensuring high-precision navigation. Before leaving the factory, the binocular vision system needs precise calibration to establish a mathematical model between its input and output, compensating for camera scaling factor errors, installation errors, and other factors. However, due to external environmental influences and camera characteristics, the error parameters of the binocular vision system will change after prolonged use. To ensure navigation accuracy, the system needs periodic calibration using calibration references. This increases the complexity of the calibration process, and delays in calibration can lead to a decrease in the system's accuracy.

[0030] This application provides a method, apparatus, and agricultural drone for calibrating binocular cameras, applicable to all binocular system platforms. In the absence of accurate external reference information, feature points between the binocular cameras are calculated using binocular visual information through a traditional feature matching algorithm. Simultaneously, a disparity map is calculated from the binocular visual information using a deep neural network, and feature points are filtered based on the disparity map. N rotation matrices for N scenes are calculated using the filtered feature points, and the number of identity matrices in each of the N rotation matrices determines whether the calibration parameters can continue to be used.

[0031] Figure 1 A schematic block diagram of a binocular vision system 100 provided in an embodiment of this application is shown. Figure 1 As shown, the binocular vision system 100 may include a binocular camera 110 and a processor 120. The binocular camera 110 can send the acquired images to the processor 120. The processor 120 can determine whether the previously calibrated parameters are usable based on the images acquired by the binocular camera. The calibration method in this embodiment can correct small camera deflections caused by drone vibration or installation deviations without relying on calibration reference objects (e.g., calibration boards) or manual intervention.

[0032] The processor 120 in this embodiment can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0033] Figure 2 A schematic flowchart of a binocular camera calibration method 200 provided in an embodiment of this application is shown. Figure 2 As shown, method 200 can be executed by the aforementioned binocular vision system 100, or by the processor 120, or by a drone including the binocular vision system 100; however, the embodiments of this application are not limited thereto. Method 200 includes:

[0034] S210, obtain the calibration parameters of the stereo camera.

[0035] For example, the calibration parameters in S210 can be the initial calibration parameters of the stereo camera, or the calibration parameters of the stereo camera at the time of manufacture.

[0036] S220, according to the calibration parameters of the binocular camera, perform stereo correction on the first image acquired by the binocular camera to obtain the corrected second image. The first image includes images acquired in N scenes, where N is a positive integer.

[0037] Optionally, the number of matching feature points between the left-eye captured image in the current frame and the left-eye captured image in the previous frame can be used to determine whether the left-eye captured image in the current frame and the left-eye captured image in the previous frame belong to the same scene. For example, if the number of matching feature points is less than or equal to a preset number, it can be determined that the left-eye captured image in the current frame and the left-eye captured image in the previous frame do not belong to the same scene; otherwise, the left-eye captured image in the current frame and the left-eye captured image in the previous frame belong to the same scene.

[0038] Optionally, N is a positive integer greater than 1. For example, N is 100.

[0039] The first image above may include images captured by the left eye and images captured by the right eye at multiple moments, or the first image may include multiple frames, each frame including images captured by the left eye and images captured by the right eye.

[0040] S230, Based on the second image, feature points are extracted using machine vision to obtain a first set of feature point pairs.

[0041] For example, the first feature point pair set is obtained by extracting feature points from the second image using machine vision, including: extracting feature points from the second image using the ORB-based BF algorithm.

[0042] Optionally, the process of extracting feature points from the second image using machine vision to obtain a first set of feature point pairs includes: filtering images in the second image where the number of feature point pairs is less than or equal to a second preset number to obtain a filtered image; extracting feature points from the filtered image using machine vision to obtain a third set of feature point pairs; and filtering feature point pairs in the third set of feature point pairs where the disparity value is less than or equal to a preset disparity value to obtain the first set of feature point pairs.

[0043] For example, if the number of feature point pairs extracted from the left-eye and right-eye images of a frame using the BF algorithm is less than or equal to 50, then the left-eye and right-eye images of that frame can be filtered. After filtering out images with fewer than or equal to 50 feature point pairs, the feature point pairs on the remaining images form the aforementioned third set of feature point pairs.

[0044] For example, after determining the third set of feature point pairs, the feature points in the third set can be further filtered. For instance, this third set of feature point pairs includes feature point pair 1, which includes feature point 'a' from a frame captured by the left eye and feature point 'b' from a frame captured by the right eye. If the disparity between these two feature points is less than or equal to 8 pixels, this feature point pair can be filtered out. Since feature point pairs with disparity values ​​less than or equal to 8 pixels are far from the stereo camera, they can be filtered out. After filtering out feature point pairs with disparity values ​​less than or equal to 8 pixels, the first set of feature point pairs is obtained.

[0045] S240, output the second image to a deep neural network to obtain a disparity map, and filter out abnormal feature point pairs in the first feature point pair set on the disparity map to obtain a second feature point pair set.

[0046] Optionally, the first feature point pair set includes a first feature point pair, which includes a first feature point and a second feature point. The first feature point is a feature point on the image acquired by the left eye of the binocular camera, and the second feature point is a feature point on the image acquired by the right eye of the binocular camera. Filtering abnormal feature point pairs in the first feature point pair set on the disparity map to obtain the second feature point pair set includes: predicting a third feature point of the first feature point on the image acquired by the right eye based on the disparity map; and determining the first feature point pair as an abnormal feature point pair when the depth value error between the second feature point and the third feature point is greater than or equal to a preset error.

[0047] The deep neural network described above can be an AANet neural network. Alternatively, it can be a residual neural network (ResNet) or a recurrent neural network (RNN).

[0048] The disparity map detection process involves inputting the left and right target images into a deep neural network to obtain a binocular disparity map. Abnormal feature point pairs in the first feature point pair set are filtered out according to filtering conditions to obtain the second feature point pair.

[0049] For example, using the machine vision matching algorithm in S230 above, feature point pair 2 can be obtained. This feature point pair 2 includes feature point c (located on a left-eye captured image in a certain frame) and feature point d (located on a right-eye captured image in a certain frame). The left-eye captured image and the right-eye captured image in this frame can be input into a deep neural network to predict the position of each feature point on the left-eye captured image in the right-eye captured image. For example, the deep neural network can output the position of feature point c in the right-eye captured image as position 1 (corresponding to feature point e). When the difference in depth values ​​between feature point d and feature point e is less than or equal to 10, feature point pair 2 can be considered a feature point pair that can be used to determine camera extrinsic parameters; otherwise, feature point pair 2 needs to be filtered out. In this way, feature point pairs in the first feature point pair set can be further filtered to obtain the second feature point pair set.

[0050] S250, based on the feature point pairs in the second feature point pair set, determine the N rotation matrices in the N scenes.

[0051] Optionally, before determining the first rotation matrix based on the feature point pairs in the second feature point pair set, the method 200 further includes: determining that the number of feature point pairs in the second feature point pair set is greater than or equal to a third preset number.

[0052] For example, the third preset quantity can be 10 pairs.

[0053] After filtering the feature point pairs, the essential matrix E (or fundamental matrix E) can be determined based on the feature point pairs in the second feature point pair set. Then, based on the intrinsic parameters in the calibration parameters of the binocular camera, the first rotation matrix R1 and the first translation matrix T1 can be determined.

[0054] For example, the rotation matrix R and translation matrix T for each scenario can be determined by the following steps (1)-(3).

[0055] (1) Based on the epipolar geometry constraints:

[0056]

[0057] Where x1 and x2 are the coordinates of the two matched feature points on the normalized plane, T is the translation matrix between the two frames, R is the rotation matrix between the two frames, and E is the essential matrix.

[0058] (2) Use the eight-point method to obtain the essential matrix E. Let the normalized coordinates of a pair of matched feature points be:

[0059] x1=[u1, v1, 1], x2=[u2, v2, 1]

[0060] Based on the epipolar geometry constraints, we can obtain:

[0061]

[0062] The equation is transformed into linear form as follows:

[0063]

[0064] in This represents the normalized coordinates of the j-th matching point in the i-th pair of matched feature points.

[0065] By substituting the coordinates of the 8 pairs of matching points, the essential matrix E can be solved.

[0066] (3) Solve for the rotation matrix R and translation matrix T by performing singular value decomposition (SVD) on the essential matrix. Let the SVD decomposition of the essential matrix E be:

[0067]

[0068] There are two possible rotations and translations corresponding to it:

[0069]

[0070]

[0071] Based on prior knowledge, the rotation matrix R and translation matrix T can be determined for each scenario.

[0072] S260, when M of the N rotation matrices are identity matrices and the ratio of M to N is greater than or equal to a preset threshold, it is determined that the calibration parameter can continue to be used.

[0073] For example, when N is 100 and M is greater than or equal to 50, it can be determined that the calibration parameter can continue to be used.

[0074] S261, if M of the N rotation matrices are identity matrices and the ratio of M to N is less than the preset threshold, then the calibration parameter is determined to be unusable.

[0075] For example, when N is 100 and M is less than 50, it can be determined that the calibration parameter can no longer be used.

[0076] The above statement that calibration parameters are unavailable can be understood as indicating that either the intrinsic or extrinsic parameters in the calibration parameters are unavailable.

[0077] Figure 3 A schematic flowchart of a method 300 for checking calibration parameters provided in an embodiment of this application is shown. Figure 3 As shown, method 300 can be executed by the aforementioned binocular vision system 100, or by the processor 120, or by a drone including the binocular vision system 100; however, the embodiments of this application are not limited thereto. Method 300 includes:

[0078] S301, Obtain the calibration parameter file, which includes calibration parameters.

[0079] For example, calibration parameter 1 includes intrinsic and extrinsic parameters of the binocular camera.

[0080] S302, acquire images captured by the stereo camera.

[0081] For example, the image could be a frame captured by a stereo camera.

[0082] For example, the image includes an image captured by the left eye and an image captured by the right eye.

[0083] S303 performs stereo correction on the images captured by the binocular camera.

[0084] S304: Feature point pairs are extracted from the corrected image using the ORB algorithm.

[0085] S305, the corrected image is input into the deep neural network to obtain the disparity map.

[0086] There is no actual order between S304 and S305 above.

[0087] S306, Based on the disparity map, filter the feature point pairs extracted by the ORB algorithm, and calculate the rotation vector based on the filtered feature point pairs.

[0088] The processes S303-S306 above can be referred to as the processes S230 and S250 above, and will not be repeated here.

[0089] S307, determine whether the image obtained in S302 is an image from a new scene.

[0090] For example, if the number of matching feature points on the image captured by the left eye in the current frame and the number of matching feature points on the image captured by the left eye in the previous frame is less than a preset number, the image captured in the current frame is determined to be an image from a new scene.

[0091] For example, if the image in S302 is a new scene image, then continue to execute S308; otherwise, return to continue executing S302.

[0092] S308 determines whether the calculation of the rotation matrix for N scenarios has been completed, where N is a positive integer.

[0093] Optionally, N is a positive integer greater than 1.

[0094] If the rotation matrix calculations for N scenarios have been completed, proceed to step S309; ​​otherwise, return to step S302.

[0095] S309, determine the number M of identity matrices among N rotation matrices, where M is an integer greater than or equal to 0 and less than or equal to N.

[0096] The above determination of the number M of identity matrices among N rotation matrices can also be understood as determining the number M of rotation matrices that are identity matrices among N rotation matrices.

[0097] S310, determine whether the ratio of M to N is greater than or equal to a preset threshold.

[0098] If the ratio of M to N is greater than or equal to a preset threshold, execute S311; otherwise, execute S312.

[0099] N represents the number of scenes, meaning that the stereo camera needs to acquire N images of different scenes and calculate the rotation matrix based on the image of each scene to obtain N rotation matrices. Among these N rotation matrices, M are identity matrices, where N is a positive integer, M is an integer greater than or equal to 0, and M is less than or equal to N.

[0100] S311, confirm that the calibration parameter can continue to be used.

[0101] S312, confirming that this calibration parameter can no longer be used.

[0102] Optionally, when it is determined that the calibration parameters can no longer be used, the binocular vision system can also send an instruction message to the ground station through the communication module of the agricultural drone. This instruction message indicates that the calibration of the binocular camera can no longer be used. When the user sees the information that the calibration parameters are unusable through the ground station, the user can promptly recalibrate the calibration parameters of the binocular camera.

[0103] This application embodiment also provides a binocular vision system, which includes: a binocular camera 110 for acquiring a first image and sending the first image to a processor 120, the first image including images acquired in N scenes, where N is a positive integer; the processor 120 for acquiring calibration parameters of the binocular camera; the processor 120 for performing stereo correction on the first image based on the calibration parameters to obtain a corrected second image; the processor 120 for performing machine vision feature point extraction on the second image to obtain a first set of feature point pairs; and the processor 120 for inputting the second image into... The deep neural network obtains a disparity map and filters out abnormal feature point pairs in the first feature point pair set on the disparity map to obtain a second feature point pair set. The processor 120 is further configured to determine N rotation matrices in the N scenes based on the feature point pairs in the second feature point pair set. The processor 120 is further configured to determine that the calibration parameter can continue to be used when M rotation matrices in the N rotation matrices are identity matrices and the ratio of M to N is greater than or equal to a preset threshold; or, when M rotation matrices in the N rotation matrices are identity matrices and the ratio of M to N is less than the preset threshold, determine that the calibration parameter cannot be used.

[0104] Optionally, the processor 120 is configured to: filter images in the second image where the number of feature point pairs is less than or equal to a second preset number, to obtain a filtered image; perform machine vision feature point extraction based on the filtered image to obtain a third feature point pair set; and filter feature point pairs in the third feature point pair set where the disparity value is less than or equal to a preset disparity value to obtain the first feature point pair set.

[0105] Optionally, the first feature point pair set includes a first feature point pair, which includes a first feature point and a second feature point. The first feature point is a feature point on the image acquired by the left eye of the binocular camera, and the second feature point is a feature point on the image acquired by the right eye of the binocular camera. The processor is configured to: predict a third feature point of the first feature point on the image acquired by the right eye based on the disparity map; and determine the first feature point pair as an abnormal feature point pair when the depth value error between the second feature point and the third feature point is greater than or equal to a preset error.

[0106] Optionally, the processor 120 is further configured to: determine, before determining the N rotation matrices in the N scenes based on the feature point pairs in the second feature point pair set, that the number of feature point pairs in the second feature point pair set is greater than or equal to a third preset number.

[0107] This application also provides an agricultural drone that includes the aforementioned binocular vision system.

[0108] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0109] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0110] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0112] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0113] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0114] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0115] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting calibration parameters, characterized in that, include: Obtain the calibration parameters of the stereo camera; Based on the calibration parameters of the binocular camera, stereo correction is performed on the first image acquired by the binocular camera to obtain a corrected second image. The first image includes images acquired in N scenes, where N is a positive integer greater than 1, and the N scenes are different scenes. Based on the second image, feature points are extracted using machine vision to obtain a first set of feature point pairs. The first set of feature point pairs includes a first feature point pair, which includes a first feature point and a second feature point. The first feature point is a feature point on the image captured by the left eye of the binocular camera, and the second feature point is a feature point on the image captured by the right eye of the binocular camera. The second image is input into a deep neural network to obtain a disparity map. Based on the disparity map, the third feature point of the first feature point on the image acquired by the right eye is predicted. When the error of the depth value between the second feature point and the third feature point is greater than or equal to a preset error, the first feature point pair is determined to be an abnormal feature point pair. The abnormal feature point pair is filtered on the disparity map to obtain a set of second feature point pairs. Based on the feature point pairs in the second feature point pair set, determine the N rotation matrices for the N scenarios; When M out of the N rotation matrices are identity matrices and the ratio of M to N is greater than or equal to a preset threshold, the calibration parameters are determined to be usable, where M is an integer greater than or equal to 0 and less than or equal to N; or, If M of the N rotation matrices are identity matrices and the ratio of M to N is less than the preset threshold, then the calibration parameters are determined to be unusable.

2. The method according to claim 1, characterized in that, The step of extracting feature points from the second image using machine vision to obtain a first set of feature point pairs includes: Filter out images in the second image where the number of feature point pairs is less than or equal to a second preset number to obtain the filtered image; Based on the filtered image, feature points are extracted using machine vision to obtain a third set of feature point pairs. Filter the feature point pairs in the third feature point pair set whose disparity value is less than or equal to a preset disparity value to obtain the first feature point pair set.

3. The method according to claim 1 or 2, characterized in that, Before determining the N rotation matrices for the N scenes based on the feature point pairs in the second feature point pair set, the method further includes: The number of feature point pairs in the second feature point pair set is determined to be greater than or equal to a third preset number.

4. A binocular vision system, characterized in that, include: A binocular camera is used to acquire a first image and send the first image to a processor. The first image includes images acquired in N scenes, where N is a positive integer greater than 1, and the N scenes are different scenes. The processor is used to acquire the calibration parameters of the binocular camera; The processor is further configured to perform stereoscopic correction on the first image according to the calibration parameters to obtain a corrected second image; The processor is further configured to extract feature points for machine vision based on the second image to obtain a first set of feature point pairs. The first set of feature point pairs includes a first feature point pair, which includes a first feature point and a second feature point. The first feature point is a feature point on the image captured by the left eye of the binocular camera, and the second feature point is a feature point on the image captured by the right eye of the binocular camera. The processor is further configured to input the second image into a deep neural network to obtain a disparity map, predict a third feature point of the first feature point on the image acquired by the right eye based on the disparity map, and determine the first feature point pair as an abnormal feature point pair when the error of the depth value between the second feature point and the third feature point is greater than or equal to a preset error, and filter the abnormal feature point pair on the disparity map to obtain a set of second feature point pairs. The processor is further configured to determine N rotation matrices in the N scenes based on the feature point pairs in the second feature point pair set; When M out of the N rotation matrices are identity matrices and the ratio of M to N is greater than or equal to a preset threshold, the calibration parameters are determined to be usable, where M is an integer greater than or equal to 0 and less than or equal to N; or, If M of the N rotation matrices are identity matrices and the ratio of M to N is less than the preset threshold, then the calibration parameters are determined to be unusable.

5. The binocular vision system according to claim 4, characterized in that, The processor is used for: Filter out images in the second image where the number of feature point pairs is less than or equal to a second preset number to obtain the filtered image; Based on the filtered image, feature points are extracted using machine vision to obtain a third set of feature point pairs. Filter the feature point pairs in the third feature point pair set whose disparity value is less than or equal to a preset disparity value to obtain the first feature point pair set.

6. The binocular vision system according to claim 4 or 5, characterized in that, The processor is also used for: Before determining the N rotation matrices for the N scenarios based on the feature point pairs in the second feature point pair set, it is determined that the number of feature point pairs in the second feature point pair set is greater than or equal to a third preset number.

7. An agricultural unmanned aerial vehicle, characterized in that, The agricultural drone includes a binocular vision system as described in any one of claims 4 to 6.

Images