Dual-target targeting method, device, equipment and storage medium

Abstract

The present application discloses a binocular calibration method, apparatus, binocular device and readable storage medium, including: obtaining a first original image and a second original image of a first image sensor and a second image sensor, correcting and matching the first original image and the second original image according to preset calibration parameters to obtain a first processed image and a second processed image, preliminarily aligning the first processed image and the second processed image in the polar line direction, extracting and matching feature points of the first processed image and the second processed image respectively to obtain a plurality of matching points, calculating the parallax of the matching points to cluster the matching points according to the parallax to obtain a clustering result, screening the matching points according to the clustering result to obtain target matching points, and calculating the target calibration parameters according to the coordinates and parallax of the target matching points. The present application calculates accurate calibration parameters by obtaining binocular images to realize the calibration of the binocular device, without setting a calibration plate, thereby reducing the calibration cost of the binocular device.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to a binocular positioning method, a binocular positioning device, a binocular device and a computer-readable storage medium. Background Art

[0002] In actual use, the binocular camera imaging system must first be calibrated. However, the purpose of binocular camera calibration is different from that of monocular camera. It is mainly to ensure that the left and right cameras are aligned, thereby providing convenient data support for other subsequent processing steps. However, in related technologies, binocular calibration algorithms require additional auxiliary materials such as chessboards or dot chart calibration boards to detect feature points in the image, so as to obtain accurate calibration parameters, which is very costly. Summary of the invention

[0003] The present application provides a dual-target calibration method, the calibration method comprising:

[0004] Acquire a first original image and a second original image from the first image sensor and the second image sensor;

[0005] Correcting and matching the first original image and the second original image according to preset calibration parameters to obtain a first processed image and a second processed image, so as to preliminarily align the first processed image and the second processed image in the epipolar direction;

[0006] Extracting and matching feature points of the first processed image and the second processed image respectively to obtain a plurality of matching points;

[0007] Calculating the disparity of the matching points to cluster the matching points according to the disparity to obtain a clustering result;

[0008] Screening the matching points according to the clustering results to obtain target matching points; and

[0009] The target calibration parameters are calculated based on the coordinates and disparity of the target matching points.

[0010] The present application also provides a dual-target positioning device, comprising:

[0011] An acquisition module, used for acquiring a first original image and a second original image from a first image sensor and a second image sensor;

[0012] a matching module, configured to perform correction and matching processing on the first original image and the second original image according to preset calibration parameters to obtain a first processed image and a second processed image, so as to preliminarily align the first processed image and the second processed image in an epipolar direction;

[0013] An extraction module, used for extracting and matching feature points of the first processed image and the second processed image respectively to obtain a plurality of matching points;

[0014] a processing module, configured to calculate the disparity of the matching points to cluster the matching points according to the disparity to obtain a clustering result, and to screen the matching points according to the clustering result to obtain a target matching point; and

[0015] The calculation module is used to calculate the target calibration parameters according to the coordinates and disparity of the target matching points.

[0016] The present application also provides a binocular device, including a processor and a memory, wherein the memory stores at least one program, and the program is executed by the processor, and the program includes instructions for executing the calibration method. The calibration method includes: acquiring a first original image and a second original image of a first image sensor and a second image sensor, correcting and matching the first original image and the second original image according to preset calibration parameters to obtain a first processed image and a second processed image, so as to preliminarily align the first processed image and the second processed image in the epipolar direction. Feature points are extracted and matched for the first processed image and the second processed image respectively to obtain a plurality of matching points. The disparity of the matching points is calculated to cluster the matching points according to the disparity of the matching points to obtain a clustering result, the matching points are screened according to the clustering result to obtain a target matching point, and the target calibration parameters are calculated according to the coordinates and disparity of the target matching point.

[0017] The present application provides a non-volatile computer-readable storage medium containing a computer program, and when the computer program is executed by a processor, the processor executes the calibration method. The calibration method includes: acquiring a first original image and a second original image of a first image sensor and a second image sensor, correcting and matching the first original image and the second original image according to preset calibration parameters to obtain a first processed image and a second processed image, preliminarily aligning the first processed image and the second processed image in the polar line direction, and extracting and matching feature points of the first processed image and the second processed image respectively to obtain a plurality of matching points. Calculating the disparity of the matching points to cluster the matching points according to the disparity to obtain a clustering result, screening the matching points according to the clustering result to obtain a target matching point, and calculating the target calibration parameters according to the coordinates and disparity of the target matching point.

[0018] In the binocular positioning method, binocular positioning device, binocular device and computer-readable storage medium of the embodiments of the present application, the first original image and the second original image are subjected to distortion correction and matching processing by preset calibration parameters to obtain the first processed image and the second processed image which are preliminarily aligned in the epipolar direction, and then feature points are extracted from the first processed image and the second processed image respectively, and the feature points are matched to filter out unmatched feature points to obtain mutually matching matching points, and then the disparity of the matching points is clustered to filter out the target matching points according to the clustering results, and finally accurate target calibration parameters are calculated according to the disparity and coordinates of the target matching points, so that the binocular device can be calibrated according to the image, avoiding the use of additional auxiliary materials to realize the calibration of the binocular device and reducing the cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0020] Figure 1 It is a flow chart of a calibration method of certain embodiments of the present application.

[0021] Figure 2 It is a module schematic diagram of a dual-target positioning device according to certain embodiments of the present application.

[0022] Figure 3 It is a module schematic diagram of a binocular device in certain embodiments of the present application.

[0023] Figure 4-5 It is a flowchart of the calibration method of certain embodiments of the present application.

[0024] Figure 6 A schematic diagram of a scenario of a calibration method for certain embodiments of the present application.

[0025] Figure 7 is a disparity histogram of the calibration method of certain embodiments of the present application.

[0026] Figure 8-9 It is a flowchart of the calibration method of certain embodiments of the present application.

[0027] Fig.10 It is a module schematic diagram of a binocular device in certain embodiments of the present application.

[0028] Fig.11 It is a connection diagram of a processor and a computer-readable storage medium in certain embodiments of the present application. DETAILED DESCRIPTION

[0029] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0030] See also Figure 1 , the embodiment of the present application provides a dual-target positioning method, comprising:

[0031] S11, acquiring a first original image and a second original image from a first image sensor and a second image sensor;

[0032] S12, performing correction and matching processing on the first original image and the second original image according to preset calibration parameters to obtain a first processed image and a second processed image, so as to preliminarily align the first processed image and the second processed image in the epipolar direction;

[0033] S13, extracting and matching feature points from the first processed image and the second processed image respectively to obtain a plurality of matching points;

[0034] S14, clustering is performed according to the disparity of the matching points to obtain a clustering result;

[0035] S15, screening the matching points according to the clustering results to obtain target matching points; and

[0036] S16, obtaining target calibration parameters according to the coordinates and disparity of the target matching points.

[0037] Please refer to Figure 2 The embodiment of the present application provides a binocular positioning device 10. The binocular positioning device 10 includes an acquisition module 11, an alignment module 12, a matching module 13, a processing module 14 and a calculation module 15.

[0038] S11 can be implemented by the acquisition module 11, S12 can be implemented by the alignment module 12, S13 can be implemented by the matching module 13, S14 and S15 can be implemented by the processing module 14, and S16 can be implemented by the calculation module.

[0039] The acquisition module 11 may be used to acquire a first original image and a second original image from the first image sensor and the second image sensor.

[0040] The alignment module 12 can be used to correct and match the first original image and the second original image according to preset calibration parameters to obtain a first processed image and a second processed image, so as to preliminarily align the first processed image and the second processed image in the epipolar direction.

[0041] The matching module 13 may be used to extract and match feature points from the first processed image and the second processed image respectively to obtain a plurality of matching points.

[0042] The processing module 14 can be used to calculate the disparity of the matching points to cluster the matching points according to the disparity to obtain a clustering result. The processing module 14 can also be used to filter the matching points according to the clustering result to obtain a target matching point.

[0043] The calculation module 15 can be used to calculate the target calibration parameters according to the coordinates and disparity of the target matching points.

[0044] Please combine Figure 3 An embodiment of the present application provides a binocular device 100, including a processor 20 and multiple image sensors 30, the multiple sensors 30 include a first image sensor 31 and a second image sensor 32, and the above calibration method can be implemented by the processor 20.

[0045] The processor 20 can be used to obtain the first original image and the second original image of the first image sensor 31 and the second image sensor 32; the processor 20 can be used to correct and match the first original image and the second original image according to the preset calibration parameters to obtain the first processed image and the second processed image, so as to preliminarily align the first processed image and the second processed image in the epipolar direction; the processor 20 can also be used to extract and match the feature points of the first processed image and the second processed image respectively to obtain a plurality of matching points. The processor 20 can also be used to calculate the disparity of the matching points to cluster the matching points according to the disparity to obtain the clustering results, and to screen the matching points according to the clustering results to obtain the target matching points. The processor 20 can also be used to calculate the target calibration parameters according to the coordinates and disparity of the target matching points.

[0046] In the calibration method, binocular calibration device 10 and binocular device 100 of these embodiments, the first original image and the second original image acquired by the first image sensor 31 and the second image sensor 32 are corrected and matched according to preset calibration parameters to obtain the first processed image and the second processed image which are preliminarily aligned in the polar line direction, and then feature points are extracted from the first processed image and the second processed image respectively, and the feature points are matched to obtain matching points, and then the parallax of the matching points is clustered to obtain clustering results, so as to screen and remove matching points with abnormal parallax values ​​according to the clustering results to obtain target matching points, and finally calculate accurate target calibration parameters according to the parallax and coordinates of the target matching points. In this way, the binocular device 100 can realize calibration according to the image, avoid the use of additional auxiliary materials for calibration, and reduce costs.

[0047] In some embodiments, the binocular device 100 can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a drone, a robot, a smart wearable device (smart helmet, smart glasses, etc.), a virtual reality device, etc.

[0048] It should be noted that the preset calibration parameters refer to parameters preset on the processor 20 for calibrating the first image sensor 31 and the second image sensor 32. The preset calibration parameters can be pre-set when the binocular device leaves the factory, or can be target calibration parameters generated last time based on the first original image and the second original image. The preset calibration parameters include distortion parameters and internal and external parameter matrices. The distortion parameters refer to the deviation between the actual corresponding pixel position of the object point in the image and the theoretical projection point calculated based on the imaging model during the camera shooting process. The deviation is generally described by radial distortion parameters k1, k2, k3 and tangential distortion parameters p1, p2.

[0049] The intrinsic parameter matrix can be expressed as:

[0050]

[0051] Where f is the focal length of the camera, usually in mm, dx, dy are the pixel sizes, and u0, v0 are the image centers.

[0052] The extrinsic parameter matrix may be a matrix expressing the relative position of the first image sensor and the second image sensor in the binocular device, and may include an offset position, a rotation angle, etc. of one image sensor relative to the other image sensor. The extrinsic parameter matrix may be expressed as:

[0053]

[0054] Among them, R is a 3*3 rotation matrix and T is a 3*1 translation matrix.

[0055] The processor 20 can control the first image sensor 31 and the second image sensor 32 to shoot the same shooting object, which can be any physical object, such as trees, flowers, plants, people, etc., so that the first image sensor 31 generates a first original image and the second image sensor 32 generates a second original image.

[0056] Further, according to the intrinsic parameter matrix, extrinsic parameter rotation matrix, translation matrix and distortion coefficient of the first image sensor 31 and the second image sensor 32, the rotation matrix of the first original image and the second original image is calculated using the stereoRctify function in Open CV, and the mapping matrices in the X direction and the Y direction are calculated using the initUndistortRectifyMap function in OpenCV on the first original image and the second original image according to the intrinsic parameter matrix and the extrinsic parameter rotation matrix, respectively, and the first original image and the second original image are respectively corrected using the remap function in OpenCV according to the obtained mapping matrix. After correction, stereo matching is performed by the rotation matrix of the extrinsic parameter matrix, so that the two images that are actually aligned in non-coplanar rows after eliminating the distortion are corrected to be aligned in coplanar rows (in the direction of the epipolar line). (Coplanar row alignment: when the image planes of the two cameras are on the same plane and the same point is projected onto the image planes of the two cameras, the same row of the two pixel coordinate systems), and then the camera coordinate system is converted into the image pixel coordinate system by the intrinsic parameter matrix, and the first processed image and the second processed image are preliminarily aligned in the direction of the epipolar line.

[0057] The processor 20 also includes a preset feature point algorithm, which refers to an algorithm pre-stored in the binocular positioning device 10 or the binocular device 100 for extracting characteristic information from an image.

[0058] The preset feature point algorithms include but are not limited to feature extraction algorithms such as ORB (Oriented Fast and Rotated Brief), Scale-Invariant Feature Transform (Sift) or Speeded Up Robust Features (Surf).

[0059] Among them, the ORB feature extraction algorithm is used to quickly create feature vectors for key points in the image, and these feature vectors can be used to identify objects in the image. Among them, Fast and Brief are feature detection algorithms and vector creation algorithms respectively. The Surf feature extraction algorithm is a robust local feature point detection and description algorithm. Like the Sift algorithm, the basic process of the Surf algorithm can be divided into three parts: extraction of local feature points, description of feature points, and matching of feature points. Sift is an image local feature description operator based on scale space that remains invariant to image scaling, rotation, and even affine transformation. The Sift feature extraction algorithm has four steps: 1. Extreme value detection in scale space, 2. Feature point positioning, 3. Feature direction assignment, and 4. Feature point description.

[0060] After obtaining the first processed image and the second processed image preliminarily aligned in the polar line direction, the processor 20 may extract feature points from the first processed image and the second processed image according to a preset feature point algorithm to obtain a plurality of first feature points of the first processed image and a plurality of second feature points of the second processed image. Further, the first feature points and the second feature points are matched based on a preset matching algorithm to obtain mutually matching first feature points and second feature points as matching points, and the first feature points and second feature points that are mismatched are filtered out. The matching points of the first processed image and the matching points of the second processed image correspond one to one. The preset matching algorithm may include, but is not limited to, a brute force matching algorithm (BruteForce), a K-nearest neighbor algorithm (KNN) or an exhaustive search algorithm, etc.

[0061] Furthermore, the processor 20 calculates the disparity of the matching points and clusters the disparity of the matching points to obtain a clustering result. Since most of the abnormal values ​​deviate from the mean value, the processor 20 can further screen the corresponding matching points according to the number of disparities of the matching points in the maximum disparity class or the minimum disparity class to further remove the matching points with abnormal disparity values ​​to obtain the target matching points, and finally, calculate the target calibration parameters according to the coordinates and disparity of the target matching points.

[0062] It should be noted that after correction according to the preset calibration parameters, the first processed image and the second processed image have been preliminarily aligned in the epipolar direction. In some embodiments, the processor 20 can narrow the matching range of the feature points during the matching process, that is, limit the matching range of the feature points to a certain area in the first processed image, thereby improving the matching accuracy of the feature points. For example, the matching range of the feature points is 10% of the size of the first processed image. In this way, it is possible to avoid abnormal points with too large matching distances, improve the matching accuracy of the feature points, and at the same time, reduce the amount of calculation and improve efficiency.

[0063] In addition, in some other embodiments, in order to improve the accuracy and efficiency of feature point matching during the feature point matching process, the processor 20 may also use repeated texture detection to detect the repeated texture area of ​​the first processed image and the second processed image, thereby removing the feature points detected in the repeated texture area as abnormal points. In this way, interference calculation can be avoided, thereby further improving the matching accuracy of the feature points.

[0064] Please combine Figure 4 In some embodiments, the calibration method further comprises:

[0065] S17, calibrating the first image sensor and the second image sensor according to the target calibration parameters.

[0066] In some embodiments, the dual-target calibration device further includes a calibration module 16. S16 can be implemented by the calibration module 16. That is, the calibration module 16 can be used to calibrate the first image sensor and the second image sensor according to the target calibration parameters.

[0067] In some embodiments, the processor 20 is further configured to calibrate the first image sensor and the second image sensor according to the target calibration parameters.

[0068] It should be noted that when calibrating the first image sensor 31 and the second image sensor 32 according to the target calibration parameters, the preset calibration parameters in the first image sensor 31 and the second image sensor 32 are actually corrected so that the preset calibration parameters are equal to or approximate to the target calibration parameters, and after the correction, the coordinates of the target matching points in the first processed image and the second processed image generated by the first image sensor 31 and the second image sensor 32 according to the target calibration parameters can be aligned.

[0069] In this way, the binocular device 100 can be accurately calibrated according to the obtained images, which greatly reduces the calibration cost of the binocular device 100.

[0070] Please combine Figure 5 In some embodiments, S14 includes:

[0071] S141: Determine the coordinates of the matching points to calculate the disparity of the matching points;

[0072] S142: Clustering the disparities of the matching points according to a preset clustering algorithm to obtain a clustering result.

[0073] Please read further Figure 2 In some embodiments, S141 and S142 may be implemented by the processing module 14. That is, the processing module 14 is used to determine the coordinates of the matching points to calculate the disparity of the matching points;

[0074] The processing module 14 is used to cluster the disparities of the matching points according to a preset clustering algorithm to obtain a clustering result.

[0075] In some embodiments, the processor 20 is used to determine the coordinates of the matching points to calculate the disparity of the matching points. The processor 20 is also used to cluster the disparity of the matching points according to a preset clustering algorithm to obtain a clustering result.

[0076] It can be understood that in the actual imaging process, the closer the target is, the larger the parallax is, and at a longer distance, the parallax approaches zero, and the parallax values ​​of all targets have the same sign (all non-negative or all non-positive). For example, see Figure 6 , Figure 6is a schematic diagram of a first processed image and a second processed image in the present application, that is, in this embodiment, Figure 6 Here, the main image is the first processed image, the secondary image is the second processed image, the black triangle represents a close target, the gray rectangle represents a medium-distance target, the oblique line area represents a long-distance target, the horizontal direction is the y-axis, that is, the polar line direction, and the vertical direction is the y-axis, that is, the non-polar line direction.

[0077] Specifically, the coordinates (y, x) of each matching point in the first processed image and the second processed image are determined, where y is the value of the matching point in the polar direction (horizontal direction) and x is the value of the matching point in the non-polar direction (vertical direction). The difference between the matching points in the first processed image and the second processed image that match each other in the non-polar direction is calculated, that is, the disparity of the matching points is obtained, and then the disparity of the matching points is clustered based on the preset clustering algorithm to obtain a clustering result.

[0078] The preset clustering algorithm may include a K-means clustering algorithm, which is a partitioning clustering algorithm. Given a data point set and a required number of clusters K, K is specified by a user, the K-means algorithm repeatedly divides the data into K clusters according to a certain distance function. The processor 20 uses the K-means clustering algorithm to quickly and effectively cluster matching points with similar parallax into the same class, and the classification effect is good.

[0079] It is understandable that in the feature point matching process, mismatching is very likely to occur in areas with repeated textures or few textures, and in most cases, the disparity of such abnormal points deviates greatly from the average disparity, which easily affects the accuracy of the target calibration parameter calculation. Therefore, after the processor 20 obtains the clustering result using the K-means clustering algorithm, it screens the matching points according to the clustering result, removes the matching points that deviate greatly from the average disparity, and obtains the target matching points. Then, the target calibration parameters are calculated based on the coordinates and disparity of the target matching points.

[0080] Specifically, (1) clustering is performed using the disparity of matching points as the clustering index of the K-means clustering algorithm. The number of clusters can be 3, 4, 5, 6, 8 or even more. A disparity equal to the number of clusters is randomly selected from the disparities of all matching points and used as the initial cluster center (initial seed). (2) The clustering iteration termination adjustment is set (usually the number of clustering iterations or the change error of the cluster center is set). (3) The disparity is assigned to the class with the closest distance according to the distance criterion. (4) The average vector of each class is used as the cluster center of the next iteration. Steps (3) and (4) are repeated until the iteration termination condition in (2) is met, and the clustering result is obtained. The clustering result can be arranged in order from small to large according to the disparity mean. For example, in this embodiment, the clustering result includes 4 disparity classes. The disparity classes can be divided into 1, 2, 3, and 4 classes according to the disparity mean from small to large. The disparity value of class 1 is the smallest, and the disparity value of class 4 is the largest.

[0081] In addition, please combine Figure 7 In order to avoid excessive iteration of the clustering algorithm resulting in individual abnormal points being classified as normal classes, the processor 20 can also sort the disparities of the matching points by size after calculating the disparities of all matching points, and use the disparities at the positions of 0%, 33%, 66%, and 100% as the initial clustering centers (initial seeds) of the four classes from small to large, and strictly control the number of clustering iterations.

[0082] See also Figure 8 In some embodiments, S15 includes:

[0083] S151: respectively obtaining the number of first matching points of the maximum disparity class and the number of second matching points of the minimum disparity class in the clustering result, where the matching points include the first matching points and the second matching points;

[0084] S152: if the ratio of the number of the first matching points and the second matching points to the total number of disparities of the matching points is less than a first preset threshold, remove the first matching points and the second matching points to obtain target matching points;

[0085] S153: If the mean deviation of the first matching point and the second matching point from the disparity mean is greater than a second preset threshold, the first matching point and the second matching point are eliminated to obtain a target matching point.

[0086] In some embodiments, S151 , S152 , and S153 may be implemented by the processing module 14 .

[0087] The processing module 14 may be used to respectively obtain the number of first matching points of the maximum disparity class and the number of second matching points of the minimum disparity class in the clustering result, where the matching points include the first matching points and the second matching points.

[0088] The processing module 14 may be configured to remove the first matching point and the second matching point to obtain the target matching point if the ratio of the number of the first matching point and the number of the second matching point to the total number of disparities of the matching points is less than a first preset threshold;

[0089] The processing module 14 may also be configured to remove the first matching point and the second matching point to obtain a target matching point if the disparity mean of the first matching point and the second matching point deviates from the disparity mean of the matching point by more than a second preset threshold.

[0090] In some embodiments, the processor 20 can be used to respectively obtain the number of first matching points of the maximum disparity class and the number of second matching points of the minimum disparity class in the clustering result, where the matching points include the first matching points and the second matching points. The processor 20 can be used to remove the first matching points and the second matching points to obtain the target matching points if the ratio of the number of the first matching points and the number of the second matching points to the total number of disparities of the matching points is less than a first preset threshold. The processor 20 can also be used to remove the first matching points and the second matching points to obtain the target matching points if the mean deviation of the first matching points and the second matching points from the disparity mean is greater than a second preset threshold.

[0091] There is no restriction on the specific values ​​of the first preset threshold and the second preset threshold. For example, in the present application, the first preset threshold may be 6% and the second preset threshold may be 100. That is, when the processor 20 calculates that the sum of the first matching point and the second matching point in the maximum class disparity and the minimum disparity class is less than 6%, the first matching point and the second matching point are removed to generate the target matching point. Alternatively, when the processor 20 calculates that the mean deviation of the first matching point and the second matching point from the disparity mean is greater than 100, the first matching point and the second matching point in the matching point are removed to generate the target matching point. In this way, the processor 20 calculates the target calibration parameters according to the target matching points, which can further improve the accuracy of the target calibration parameters.

[0092] In addition, if the ratio of the number of the first matching points and the number of the second matching points to the total number of disparities of the matching points is greater than or equal to a first preset threshold, all the matching points are taken as target matching points. Alternatively, if the mean deviation of the first matching point and the second matching point from the mean disparity is less than or equal to a second preset threshold, all the matching points are taken as target matching points.

[0093] Please combine Fig. 9 In some embodiments, S16 includes:

[0094] S161: Determine a normal parallax range according to the clustering result;

[0095] S162: When the disparity of the first matching point does not fall within the normal disparity range, the disparity of the first matching point is set to zero to calculate the target calibration parameter.

[0096] In some embodiments, S161 and S162 may be implemented by the computing module 15 .

[0097] The calculation module 15 may be used to determine a normal disparity range according to the clustering result.

[0098] The calculation module 15 may also be configured to set the disparity of the first matching point to zero to calculate the target calibration parameter when the disparity of the first matching point does not fall within the normal disparity range.

[0099] In some embodiments, the processor 20 may be configured to determine a normal disparity range according to the clustering result. The processor 20 may also be configured to set the first matching point to zero to calculate the target calibration parameter when the first matching point does not belong to the normal disparity range.

[0100] It can be understood that since the target calibration parameters are calculated based on the coordinates and disparity of the target matching points, the disparity range of the target matching points may lead to calculation deviations. Therefore, in this embodiment, the disparity of the target matching points is also constrained, thereby solving the calculation deviation caused by each disparity constraint of the optimization result and effectively improving the accuracy of the target calibration algorithm.

[0101] For example, in Figure 7 In the case of , most of the disparities of the matching points are positive, while in the minimum class disparity, the disparity of the first matching point is negative. Then the disparity position of the first matching point can be set to zero, and then the target calibration parameters can be calculated based on the disparities of all matching points.

[0102] In addition, it can be understood that if the disparity of the second matching point of the maximum disparity class does not fall within the normal disparity range, for example, the normal disparity range is negative and the disparity of the second matching point of the maximum disparity class is positive, the disparity of the second matching point can be reset to zero.

[0103] The calculation formula of the target calibration parameters is:

[0104]

[0105] The horizontal coordinate in the image, m is the number of the first matching points with negative parallax; l q y is the ordinate of the matching point q in the main image, r q y is the ordinate of the matching point q in the secondary image, n is the total number of matching points, abs refers to the absolute value, and the M matrix is ​​the positional relationship matrix of the pixel coordinates in binocular imaging, which is related to the external parameters. The overall optimization goal of the calculation formula is to approach zero. The first half indicates that the parallax of the matching point is zero at a very far distance (x-axis), and the second half indicates that the matching points are aligned in the polar direction (y-axis). The processor 20 can use an optimization method such as the least squares method to solve this calculation formula and obtain the target calibration parameters.

[0106] See also Fig.10 The present application provides a binocular device 100, including a processor 20 and a memory 40; wherein the memory 40 stores one or more programs 42 and is executed by the processor 20, and the program 42 is executed by the processor 20 to execute instructions of any of the above-mentioned calibration methods.

[0107] Please combine Fig.11 The present application provides a non-volatile computer-readable storage medium 50 containing a computer program. When the computer program 50 is executed by the processor 20, the processor 20 executes the above-mentioned dual-target positioning method.

[0108] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A binocular positioning method, characterized in that: The calibration method comprises: Acquire a first original image and a second original image from the first image sensor and the second image sensor; Correcting and matching the first original image and the second original image according to preset calibration parameters to obtain a first processed image and a second processed image, so as to preliminarily align the first processed image and the second processed image in the epipolar direction; Extracting and matching feature points from the first processed image and the second processed image respectively to obtain a plurality of matching points; determining coordinates of the matching points to calculate the disparity of the matching points; Clustering the disparities of the matching points according to a preset clustering algorithm to cluster matching points with similar disparities into the same category and obtain a clustering result; The matching points are screened according to the clustering result to obtain target matching points, so as to remove matching points with abnormal disparity values; The target calibration parameters are calculated based on the coordinates and disparity of the target matching points.

2. The calibration method according to claim 1, characterized in that: The calibration method further includes: calibrating the first image sensor and the second image sensor according to the target calibration parameters.

3. The calibration method according to claim 1, characterized in that: The clustering of the disparities of the matching points according to a preset clustering algorithm to obtain a clustering result includes: The disparities of the matching points are clustered according to a K-means clustering algorithm to obtain the clustering result.

4. The calibration method according to claim 1, characterized in that: The screening of the matching points according to the clustering results to obtain target matching points includes: Respectively obtaining the number of first matching points of the maximum disparity class and the number of second matching points of the minimum disparity class in the clustering result, wherein the matching points include the first matching points and the second matching points; If the ratio of the number of the first matching points and the number of the second matching points to the total number of disparities of the matching points is less than a first preset threshold, the first matching points and the second matching points are eliminated to obtain the target matching points; or If the disparity mean of the first matching point and the second matching point deviates from the disparity mean of the matching point by more than a second preset threshold, the first matching point and the second matching point are eliminated to obtain the target matching point.

5. The calibration method according to claim 1, characterized in that: Calculating the target calibration parameters according to the coordinates and disparity of the target matching points includes: Determine the normal disparity range according to the clustering results; When the disparity of the first matching point of the minimum disparity class does not belong to the normal disparity range, the disparity of the first matching point is set to zero to calculate the target calibration parameter.

6. The calibration method according to claim 1, characterized in that: The extracting feature points from the first processed image and the second processed image to obtain a plurality of matching points comprises: Obtaining first feature points of the first processed image and second feature points of the second processed image by using an ORB or SIFT feature detection algorithm; The first feature point and the second feature point are matched according to a preset matching algorithm to obtain a plurality of matching points.

7. A binocular positioning device, characterized in that: The dual-target positioning device comprises: An acquisition module, used for acquiring a first original image and a second original image from a first image sensor and a second image sensor; a matching module, configured to perform correction and matching processing on the first original image and the second original image according to preset calibration parameters to obtain a first processed image and a second processed image, so as to preliminarily align the first processed image and the second processed image in an epipolar direction; An extraction module, used for extracting and matching feature points of the first processed image and the second processed image respectively to obtain a plurality of matching points; A processing module, used to determine the coordinates of the matching points to calculate the disparity of the matching points; cluster the disparities of the matching points according to a preset clustering algorithm to cluster matching points with similar disparities into the same class and obtain clustering results; and used to screen the matching points according to the clustering results to obtain target matching points to remove matching points with abnormal disparity values; The calculation module is used to calculate the target calibration parameters according to the coordinates and disparity of the target matching points.

8. A binocular device, characterized in that: The system comprises a processor and a memory, wherein the memory stores at least one program and is executed by the processor, wherein the program comprises instructions for executing the dual-target positioning method according to any one of claims 1 to 6.

9. A non-volatile computer-readable storage medium containing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the binary positioning method according to any one of claims 1 to 6.

Images